Skip to content

Fix eigen/eigvals rules for Hermitian StridedMatrix - #853

Open
devmotion wants to merge 6 commits into
mainfrom
dmw/eigen-complex-hermitian
Open

devmotion wants to merge 6 commits into
mainfrom
dmw/eigen-complex-hermitian

Conversation

@devmotion

Copy link
Copy Markdown
Member

At a Hermitian A, the eigen/eigvals frules and rrules for StridedMatrix delegated to the rules for Hermitian(A). Their (co)tangents are those of x -> eigen(Hermitian(x)), which reads only the upper triangle of x. That is wrong for unstructured matrices: the rrules returned upper-triangular cotangents and the frules ignored the lower triangle of ΔA (FluxML/Zygote.jl#1369).

This PR:

  • Eigenvalues: tangents uᵢ' ΔA uᵢ and cotangents U Diagonal(Δλ) U', with the type of the primal eigenvalues (so they are only projected onto the reals when the primal eigenvalues are real-typed).
  • Eigenvectors: at Hermitian A, eigen uses the Hermitian solver, whose eigenvectors follow a different phase convention than those of non-Hermitian matrices. They are therefore only differentiable along Hermitian directions, so the rules use the Hermitian part of the eigenvector (co)tangents (and the Hermitian phase convention).
  • Both cases now share the code path of the generic rules (V' instead of V \ since V is unitary); _sorteig!_fwd became unused and is removed.
  • Fixes the failures on Julia 1.14 (pre): LinearAlgebra.jl#1717 made eigenvalues of complex matrices complex-typed also on the Hermitian path, so the old rules returned real tangents for complex primals. It also exposed a Core.Box in the eigen pullback (self-recursive thunk method), which made the eigvals rrule non-inferable; the pullback is now a single method.
  • Tests compare against finite differences of the unwrapped eigvals (whenever the eigenvalue type admits all directions) and of eigen ∘ hermitianpart for eigenvectors, with non-Hermitian ΔA, instead of x -> eigen(Matrix(Hermitian(x))). Includes a regression test for Zygote#1369 (∇ sum(eigvals(A)) == I).

Supersedes #841 (thanks @CarloLucibello for the analysis there), which fixed the values-only rrule for real matrices.

Verified locally: rulesets/LinearAlgebra/factorization passes on Julia 1.10, 1.13 and 1.14.0-alpha1 (the new tests fail against the released version).

🤖 Generated with Claude Code

At Hermitian `A`, the `eigen`/`eigvals` rules for `StridedMatrix` delegated
to the rules for `Hermitian(A)`. The resulting (co)tangents were those of
`x -> eigen(Hermitian(x))`, which only reads the upper triangle, so they
were wrong for unstructured matrices (FluxML/Zygote.jl#1369).

Now the eigenvalue (co)tangents are `uᵢ' ΔA uᵢ` and `U Diagonal(Δλ) U'`, with
the type of the primal eigenvalues. The eigenvectors are only differentiable
along Hermitian directions (phase convention of the Hermitian solver), so
their (co)tangents use the Hermitian part.

This also fixes the failures on Julia 1.14, where LinearAlgebra.jl#1717 made
eigenvalues of complex matrices complex also for Hermitian matrices, and
removes a `Core.Box` in the `eigen` pullback that broke inference.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.64%. Comparing base (e6e8fae) to head (ef12a50).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #853      +/-   ##
==========================================
+ Coverage   96.57%   96.64%   +0.06%     
==========================================
  Files          28       28              
  Lines        3618     3604      -14     
==========================================
- Hits         3494     3483      -11     
+ Misses        124      121       -3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

devmotion and others added 2 commits October 23, 2026 15:43
Reuse the cotangent buffer for the product with the eigenvectors in the
`eigen` pullback for Hermitian matrices.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Comment thread src/rulesets/LinearAlgebra/factorization.jl Outdated

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant