Skip to content

Zygote gradient fails on LinearSolve for more than 10 x 10 matrix #601

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
mchitre opened this issue Apr 28, 2025 · 0 comments · Fixed by #603
Closed

Zygote gradient fails on LinearSolve for more than 10 x 10 matrix #601

mchitre opened this issue Apr 28, 2025 · 0 comments · Fixed by #603
Labels
bug Something isn't working

Comments

@mchitre
Copy link

mchitre commented Apr 28, 2025

Describe the bug 🐞

This works:

using LinearSolve, Zygote, SciMLSensitivity

const N = 10
f(b) = sum(solve(LinearProblem(randn(N, N), b)).u)
Zygote.gradient(f, randn(N))

but it fails for N > 10 with:

ERROR: MethodError: no method matching getrs!(::Char, ::Matrix{Float64}, ::Vector{Int32}, ::Vector{Float64})
The function `getrs!` exists, but no method is defined for this combination of argument types.

Expected behavior

It should work!

Minimal Reproducible Example 👇

using LinearSolve, Zygote, SciMLSensitivity

const N = 11
f(b) = sum(solve(LinearProblem(randn(N, N), b)).u)
Zygote.gradient(f, randn(N))

Error & Stacktrace ⚠️

ERROR: MethodError: no method matching getrs!(::Char, ::Matrix{Float64}, ::Vector{Int32}, ::Vector{Float64})
The function `getrs!` exists, but no method is defined for this combination of argument types.

Closest candidates are:
  getrs!(::AbstractChar, ::AbstractMatrix{Float64}, ::AbstractVector{Int64}, ::AbstractVecOrMat{Float64})
   @ LinearAlgebra ~/.julia/juliaup/julia-1.11.5+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/LinearAlgebra/src/lapack.jl:1043
  getrs!(::AbstractChar, ::AbstractMatrix{ComplexF32}, ::AbstractVector{Int64}, ::AbstractVecOrMat{ComplexF32})
   @ LinearAlgebra ~/.julia/juliaup/julia-1.11.5+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/LinearAlgebra/src/lapack.jl:1043
  getrs!(::AbstractChar, ::AbstractMatrix{ComplexF64}, ::AbstractVector{Int64}, ::AbstractVecOrMat{ComplexF64})
   @ LinearAlgebra ~/.julia/juliaup/julia-1.11.5+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/LinearAlgebra/src/lapack.jl:1043
  ...

Stacktrace:
  [1] ldiv!(transA::LinearAlgebra.TransposeFactorization{Float64, LU{Float64, Matrix{Float64}, Vector{Int32}}}, B::Vector{Float64})
    @ LinearAlgebra ~/.julia/juliaup/julia-1.11.5+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/LinearAlgebra/src/lu.jl:507
  [2] ldiv(F::LinearAlgebra.TransposeFactorization{Float64, LU{Float64, Matrix{Float64}, Vector{Int32}}}, B::Vector{Float64})
    @ LinearAlgebra ~/.julia/juliaup/julia-1.11.5+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/LinearAlgebra/src/LinearAlgebra.jl:656
  [3] \(F::LinearAlgebra.TransposeFactorization{Float64, LU{Float64, Matrix{Float64}, Vector{Int32}}}, B::Vector{Float64})
    @ LinearAlgebra ~/.julia/juliaup/julia-1.11.5+0.aarch64.apple.darwin14/share/julia/stdlib/v1.11/LinearAlgebra/src/LinearAlgebra.jl:633
  [4] macro expansion
    @ ~/.julia/packages/LinearSolve/O6GSm/src/default.jl:371 [inlined]
  [5] defaultalg_adjoint_eval(cache::LinearSolve.LinearCache{Matrix{…}, Vector{…}, Vector{…}, SciMLBase.NullParameters, LinearSolve.DefaultLinearSolver, LinearSolve.DefaultLinearSolverInit{…}, IdentityOperator, IdentityOperator, Float64, Bool, LinearSolveAdjoint{…}}, dy::Vector{Float64})
    @ LinearSolve ~/.julia/packages/LinearSolve/O6GSm/src/default.jl:364
  [6] (::LinearSolve.var"#∇linear_solve#107"{LinearSolve.DefaultLinearSolver, Tuple{}, SciMLBase.LinearSolution{…}, LinearSolveAdjoint{…}, LinearSolve.LinearCache{…}})(∂sol::SciMLBase.LinearSolution{Float64, 1, Vector{…}, Nothing, LinearSolve.DefaultLinearSolver, LinearSolve.LinearCache{…}, Nothing})
    @ LinearSolve ~/.julia/packages/LinearSolve/O6GSm/src/adjoint.jl:68
  [7] ZBack
    @ ~/.julia/packages/Zygote/HdT4O/src/compiler/chainrules.jl:222 [inlined]
  [8] (::Zygote.var"#305#306"{Tuple{…}, Zygote.ZBack{…}})(Δ::SciMLBase.LinearSolution{Float64, 1, Vector{…}, Nothing, LinearSolve.DefaultLinearSolver, LinearSolve.LinearCache{…}, Nothing})
    @ Zygote ~/.julia/packages/Zygote/HdT4O/src/lib/lib.jl:202
  [9] (::Zygote.var"#2189#back#307"{Zygote.var"#305#306"{…}})(Δ::SciMLBase.LinearSolution{Float64, 1, Vector{…}, Nothing, LinearSolve.DefaultLinearSolver, LinearSolve.LinearCache{…}, Nothing})
    @ Zygote ~/.julia/packages/ZygoteRules/CkVIK/src/adjoint.jl:72
 [10] #solve#6
    @ ~/.julia/packages/LinearSolve/O6GSm/src/common.jl:298 [inlined]
 [11] (::Zygote.Pullback{Tuple{…}, Any})(Δ::SciMLBase.LinearSolution{Float64, 1, Vector{…}, Nothing, LinearSolve.DefaultLinearSolver, LinearSolve.LinearCache{…}, Nothing})
    @ Zygote ~/.julia/packages/Zygote/HdT4O/src/compiler/interface2.jl:0
 [12] #305
    @ ~/.julia/packages/Zygote/HdT4O/src/lib/lib.jl:202 [inlined]
 [13] #2189#back
    @ ~/.julia/packages/ZygoteRules/CkVIK/src/adjoint.jl:72 [inlined]
 [14] solve
    @ ~/.julia/packages/LinearSolve/O6GSm/src/common.jl:296 [inlined]
 [15] (::Zygote.Pullback{Tuple{…}, Tuple{…}})(Δ::SciMLBase.LinearSolution{Float64, 1, Vector{…}, Nothing, LinearSolve.DefaultLinearSolver, LinearSolve.LinearCache{…}, Nothing})
    @ Zygote ~/.julia/packages/Zygote/HdT4O/src/compiler/interface2.jl:0
 [16] #305
    @ ~/.julia/packages/Zygote/HdT4O/src/lib/lib.jl:202 [inlined]
 [17] (::Zygote.var"#2189#back#307"{Zygote.var"#305#306"{…}})(Δ::SciMLBase.LinearSolution{Float64, 1, Vector{…}, Nothing, LinearSolve.DefaultLinearSolver, LinearSolve.LinearCache{…}, Nothing})
    @ Zygote ~/.julia/packages/ZygoteRules/CkVIK/src/adjoint.jl:72
 [18] #solve#5
    @ ~/.julia/packages/LinearSolve/O6GSm/src/common.jl:293 [inlined]
 [19] (::Zygote.Pullback{Tuple{…}, Any})(Δ::SciMLBase.LinearSolution{Float64, 1, Vector{…}, Nothing, LinearSolve.DefaultLinearSolver, LinearSolve.LinearCache{…}, Nothing})
    @ Zygote ~/.julia/packages/Zygote/HdT4O/src/compiler/interface2.jl:0
 [20] #305
    @ ~/.julia/packages/Zygote/HdT4O/src/lib/lib.jl:202 [inlined]
 [21] #2189#back
    @ ~/.julia/packages/ZygoteRules/CkVIK/src/adjoint.jl:72 [inlined]
 [22] solve
    @ ~/.julia/packages/LinearSolve/O6GSm/src/common.jl:292 [inlined]
 [23] (::Zygote.Pullback{Tuple{…}, Tuple{…}})(Δ::SciMLBase.LinearSolution{Float64, 1, Vector{…}, Nothing, LinearSolve.DefaultLinearSolver, LinearSolve.LinearCache{…}, Nothing})
    @ Zygote ~/.julia/packages/Zygote/HdT4O/src/compiler/interface2.jl:0
 [24] f7
    @ ./REPL[49]:1 [inlined]
 [25] (::Zygote.Pullback{Tuple{typeof(f7), Vector{…}}, Tuple{Zygote.Pullback{…}, Zygote.ZBack{…}, Zygote.var"#2006#back#208"{…}, Zygote.var"#2006#back#208"{…}, Zygote.ZBack{…}, SciMLBaseZygoteExt.var"#239#back#127"{…}, Zygote.Pullback{…}}})(Δ::Float64)
    @ Zygote ~/.julia/packages/Zygote/HdT4O/src/compiler/interface2.jl:0
 [26] (::Zygote.var"#88#89"{Zygote.Pullback{Tuple{typeof(f7), Vector{…}}, Tuple{Zygote.Pullback{…}, Zygote.ZBack{…}, Zygote.var"#2006#back#208"{…}, Zygote.var"#2006#back#208"{…}, Zygote.ZBack{…}, SciMLBaseZygoteExt.var"#239#back#127"{…}, Zygote.Pullback{…}}}})(Δ::Float64)
    @ Zygote ~/.julia/packages/Zygote/HdT4O/src/compiler/interface.jl:97
 [27] gradient(f::Function, args::Vector{Float64})
    @ Zygote ~/.julia/packages/Zygote/HdT4O/src/compiler/interface.jl:154
 [28] top-level scope
    @ REPL[50]:1

Environment (please complete the following information):

  • Output of using Pkg; Pkg.status()
  [7ed4a6bd] LinearSolve v3.9.0
  [1ed8b502] SciMLSensitivity v7.78.0
  [90137ffa] StaticArrays v1.9.13
  [e88e6eb3] Zygote v0.7.6
  • Output of using Pkg; Pkg.status(; mode = PKGMODE_MANIFEST)
  [47edcb42] ADTypes v1.14.0
  [621f4979] AbstractFFTs v1.5.0
  [7d9f7c33] Accessors v0.1.42
  [79e6a3ab] Adapt v4.3.0
  [66dad0bd] AliasTables v1.1.3
  [4fba245c] ArrayInterface v7.18.0
  [4c555306] ArrayLayouts v1.11.1
  [a9b6321e] Atomix v1.1.1
  [62783981] BitTwiddlingConvenienceFunctions v0.1.6
  [fa961155] CEnum v0.5.0
  [2a0fbf3d] CPUSummary v0.2.6
  [7057c7e9] Cassette v0.3.14
  [082447d4] ChainRules v1.72.3
  [d360d2e6] ChainRulesCore v1.25.1
  [fb6a15b2] CloseOpenIntervals v0.1.13
  [38540f10] CommonSolve v0.2.4
  [bbf7d656] CommonSubexpressions v0.3.1
  [f70d9fcc] CommonWorldInvalidations v1.0.0
  [34da2185] Compat v4.16.0
  [a33af91c] CompositionsBase v0.1.2
  [2569d6c7] ConcreteStructs v0.2.3
  [187b0558] ConstructionBase v1.5.8
  [adafc99b] CpuId v0.3.1
  [a8cc5b0e] Crayons v4.1.1
  [9a962f9c] DataAPI v1.16.0
  [864edb3b] DataStructures v0.18.22
  [e2d170a0] DataValueInterfaces v1.0.0
  [2b5f629d] DiffEqBase v6.170.1
  [459566f4] DiffEqCallbacks v4.4.1
  [77a26b50] DiffEqNoiseProcess v5.24.1
  [163ba53b] DiffResults v1.1.0
  [b552c78f] DiffRules v1.15.1
  [a0c0ee7d] DifferentiationInterface v0.6.52
  [31c24e10] Distributions v0.25.119
  [ffbed154] DocStringExtensions v0.9.4
  [4e289a0a] EnumX v1.0.5
  [7da242da] Enzyme v0.13.38
  [f151be2c] EnzymeCore v0.8.8
  [e2ba6199] ExprTools v0.1.10
  [55351af7] ExproniconLite v0.10.14
  [7034ab61] FastBroadcast v0.3.5
  [9aa1b823] FastClosures v0.3.2
  [a4df4552] FastPower v1.1.2
  [1a297f60] FillArrays v1.13.0
  [6a86dc24] FiniteDiff v2.27.0
⌅ [f6369f11] ForwardDiff v0.10.38
  [f62d2435] FunctionProperties v0.1.2
  [069b7b12] FunctionWrappers v1.1.3
  [77dc65aa] FunctionWrappersWrappers v0.1.3
  [d9f16b24] Functors v0.5.2
  [46192b85] GPUArraysCore v0.2.0
  [61eb1bfa] GPUCompiler v1.4.0
  [076d061b] HashArrayMappedTries v0.2.0
  [34004b35] HypergeometricFunctions v0.3.28
  [7869d1d1] IRTools v0.4.14
  [615f187c] IfElse v0.1.1
  [3587e190] InverseFunctions v0.1.17
  [92d709cd] IrrationalConstants v0.2.4
  [82899510] IteratorInterfaceExtensions v1.0.0
  [692b3bcd] JLLWrappers v1.7.0
  [ae98c720] Jieko v0.2.1
  [63c18a36] KernelAbstractions v0.9.34
  [ba0b0d4f] Krylov v0.10.0
  [929cbde3] LLVM v9.3.1
  [b964fa9f] LaTeXStrings v1.4.0
  [10f19ff3] LayoutPointers v0.1.17
  [5078a376] LazyArrays v2.6.1
  [d3d80556] LineSearches v7.3.0
  [7ed4a6bd] LinearSolve v3.9.0
  [2ab3a3ac] LogExpFunctions v0.3.29
  [1914dd2f] MacroTools v0.5.16
  [d125e4d3] ManualMemory v0.1.8
  [e1d29d7a] Missings v1.2.0
  [2e0e35c7] Moshi v0.3.5
  [46d2c3a1] MuladdMacro v0.2.4
  [d41bc354] NLSolversBase v7.9.1
  [872c559c] NNlib v0.9.30
  [77ba4419] NaNMath v1.1.3
  [d8793406] ObjectFile v0.4.4
  [429524aa] Optim v1.12.0
  [3bd65402] Optimisers v0.4.6
  [bac558e1] OrderedCollections v1.8.0
  [bbf590c4] OrdinaryDiffEqCore v1.23.0
  [90014a1f] PDMats v0.11.34
  [d96e819e] Parameters v0.12.3
  [e409e4f3] PoissonRandom v0.4.4
  [f517fe37] Polyester v0.7.16
  [1d0040c9] PolyesterWeave v0.2.2
  [85a6dd25] PositiveFactorizations v0.2.4
  [d236fae5] PreallocationTools v0.4.26
⌅ [aea7be01] PrecompileTools v1.2.1
  [21216c6a] Preferences v1.4.3
  [08abe8d2] PrettyTables v2.4.0
  [43287f4e] PtrArrays v1.3.0
  [1fd47b50] QuadGK v2.11.2
  [74087812] Random123 v1.7.0
  [e6cf234a] RandomNumbers v1.6.0
  [c1ae055f] RealDot v0.1.0
  [3cdcf5f2] RecipesBase v1.3.4
  [731186ca] RecursiveArrayTools v3.33.0
  [189a3867] Reexport v1.2.2
  [ae029012] Requires v1.3.1
  [ae5879a3] ResettableStacks v1.1.1
  [37e2e3b7] ReverseDiff v1.16.1
  [79098fc4] Rmath v0.8.0
  [7e49a35a] RuntimeGeneratedFunctions v0.5.14
  [94e857df] SIMDTypes v0.1.0
  [0bca4576] SciMLBase v2.86.2
  [19f34311] SciMLJacobianOperators v0.1.3
  [c0aeaf25] SciMLOperators v0.3.13
  [1ed8b502] SciMLSensitivity v7.78.0
  [53ae85a6] SciMLStructures v1.7.0
  [7e506255] ScopedValues v1.3.0
  [6c6a2e73] Scratch v1.2.1
  [efcf1570] Setfield v1.1.2
  [ce78b400] SimpleUnPack v1.1.0
  [a2af1166] SortingAlgorithms v1.2.1
  [dc90abb0] SparseInverseSubset v0.1.2
  [276daf66] SpecialFunctions v2.5.1
  [aedffcd0] Static v1.2.0
  [0d7ed370] StaticArrayInterface v1.8.0
  [90137ffa] StaticArrays v1.9.13
  [1e83bf80] StaticArraysCore v1.4.3
  [10745b16] Statistics v1.11.1
  [82ae8749] StatsAPI v1.7.0
  [2913bbd2] StatsBase v0.34.4
  [4c63d2b9] StatsFuns v1.5.0
  [7792a7ef] StrideArraysCore v0.5.7
  [892a3eda] StringManipulation v0.4.1
  [09ab397b] StructArrays v0.7.1
  [53d494c1] StructIO v0.3.1
  [2efcf032] SymbolicIndexingInterface v0.3.40
  [3783bdb8] TableTraits v1.0.1
  [bd369af6] Tables v1.12.0
  [8290d209] ThreadingUtilities v0.5.3
  [9f7883ad] Tracker v0.2.38
  [e689c965] Tracy v0.1.4
  [781d530d] TruncatedStacktraces v1.4.0
  [3a884ed6] UnPack v1.0.2
  [013be700] UnsafeAtomics v0.3.0
  [e88e6eb3] Zygote v0.7.6
  [700de1a5] ZygoteRules v0.2.7
⌅ [7cc45869] Enzyme_jll v0.0.174+0
  [1d5cc7b8] IntelOpenMP_jll v2025.0.4+0
  [dad2f222] LLVMExtra_jll v0.0.35+0
  [ad6e5548] LibTracyClient_jll v0.9.1+6
  [856f044c] MKL_jll v2025.0.1+1
  [efe28fd5] OpenSpecFun_jll v0.5.6+0
  [f50d1b31] Rmath_jll v0.5.1+0
  [1317d2d5] oneTBB_jll v2022.0.0+0
  [0dad84c5] ArgTools v1.1.2
  [56f22d72] Artifacts v1.11.0
  [2a0f44e3] Base64 v1.11.0
  [ade2ca70] Dates v1.11.0
  [8ba89e20] Distributed v1.11.0
  [f43a241f] Downloads v1.6.0
  [7b1f6079] FileWatching v1.11.0
  [9fa8497b] Future v1.11.0
  [b77e0a4c] InteractiveUtils v1.11.0
  [4af54fe1] LazyArtifacts v1.11.0
  [b27032c2] LibCURL v0.6.4
  [76f85450] LibGit2 v1.11.0
  [8f399da3] Libdl v1.11.0
  [37e2e46d] LinearAlgebra v1.11.0
  [56ddb016] Logging v1.11.0
  [d6f4376e] Markdown v1.11.0
  [ca575930] NetworkOptions v1.2.0
  [44cfe95a] Pkg v1.11.0
  [de0858da] Printf v1.11.0
  [9a3f8284] Random v1.11.0
  [ea8e919c] SHA v0.7.0
  [9e88b42a] Serialization v1.11.0
  [6462fe0b] Sockets v1.11.0
  [2f01184e] SparseArrays v1.11.0
  [4607b0f0] SuiteSparse
  [fa267f1f] TOML v1.0.3
  [a4e569a6] Tar v1.10.0
  [cf7118a7] UUIDs v1.11.0
  [4ec0a83e] Unicode v1.11.0
  [e66e0078] CompilerSupportLibraries_jll v1.1.1+0
  [deac9b47] LibCURL_jll v8.6.0+0
  [e37daf67] LibGit2_jll v1.7.2+0
  [29816b5a] LibSSH2_jll v1.11.0+1
  [c8ffd9c3] MbedTLS_jll v2.28.6+0
  [14a3606d] MozillaCACerts_jll v2023.12.12
  [4536629a] OpenBLAS_jll v0.3.27+1
  [05823500] OpenLibm_jll v0.8.5+0
  [bea87d4a] SuiteSparse_jll v7.7.0+0
  [83775a58] Zlib_jll v1.2.13+1
  [8e850b90] libblastrampoline_jll v5.11.0+0
  [8e850ede] nghttp2_jll v1.59.0+0
  [3f19e933] p7zip_jll v17.4.0+2
  • Output of versioninfo()
Julia Version 1.11.5
Commit 760b2e5b739 (2025-04-14 06:53 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: macOS (arm64-apple-darwin24.0.0)
  CPU: 10 × Apple M1 Pro
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, apple-m1)
Threads: 1 default, 0 interactive, 1 GC (on 8 virtual cores)
@mchitre mchitre added the bug Something isn't working label Apr 28, 2025
ChrisRackauckas added a commit that referenced this issue May 3, 2025
@ChrisRackauckas
Fix default adjoint for AppleAccelerate and MKL
1fe3ffa 
This can't be easily tested because it's very architecture-dependent, but it fixes #601
@ChrisRackauckas ChrisRackauckas mentioned this issue May 3, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix default adjoint for AppleAccelerate and MKL SciML/LinearSolve.jl
1 participant
@mchitre