Skip to content

[AutoDiff] Specifically-shaped differentiable functions yield "conflicting debug info for argument" assertion failure #58660

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
BradLarson opened this issue May 4, 2022 · 3 comments · Fixed by #58763
Assignees
Labels
AutoDiff bug A deviation from expected or documented behavior. Also: expected but undesirable behavior.

Comments

@BradLarson
Copy link
Contributor

Describe the bug

Specifically-shaped differentiable functions can lead to an assertion failure of "conflicting debug info for argument" in optimized builds with debug symbols enabled. This has been an issue for some more complex differentiable Swift code top-of-tree Swift since at least September of 2021, but only recently have we been able to reproduce this in a single file.

To Reproduce

The following single-file reproducer will exhibit this behavior on the 2022-04-23 nightly toolchain snapshot or newer (PR #42245 needs to be in place for this to not hit an earlier assertion failure):

import _Differentiation

public typealias Scalar = Double

struct MyModel: Differentiable {

	struct MyState: Differentiable {
		var test0: Scalar
		var time: Scalar
		var test1: Scalar
		var allVals: AllVals

		init() {
			self.test0 = 0.01
			self.time = 0.01
			self.test1 = 0.01
			self.allVals = AllVals()
		}
	}

	var state = MyState()

	@differentiable(reverse)
	func compute(_ vals: AllValsTuple) -> [Scalar] {
        return []
	}

    struct AllVals: Differentiable {
        var val1: Scalar
        var val2: Scalar
        var val3: Scalar
        var val4: Scalar

        init() {
            self.val1 = 0
            self.val2 = 0
            self.val3 = 0
            self.val4 = 0
        }
    }

    struct AllValsTuple: Differentiable {
        var test1: Scalar
        var test0: Scalar

        init() {
            self.test1 = 0
            self.test0 = 0
        }
    }

    @differentiable(reverse)
    func compute0(_ vals: AllValsTuple) -> Scalar {
        return compute(vals)[0]
    }

    @differentiable(reverse)
    func compute1(_ vals: AllValsTuple) -> Scalar {
        return compute(vals)[1]
    }

	@differentiable(reverse)
	mutating func differentiableDoCalc() {
		let maxIters = 50
		let maxStep = 100.0
		let tol: Scalar = 2e5*Scalar.ulpOfOne
		let fdPert: Scalar = 0.00001
		var updateNorm: Scalar = 0
		var i: Int = 0
		var update = [Scalar](repeating: 0.0, count: 2)
		var calcEstimate = [state.test1, state.test0]
		repeat {
			var calcVars = AllValsTuple()
			calcVars.test1 = calcEstimate[0]
			calcVars.test0 = calcEstimate[1]
			var calcVarPert = calcVars
			var pertRow0: Scalar = 0.0
			let baseRow0 = compute0(calcVars)
			calcVarPert.test1 += fdPert
			pertRow0 = compute0(calcVarPert)
			let element00 = (pertRow0 - baseRow0)/fdPert
			calcVarPert.test1 -= fdPert
			calcVarPert.test0 += fdPert
			pertRow0 = compute0(calcVarPert)
			let element01 = (pertRow0 - baseRow0)/fdPert
			calcVarPert.test0 -= fdPert
			var pertRow1: Scalar = 0.0
			let baseRow1 = compute1(calcVars)
			calcVarPert.test1 += fdPert
			pertRow1 = compute1(calcVarPert)
			let element10 = (pertRow1 - baseRow1)/fdPert
			calcVarPert.test1 -= fdPert
			calcVarPert.test0 += fdPert
			pertRow1 = compute1(calcVarPert)
			let element11 = (pertRow1 - baseRow1)/fdPert
			calcVarPert.test0 -= fdPert
			let Afd = [[element00, element01], [element10, element11]]
			let rhs = compute(calcVars)
			update = identity1(Afd, b: rhs)
			update = identity2(update, bound: maxStep)
			updateNorm = identity4(update)
			calcEstimate = identity3(calcEstimate, update)
			i += 1
		} while (i < withoutDerivative(at: maxIters)) && (updateNorm > tol)
        state.test1 = calcEstimate[0]
        state.test0 = calcEstimate[1]
	}
}

@differentiable(reverse)
public func identity1(_ A: [[Scalar]], b: [Scalar]) -> [Scalar] {
    return A[0]
}

@differentiable(reverse)
public func identity4(_ x: [Scalar]) -> Scalar {
    return x[0]
}

@differentiable(reverse)
public func identity2(_ x: [Scalar], bound: Scalar) -> [Scalar] {
    return x
}

@differentiable(reverse)
public func identity3(_ x: [Scalar], _ y: [Scalar]) -> [Scalar] {
    return x
}

To reproduce, place the above in a file and run swiftc -O -g file.swift. Note the need for both optimization and debug symbol generation.

Expected behavior

Code compiles cleanly.

Results

The following is output:

conflicting debug info for argument
  call void @llvm.dbg.value(metadata double %1608, metadata !1677, metadata !DIExpression(DW_OP_LLVM_fragment, 0, 64)), !dbg !2090
!1675 = !DILocalVariable(name: "self", arg: 1, scope: !1647, file: !14, type: !1194, flags: DIFlagArtificial)
!1677 = !DILocalVariable(name: "self", arg: 1, scope: !1647, file: !14, line: 63, type: !1194, flags: DIFlagArtificial)
<unknown>:0: error: fatal error encountered during compilation; please submit a bug report (https://swift.org/contributing/#reporting-bugs) and include the project
<unknown>:0: note: Broken module found, compilation aborted!
Please submit a bug report (https://swift.org/contributing/#reporting-bugs) and include the project and the crash backtrace.
Stack dump:
0.	Program arguments: /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2022-04-23-a.xctoolchain/usr/bin/swift-frontend -frontend -c -primary-file ConflictingDebugInfo.swift -target x86_64-apple-macosx12.0 -enable-objc-interop -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk -color-diagnostics -g -O -new-driver-path /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2022-04-23-a.xctoolchain/usr/bin/swift-driver -empty-abi-descriptor -resource-dir /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2022-04-23-a.xctoolchain/usr/lib/swift -module-name ConflictingDebugInfo -target-sdk-version 12.3 -o /var/folders/1s/z_s04yq55wq2jvgqghm8zm7w0000gp/T/TemporaryDirectory.GzcnFi/ConflictingDebugInfo-2.o
1.	Apple Swift version 5.7-dev (LLVM 0111970099b6de6, Swift 5d55ffea514a842)
2.	Compiling with the current language version
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0  swift-frontend           0x0000000113b67aa7 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 39
1  swift-frontend           0x0000000113b66cc5 llvm::sys::RunSignalHandlers() + 85
2  swift-frontend           0x0000000113b680e0 SignalHandler(int) + 288
3  libsystem_platform.dylib 0x00007ff802464dfd _sigtramp + 29
4  libsystem_platform.dylib 0x00007fd160000000 _sigtramp + 18446743907778277920
5  libsystem_c.dylib        0x00007ff80239ad24 abort + 123
6  swift-frontend           0x000000010eb7f1f8 swift::performFrontend(llvm::ArrayRef<char const*>, char const*, void*, swift::FrontendObserver*)::$_3::__invoke(void*, char const*, bool) + 552
7  swift-frontend           0x0000000113abf22d llvm::report_fatal_error(llvm::Twine const&, bool) + 365
8  swift-frontend           0x0000000113abf0bb llvm::report_fatal_error(char const*, bool) + 43
9  swift-frontend           0x0000000113923d2b (anonymous namespace)::VerifierLegacyPass::doFinalization(llvm::Module&) + 187
10 swift-frontend           0x00000001138a1216 llvm::FPPassManager::doFinalization(llvm::Module&) + 70
11 swift-frontend           0x000000011389a7d5 llvm::legacy::FunctionPassManagerImpl::doFinalization(llvm::Module&) + 85
12 swift-frontend           0x000000010ef63b4d swift::performLLVMOptimizations(swift::IRGenOptions const&, llvm::Module*, llvm::TargetMachine*) + 1261
13 swift-frontend           0x000000010ef650a2 swift::performLLVM(swift::IRGenOptions const&, swift::DiagnosticEngine&, llvm::sys::SmartMutex<false>*, llvm::GlobalVariable*, llvm::Module*, llvm::TargetMachine*, llvm::StringRef, swift::UnifiedStatsReporter*) + 3602
14 swift-frontend           0x000000010eb781ae performCompileStepsPostSILGen(swift::CompilerInstance&, std::__1::unique_ptr<swift::SILModule, std::__1::default_delete<swift::SILModule> >, llvm::PointerUnion<swift::ModuleDecl*, swift::SourceFile*>, swift::PrimarySpecificPaths const&, int&, swift::FrontendObserver*) + 2526
15 swift-frontend           0x000000010eb77065 swift::performCompileStepsPostSema(swift::CompilerInstance&, int&, swift::FrontendObserver*) + 693
16 swift-frontend           0x000000010eb79459 swift::performFrontend(llvm::ArrayRef<char const*>, char const*, void*, swift::FrontendObserver*) + 3801
17 swift-frontend           0x000000010ea56ed4 swift::mainEntry(int, char const**) + 3220
18 dyld                     0x0000000128f8851e start + 462

Environment (please complete the following information):

  • OS: Linux, macOS for both X86_64 and aarch64
  • Xcode Version/Tag/Branch: reproducer works for 2022-04-23 nightly snapshot and newer
@BradLarson BradLarson added bug A deviation from expected or documented behavior. Also: expected but undesirable behavior. AutoDiff labels May 4, 2022
@asl
Copy link
Contributor

asl commented May 4, 2022

So, here it looks like line number information was lost somehow. Weird :)

@philipturner
Copy link
Contributor

I narrowed down the reproducer much further. On older toolchains, it still triggers the old debug info crash that was affecting PassiveLogic and S4TF. The crash's signature changed some time between the 2022-04-04 and 2022-04-24 toolchains, based on what I tested.

import _Differentiation

// May be a `struct` or `class`.
class MyState: Differentiable {
  // All of these must be stored instance properties. There must be at least 7
  // differentiable properties of any type.
  var property1: Float = 0
  var property2: Float = 0
  var property3: Float = 0
  var property4: Float = 0
  var property5: Float = 0
  var property6: Float = 0
  var property7: Float = 0
}

struct MyModel: Differentiable {
  // May be `var` or `let`, but must not be `@noDerivative`. Must be a stored
  // instance property.
  let property1 = MyState()

  // Must be an instance property, either stored or computed.
  var property2: Float {
    // `get` must exist, and may add `mutating` attribute.
    get { 0 }
    // Cannot add `nonmutating` attribute to `set`.
    set { }
  }

  // Must be an instance member. May be a function or computed property, but not
  // a stored property.
  var member3: Float {
    // May not add `mutating` attribute.
    get { 0 }
  }

  @differentiable(reverse)
  mutating func member4() {
    // Must be a differentiable type.
    var localVar: Float = 0

    // Must be assigned from the value of `localVar`, not the value of anything else.
    property2 = localVar

    // `false` may instead be any expression that returns a `Bool`.
    if false {
      localVar = member3
    }
  }
}

Terminal output:

(base) philipturner@m1-max-mbp bug % swiftc -O -g file.swift                 
error: compile command failed due to signal 6 (use -v to see invocation)
conflicting debug info for argument
  call void @llvm.dbg.value(metadata float %167, metadata !768, metadata !DIExpression(DW_OP_LLVM_fragment, 0, 32)), !dbg !793
!765 = !DILocalVariable(name: "self", arg: 1, scope: !760, file: !18, line: 37, type: !577, flags: DIFlagArtificial)
!768 = !DILocalVariable(name: "self", arg: 1, scope: !760, file: !18, type: !577, flags: DIFlagArtificial)
<unknown>:0: error: fatal error encountered during compilation; please submit a bug report (https://swift.org/contributing/#reporting-bugs) and include the project
<unknown>:0: note: Broken module found, compilation aborted!
Please submit a bug report (https://swift.org/contributing/#reporting-bugs) and include the project and the crash backtrace.
Stack dump:
0.	Program arguments: /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2022-05-04-a.xctoolchain/usr/bin/swift-frontend -frontend -c -primary-file file.swift -target arm64-apple-macosx12.0 -Xllvm -aarch64-use-tbi -enable-objc-interop -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk -color-diagnostics -g -O -new-driver-path /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2022-05-04-a.xctoolchain/usr/bin/swift-driver -empty-abi-descriptor -resource-dir /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2022-05-04-a.xctoolchain/usr/lib/swift -module-name file -target-sdk-version 12.3 -enable-default-cmo -o /var/folders/qn/86czb43d3pv03bfnxvb3x66h0000gn/T/TemporaryDirectory.YFmr3a/file-2.o
1.	Apple Swift version 5.7-dev (LLVM cd62c186b914a47, Swift c58d4dba34ef37a)
2.	Compiling with the current language version
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0  swift-frontend           0x00000001050f1be8 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 56
1  swift-frontend           0x00000001050f0e48 llvm::sys::RunSignalHandlers() + 128
2  swift-frontend           0x00000001050f224c SignalHandler(int) + 304
3  libsystem_platform.dylib 0x000000019cd194c4 _sigtramp + 56
4  libsystem_pthread.dylib  0x000000019cd01ee0 pthread_kill + 288
5  libsystem_c.dylib        0x000000019cc3c340 abort + 168
6  swift-frontend           0x0000000100c15cc4 PrettyStackTraceFrontend::~PrettyStackTraceFrontend() + 0
7  swift-frontend           0x000000010505e340 llvm::report_fatal_error(llvm::Twine const&, bool) + 268
8  swift-frontend           0x000000010505e234 llvm::report_fatal_error(llvm::Twine const&, bool) + 0
9  swift-frontend           0x0000000104ee5758 (anonymous namespace)::VerifierLegacyPass::getAnalysisUsage(llvm::AnalysisUsage&) const + 0
10 swift-frontend           0x0000000104e6c6c8 llvm::FPPassManager::doFinalization(llvm::Module&) + 68
11 swift-frontend           0x0000000104e6666c llvm::legacy::FunctionPassManagerImpl::doFinalization(llvm::Module&) + 80
12 swift-frontend           0x0000000100f87b14 swift::performLLVMOptimizations(swift::IRGenOptions const&, llvm::Module*, llvm::TargetMachine*) + 1420
13 swift-frontend           0x0000000100f88bc4 swift::performLLVM(swift::IRGenOptions const&, swift::DiagnosticEngine&, llvm::sys::SmartMutex<false>*, llvm::GlobalVariable*, llvm::Module*, llvm::TargetMachine*, llvm::StringRef, swift::UnifiedStatsReporter*) + 3092
14 swift-frontend           0x0000000100c0fd1c performCompileStepsPostSILGen(swift::CompilerInstance&, std::__1::unique_ptr<swift::SILModule, std::__1::default_delete<swift::SILModule> >, llvm::PointerUnion<swift::ModuleDecl*, swift::SourceFile*>, swift::PrimarySpecificPaths const&, int&, swift::FrontendObserver*) + 2464
15 swift-frontend           0x0000000100c0ee24 swift::performCompileStepsPostSema(swift::CompilerInstance&, int&, swift::FrontendObserver*) + 540
16 swift-frontend           0x0000000100c1bf0c withSemanticAnalysis(swift::CompilerInstance&, swift::FrontendObserver*, llvm::function_ref<bool (swift::CompilerInstance&)>, bool) + 160
17 swift-frontend           0x0000000100c10b50 swift::performFrontend(llvm::ArrayRef<char const*>, char const*, void*, swift::FrontendObserver*) + 3388
18 swift-frontend           0x0000000100b08cfc swift::mainEntry(int, char const**) + 3748
19 dyld                     0x000000010d889088 start + 516

@asl
Copy link
Contributor

asl commented May 9, 2022

This seems to be quite a weird combination of inlining, SIL mem2reg and autodiff that was required to trigger an assertion. Surprisingly enough, fixing one particular TODO resolved the issue

asl added a commit to asl/swift that referenced this issue May 11, 2022
`salvageDebugInfo` is called during SIL Mem2Reg and could produce
misleading debug info in the following case:

```
%a = alloc_stack $MyModel.TangentVector, var, name "self", argno 1, implicit, loc "debug2.swift":37:17 ...
...
store %b to %a : $*MyModel.TangentVector
```

Such SIL could be created as a result of inlining (where store comes from the inlined function).
Before this patch we'd end with `debug_value` instruction with variable information, but
without or incorrect location.

This caused LLVM IR debug info verifier assertions when there might be another instruction with
complete debug info (including location) for the same argument.

After this patch we always reuse it from destination alloca

Fixes swiftlang#58660
asl added a commit that referenced this issue May 11, 2022
…58763)

`salvageDebugInfo` is called during SIL Mem2Reg and could produce misleading debug info in the following case:

```
%a = alloc_stack $MyModel.TangentVector, var, name "self", argno 1, implicit, loc "debug2.swift":37:17 ...
...
store %b to %a : $*MyModel.TangentVector
```

Such SIL could be created as a result of inlining (where store comes from the inlined function).
Before this patch we'd end with `debug_value` instruction with variable information, but without or incorrect location.

This caused LLVM IR debug info verifier assertions when there might be another instruction with complete debug info (including location) for the same argument.

After this patch we always reuse it from destination alloca

Fixes #58660
drexin pushed a commit to drexin/swift that referenced this issue Jun 3, 2022
…wiftlang#58763)

`salvageDebugInfo` is called during SIL Mem2Reg and could produce misleading debug info in the following case:

```
%a = alloc_stack $MyModel.TangentVector, var, name "self", argno 1, implicit, loc "debug2.swift":37:17 ...
...
store %b to %a : $*MyModel.TangentVector
```

Such SIL could be created as a result of inlining (where store comes from the inlined function).
Before this patch we'd end with `debug_value` instruction with variable information, but without or incorrect location.

This caused LLVM IR debug info verifier assertions when there might be another instruction with complete debug info (including location) for the same argument.

After this patch we always reuse it from destination alloca

Fixes swiftlang#58660
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
AutoDiff bug A deviation from expected or documented behavior. Also: expected but undesirable behavior.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants