[SR-11746] lldb on Linux (Swift 5.1.2): lldb crash with {{e ev}}

[weissi]

Previous ID	SR-11746
Radar	rdar://problem/57049272
Original Reporter	@weissi
Type	Bug
Status	Resolved
Resolution	Done

Additional Detail from JIRA

Votes	0
Component/s	LLDB for Swift
Labels	Bug, Crash
Assignee	@augusto2112
Priority	Medium

md5: 998cf315c6e314400034365f07f267e8

Issue Description:

Description

When running the following 1 liner (runnable from macOS (with Docker for Mac installed) or Linux) I get an lldb crash.

cd /tmp && rm -rf swift-nio && git clone https://github.com/apple/swift-nio && cd swift-nio && git reset --hard f548a29a27d7e533f1714d2d282ca0d79f6086dd && docker run -it --rm -v "$PWD:$PWD" -w "$PWD" swift:5.1.2 swift build --build-tests && docker run -it --rm -v "$PWD:$PWD" -w "$PWD" --privileged swift:5.1.2 lldb  .build/debug/swift-nioPackageTests.xctest

which can also be broken into multiple lines:

cd /tmp
rm -rf swift-nio
git clone https://github.com/apple/swift-nio
cd swift-nio
git reset --hard f548a29a27d7e533f1714d2d282ca0d79f6086dd
docker run -it --rm -v "$PWD:$PWD" -w "$PWD" swift:5.1.2 swift build --build-tests
docker run -it --rm -v "$PWD:$PWD" -w "$PWD" --privileged swift:5.1.2 lldb  .build/debug/swift-nioPackageTests.xctest

once lldb is started, please type

(lldb) break set -f EventLoop.swift -l 857         # ignore the Unable to load module. warning
(lldb) run NIOTests.ChannelTests/testHalfClosure

Once lldb has hit the breakpoint, run

(lldb) e ev

and you'll see the output

(lldb) e ev
lldb[0x41f9f4]
lldb[0x41da3c]
lldb[0x41ff88]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x12890)[0x7fca1a88c890]
/usr/bin/../lib/liblldb.so.7svn(+0xbcd241)[0x7fca13c63241]
/usr/bin/../lib/liblldb.so.7svn(+0xde73f3)[0x7fca13e7d3f3]
/usr/bin/../lib/liblldb.so.7svn(+0xa5ba0e)[0x7fca13af1a0e]
/usr/bin/../lib/liblldb.so.7svn(+0xa255f1)[0x7fca13abb5f1]
/usr/bin/../lib/liblldb.so.7svn(+0xa25530)[0x7fca13abb530]
/usr/bin/../lib/liblldb.so.7svn(+0xa5f029)[0x7fca13af5029]
/usr/bin/../lib/liblldb.so.7svn(+0xa5e898)[0x7fca13af4898]
/usr/bin/../lib/liblldb.so.7svn(+0xa5df38)[0x7fca13af3f38]
/usr/bin/../lib/liblldb.so.7svn(+0xa5fb34)[0x7fca13af5b34]
/usr/bin/../lib/liblldb.so.7svn(+0xa60017)[0x7fca13af6017]
/usr/bin/../lib/liblldb.so.7svn(+0xa5edbe)[0x7fca13af4dbe]
/usr/bin/../lib/liblldb.so.7svn(+0xa5df54)[0x7fca13af3f54]
/usr/bin/../lib/liblldb.so.7svn(+0xa5fb34)[0x7fca13af5b34]
/usr/bin/../lib/liblldb.so.7svn(+0xa60017)[0x7fca13af6017]
/usr/bin/../lib/liblldb.so.7svn(+0xa5edbe)[0x7fca13af4dbe]
/usr/bin/../lib/liblldb.so.7svn(+0xa5df54)[0x7fca13af3f54]
/usr/bin/../lib/liblldb.so.7svn(+0xa5fb34)[0x7fca13af5b34]
/usr/bin/../lib/liblldb.so.7svn(+0xa60017)[0x7fca13af6017]
/usr/bin/../lib/liblldb.so.7svn(+0xa5edbe)[0x7fca13af4dbe]
/usr/bin/../lib/liblldb.so.7svn(+0xa5df54)[0x7fca13af3f54]
/usr/bin/../lib/liblldb.so.7svn(+0xa5fb34)[0x7fca13af5b34]
/usr/bin/../lib/liblldb.so.7svn(+0xa60017)[0x7fca13af6017]
/usr/bin/../lib/liblldb.so.7svn(+0xa5edbe)[0x7fca13af4dbe]
/usr/bin/../lib/liblldb.so.7svn(+0xa5df54)[0x7fca13af3f54]
/usr/bin/../lib/liblldb.so.7svn(+0xa5fb34)[0x7fca13af5b34]
/usr/bin/../lib/liblldb.so.7svn(+0xa60017)[0x7fca13af6017]
/usr/bin/../lib/liblldb.so.7svn(+0xa5edbe)[0x7fca13af4dbe]
/usr/bin/../lib/liblldb.so.7svn(+0xa5df54)[0x7fca13af3f54]
/usr/bin/../lib/liblldb.so.7svn(+0xa5fb34)[0x7fca13af5b34]
/usr/bin/../lib/liblldb.so.7svn(+0xa60017)[0x7fca13af6017]
/usr/bin/../lib/liblldb.so.7svn(+0xa5edbe)[0x7fca13af4dbe]
/usr/bin/../lib/liblldb.so.7svn(+0xa5df54)[0x7fca13af3f54]
/usr/bin/../lib/liblldb.so.7svn(+0xa2bb45)[0x7fca13ac1b45]
/usr/bin/../lib/liblldb.so.7svn(+0x10a8671)[0x7fca1413e671]
/usr/bin/../lib/liblldb.so.7svn(+0x10a92d4)[0x7fca1413f2d4]
/usr/bin/../lib/liblldb.so.7svn(+0xaa513e)[0x7fca13b3b13e]
/usr/bin/../lib/liblldb.so.7svn(+0xa9c1a5)[0x7fca13b321a5]
/usr/bin/../lib/liblldb.so.7svn(+0xa9fc0b)[0x7fca13b35c0b]
/usr/bin/../lib/liblldb.so.7svn(+0x9df661)[0x7fca13a75661]
/usr/bin/../lib/liblldb.so.7svn(+0x9c3bf6)[0x7fca13a59bf6]
/usr/bin/../lib/liblldb.so.7svn(+0xaa0ab6)[0x7fca13b36ab6]
/usr/bin/../lib/liblldb.so.7svn(_ZN4lldb10SBDebugger21RunCommandInterpreterEbb+0xe6)[0x7fca137ecde6]
lldb[0x40a105]
lldb[0x40b337]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7)[0x7fca11b34b97]
lldb[0x40673a]
Stack dump:
0.  Program arguments: lldb .build/debug/swift-nioPackageTests.xctest 

[weissi]

@swift-ci create

[weissi]

CC @dcci / @adrian-prantl / friss (JIRA User)

[dcci]

@augusto2112 if you want another bug to look at – this might be a good one.

[augusto2112]

I'll take a look!

[augusto2112]

Bug fixed here

[weissi]

Awesome, thanks @augusto2112. Just tried with the latest master snapshot where it still crashes but I reckon this will take another day to hit the swift master snapshot.

Here's the one-line to repro it with latest master:

docker pull swiftlang/swift:nightly-master && cd /tmp && rm -rf swift-nio && git clone https://github.com/apple/swift-nio && cd swift-nio && git reset --hard f548a29a27d7e533f1714d2d282ca0d79f6086dd && docker run -it --rm -v "$PWD:$PWD" -w "$PWD" swiftlang/swift:nightly-master swift build --build-tests && docker run -it --rm -v "$PWD:$PWD" -w "$PWD" --privileged swiftlang/swift:nightly-master lldb  .build/debug/swift-nioPackageTests.xctest

[weissi]

@augusto2112 should it be working with the latest master snapshot of today or do I need to wait for tomorrow? Just tested it and right now it still fails. I'm running

$ docker run -it --rm -v "$PWD:$PWD" -w "$PWD" --privileged swiftlang/swift:nightly-master swift -version
Swift version 5.3-dev (LLVM 06099c05ae, Swift 1d423e43c0)
Target: x86_64-unknown-linux-gnu
johannes:/tmp/swift-nio (master)
$ docker run -it --rm -v "$PWD:$PWD" -w "$PWD" --privileged swiftlang/swift:nightly-master lldb -version
lldb version 10.0.0 (git@github.com:apple/llvm-project.git revision 06099c05ae89e1a5d8256131944f04d3260b3388)
Swift version 5.3-dev (LLVM 06099c05ae, Swift 1d423e43c0)

[augusto2112]

Hi @weissi,

Hmm, yeah, I wasn't recompiling swift-nio, so running the newest lldb on the already compiled code works as expected, but if I re-clone and re-compile with the latest swift version the debugger goes back to crashing, weird...

I'll take a second look on why this is still happening, sorry for the mistake.

[weissi]

Thanks so much @augusto2112!

[augusto2112]

@weissi so what I think happened is changes in the compiler code exposed a second crash that wasn't happening before. Since I didn't recompile swift-nio with the latest compiler I wasn't getting the crash. The crash itself is a null pointer exception which I opened a PR to fix here.

[weissi]

@augusto2112 how cool and thanks so much, will monitor that PR and retest with the snapshot once that's in too 🙂

[augusto2112]

Hi @weissi. The bug is (finally!) fixed. I've tested with the latest snapshot just to make sure everything is working correctly 🙂

[weissi]

@augusto2112 You're a star! Thank you. I can confirm it works on master. Proof:

(lldb) e ev
(NIO.SelectorEvent<NIO.NIORegistration>) $R0 = {
  registration = socketChannel {
    socketChannel = {
      0 = 0x400055555643a2b0 {
        NIO.BaseSocketChannel = {
          parent = nil
          socket = 0x000055555643a220 {
            NIO.BaseSocket = {
              descriptor = 7
            }
          }
          closePromise = {
            futureResult = 0x000055555645e7c0 {
              _value = nil
              _isFulfilled = (value = 0x000055555645f100)
              eventLoop = 0x00007fffec000bb0 {
                selector = 0x00007fffec000e40 {
                  lifecycleState = open
                  eventfd = 4
                  timerfd = 5
                  earliestTimer = (_uptimeNanoseconds = 310330880637945)
                  fd = 3
                  eventsCapacity = 64
                  events = 0x00007fffec000fc0
                  registrations = 1 key/value pair {
                    [0] = {
                      key = 7
                      value = socketChannel {
                        socketChannel = {
                          0 = 0x400055555643a2b0 {...}
                          1 = (rawValue = 9)
                        }
                      }
                    }
                  }
                  deregistrationsHappened = false
                }
                thread = 0x00007fffec000b90 (pthread = 140737271047936)
                scheduledTasks = {
                  heap = {
                    type = minHeap
                    storage = {
                      _buffer = {
                        _storage = 1 value {
                          [0] = 0x00007fffec02d000
                        }
                      }
                    }
                    comparator = 0x00005555556d6040 swift-nioPackageTests.xctest`partial apply forwarder for implicit closure #&#8203;4 (A, A) -> Swift.Bool in implicit closure #&#8203;3 (A.Type) -> (A, A) -> Swift.Bool in NIO.HeapType.comparator<A where A: Swift.Comparable>(type: A.Type) -> (A, A) -> Swift.Bool at <compiler-generated>
                  }
                }
                tasksCopy = {
                  _buffer = {
                    _storage = 0 values {}
                  }
                }
                tasksLock = 0x00007fffec000d40 (mutex = 0x00007fffec000d60)
                lifecycleState = open
                _iovecs = 0x00007fffec006000
                _storageRefs = 0x00007fffec00a070
                iovecs = (_position = 0x00007fffec006000, count = 1024)
                storageRefs = (_position = 0x00007fffec00a070, count = 1024)
                _msgs = 0x00007fffec00c080
                _addresses = 0x00007ffff7ed2010
                msgs = (_position = 0x00007fffec00c080, count = 1024)
                addresses = (_position = 0x00007ffff7ed2010, count = 1024)
                promiseCreationStoreLock = 0x00007fffec000d90 (mutex = 0x00007fffec000db0)
                _promiseCreationStore = 10 key/value pairs {
                  [0] = {
                    key = (_value = 0x00005555564534a0)
                    value = (file = "/tmp/swift-nio/Sources/NIO/Bootstrap.swift", line = 519)
                  }
                  [1] = {
                    key = (_value = 0x00007fffec02cb70)
                    value = (file = "/tmp/swift-nio/Sources/NIO/Bootstrap.swift", line = 536)
                  }
                  [2] = {
                    key = (_value = 0x00007fffec02d2a0)
                    value = (file = "/tmp/swift-nio/Sources/NIO/Bootstrap.swift", line = 538)
                  }
                  [3] = {
                    key = (_value = 0x0000555556452d20)
                    value = (file = "/tmp/swift-nio/Tests/NIOTests/ChannelTests.swift", line = 1237)
                  }
                  [4] = {
                    key = (_value = 0x00007fffec02d0e0)
                    value = (file = "/tmp/swift-nio/Sources/NIO/Bootstrap.swift", line = 446)
                  }
                  [5] = {
                    key = (_value = 0x00007fffec02c600)
                    value = (file = "/tmp/swift-nio/Sources/NIO/Bootstrap.swift", line = 534)
                  }
                  [6] = {
                    key = (_value = 0x00007fffec02ceb0)
                    value = (file = "/tmp/swift-nio/Sources/NIO/Bootstrap.swift", line = 319)
                  }
                  [7] = {
                    key = (_value = 0x00007fffec02d240)
                    value = (file = "/tmp/swift-nio/Sources/NIO/EventLoop.swift", line = 740)
                  }
                  [8] = {
                    key = (_value = 0x000055555645e7c0)
                    value = (file = "/tmp/swift-nio/Sources/NIO/BaseSocketChannel.swift", line = 395)
                  }
                  [9] = {
                    key = (_value = 0x00005555564430d0)
                    value = (file = "/tmp/swift-nio/Sources/NIO/Bootstrap.swift", line = 548)
                  }
                }
              }
              _callbacks = {
                firstCallback = nil
                furtherCallbacks = nil
              }
            }
          }
          selectableEventLoop = 0x00007fffec000bb0 {...}
          addressesCached = 0x0000555556449eb0 {
            storage = 0x00005555564431c0 {
              embedded = (value = 0x000055555643c780)
            }
          }
          bufferAllocatorCached = 0x000055555644acd0 {
            storage = 0x000055555643aac0 {
              embedded = (value = 0x00005555563f1100)
            }
          }
          isActiveAtomic = 0x000055555643a040 {
            embedded = (value = 0x000055555643a130)
          }
          _pipeline = 0x0000555556449170 {
            head = 0x0000555556439ba0 {
              next = 0x00007fffec02c690 {
                next = 0x000055555645e0b0 {
                  next = nil
                  prev = 0x00007fffec02c690 {...}
                  pipeline = 0x0000555556449170 {...}
                  name = "tail"
                  inboundHandler = 0x0000555556444860
                  outboundHandler = nil
                  removeHandlerInvoked = false
                  userTriggeredRemovalStarted = false
                }
                prev = 0x0000555556439ba0 {...}
                pipeline = 0x0000555556449170 {...}
                name = "handler0"
                inboundHandler = 0x000055555643bab0 {
                  inputShutdownEventReceived = false
                  outputShutdownEventReceived = false
                  promise = {
                    futureResult = 0x0000555556452d20 {
                      _value = nil
                      _isFulfilled = (value = 0x000055555643a8f0)
                      eventLoop = 0x00007fffec000bb0 {...}
                      _callbacks = {
                        firstCallback = nil
                        furtherCallbacks = nil
                      }
                    }
                  }
                  shutdownEvent = input
                }
                outboundHandler = nil
                removeHandlerInvoked = false
                userTriggeredRemovalStarted = false
              }
              prev = nil
              pipeline = 0x0000555556449170 {...}
              name = "head"
              inboundHandler = nil
              outboundHandler = 0x000055555643b950
              removeHandlerInvoked = false
              userTriggeredRemovalStarted = false
            }
            tail = 0x000055555645e0b0 {...}
            idx = 1
            destroyed = false
            eventLoop = 0x00007fffec000bb0 {...}
            _channel = 0x000055555643a2b0 {
              NIO.BaseSocketChannel = {
                parent = nil
                socket = 0x000055555643a220 {...}
                closePromise = {
                  futureResult = 0x000055555645e7c0 {...}
                }
                selectableEventLoop = 0x00007fffec000bb0 {...}
                addressesCached = 0x0000555556449eb0 {...}
                bufferAllocatorCached = 0x000055555644acd0 {...}
                isActiveAtomic = 0x000055555643a040 {...}
                _pipeline = 0x0000555556449170 {...}
                socketDescription = "socket fd: 7"
                interestedEvent = (rawValue = 9)
                readPending = false
                pendingConnect = some {
                  futureResult = 0x00007fffec02d0e0 {
                    _value = nil
                    _isFulfilled = (value = 0x00007fffec02cfe0)
                    eventLoop = 0x00007fffec000bb0 {...}
                    _callbacks = {
                      firstCallback = 0x00005555556b5fb0 swift-nioPackageTests.xctest`partial apply forwarder for closure #&#8203;1 () -> NIO.CallbackList in NIO.EventLoopFuture.whenComplete((Swift.Result<A, Swift.Error>) -> ()) -> () at <compiler-generated>
                      furtherCallbacks = 1 value {
                        [0] = 0x00005555556ab300 swift-nioPackageTests.xctest`partial apply forwarder for reabstraction thunk helper from @escaping @callee_guaranteed () -> (@owned NIO.CallbackList) to @escaping @callee_guaranteed () -> (@out NIO.CallbackList) at <compiler-generated>
                      }
                    }
                  }
                }
                recvAllocator = {
                  minIndex = 3
                  maxIndex = 38
                  minimum = 64
                  maximum = 65536
                  initial = 1024
                  index = 32
                  nextReceiveBufferSize = 1024
                  decreaseNow = false
                }
                maxMessagesPerRead = 4
                inFlushNow = false
                autoRead = true
                lifecycleManager = {
                  eventLoop = 0x00007fffec000bb0 {...}
                  isActiveAtomic = 0x000055555643a040 {...}
                  hasSeenEOFNotification = false
                  currentState = fullyRegistered
                }
                bufferAllocator = (malloc = 0x0000555555635ae0 swift-nioPackageTests.xctest`@objc closure #&#8203;1 (Swift.Int) -> Swift.Optional<Swift.UnsafeMutableRawPointer> in NIO.ByteBufferAllocator.init() -> NIO.ByteBufferAllocator at <compiler-generated>, realloc = 0x0000555555635b30 swift-nioPackageTests.xctest`@objc closure #&#8203;2 (Swift.Optional<Swift.UnsafeMutableRawPointer>, Swift.Int) -> Swift.Optional<Swift.UnsafeMutableRawPointer> in NIO.ByteBufferAllocator.init() -> NIO.ByteBufferAllocator at <compiler-generated>, free = 0x0000555555635b70 swift-nioPackageTests.xctest`@objc closure #&#8203;3 (Swift.Optional<Swift.UnsafeMutableRawPointer>) -> () in NIO.ByteBufferAllocator.init() -> NIO.ByteBufferAllocator at <compiler-generated>, memcpy = 0x0000555555635bd0 swift-nioPackageTests.xctest`@objc closure #&#8203;4 (Swift.UnsafeMutableRawPointer, Swift.UnsafeRawPointer, Swift.Int) -> () in NIO.ByteBufferAllocator.init() -> NIO.ByteBufferAllocator at <compiler-generated>)
              }
              connectTimeout = nil
              connectTimeoutScheduled = nil
              allowRemoteHalfClosure = true
              inputShutdown = false
              outputShutdown = false
              pendingWrites = 0x000055555645eda0 {
                state = {
                  pendingWrites = {
                    _buffer = {
                      _buffer = {
                        _buffer = {
                          _storage = 16 values {
                            [0] = 0x0000555556360af0
                            [1] = 0x0000555556360af0
                            [2] = 0x0000555556360af0
                            [3] = 0x0000555556360af0
                            [4] = 0x0000555556360af0
                            [5] = 0x0000555556360af0
                            [6] = 0x0000555556360af0
                            [7] = 0x0000555556360af0
                            [8] = 0x0000555556360af0
                            [9] = 0x0000555556360af0
                            [10] = 0x0000555556360af0
                            [11] = 0x0000555556360af0
                            [12] = 0x0000555556360af0
                            [13] = 0x0000555556360af0
                            [14] = 0x0000555556360af0
                            [15] = 0x0000555556360af0
                          }
                        }
                      }
                      headBackingIndex = 0
                      tailBackingIndex = 0
                    }
                    _markedIndexOffset = nil
                  }
                  bytes = 0
                }
                iovecs = (_position = 0x00007fffec006000, count = 1024)
                storageRefs = (_position = 0x00007fffec00a070, count = 1024)
                waterMark = (low = 32768, high = 65536)
                channelWritabilityFlag = 0x00005555564495a0 {
                  embedded = (value = 0x0000555556449d00)
                }
                writeSpinCount = 16
                isOpen = true
              }
            }
          }
          socketDescription = "socket fd: 7"
          interestedEvent = (rawValue = 9)
          readPending = false
          pendingConnect = some {
            futureResult = 0x00007fffec02d0e0 {...}
          }
          recvAllocator = {
            minIndex = 3
            maxIndex = 38
            minimum = 64
            maximum = 65536
            initial = 1024
            index = 32
            nextReceiveBufferSize = 1024
            decreaseNow = false
          }
          maxMessagesPerRead = 4
          inFlushNow = false
          autoRead = true
          lifecycleManager = {
            eventLoop = 0x00007fffec000bb0 {...}
            isActiveAtomic = 0x000055555643a040 {...}
            hasSeenEOFNotification = false
            currentState = fullyRegistered
          }
          bufferAllocator = (malloc = 0x0000555555635ae0 swift-nioPackageTests.xctest`@objc closure #&#8203;1 (Swift.Int) -> Swift.Optional<Swift.UnsafeMutableRawPointer> in NIO.ByteBufferAllocator.init() -> NIO.ByteBufferAllocator at <compiler-generated>, realloc = 0x0000555555635b30 swift-nioPackageTests.xctest`@objc closure #&#8203;2 (Swift.Optional<Swift.UnsafeMutableRawPointer>, Swift.Int) -> Swift.Optional<Swift.UnsafeMutableRawPointer> in NIO.ByteBufferAllocator.init() -> NIO.ByteBufferAllocator at <compiler-generated>, free = 0x0000555555635b70 swift-nioPackageTests.xctest`@objc closure #&#8203;3 (Swift.Optional<Swift.UnsafeMutableRawPointer>) -> () in NIO.ByteBufferAllocator.init() -> NIO.ByteBufferAllocator at <compiler-generated>, memcpy = 0x0000555555635bd0 swift-nioPackageTests.xctest`@objc closure #&#8203;4 (Swift.UnsafeMutableRawPointer, Swift.UnsafeRawPointer, Swift.Int) -> () in NIO.ByteBufferAllocator.init() -> NIO.ByteBufferAllocator at <compiler-generated>)
        }
        connectTimeout = nil
        connectTimeoutScheduled = nil
        allowRemoteHalfClosure = true
        inputShutdown = false
        outputShutdown = false
        pendingWrites = 0x000055555645eda0 {...}
      }
      1 = (rawValue = 9)
    }
  }
  io = (rawValue = 8)
}