[SR-15697] Program crashes when calling asynchronous method multiple times on Windows

[MarSe32m]

Previous ID	SR-15697
Radar	None
Original Reporter	@MarSe32m
Type	Bug

Attachment: Download

Environment

Windows 10.

Swift snapshot: swift-DEVELOPMENT-SNAPSHOT-2021-12-23-a

Additional Detail from JIRA

Votes	0
Component/s
Labels	Bug, Concurrency, Windows
Assignee	None
Priority	Medium

md5: 9524581afb3bfc618b5c9a4c56b3bd35

Issue Description:

The following program crashes, after some 6000_ish_ iterations of the receive loop, in the following situations:

• running the program with swift run and using the local (LocalAsyncStack) variable

• running the program with swift run and using the module (AsyncStack) variable from AsyncStackModule

• running the program with swift run and using the module (AsyncStack) variable from the AsyncStackModule

The program doesn't crash when its run with swift run -c release and using the local (LocalAsyncStack) variable.

All of the crashing cases disappear when the async is removed from the receive method, i.e. when

public mutating func receive() async -> Element?

is changed to

public mutating func receive() -> Element?

for both LocalAsyncStack and the AsyncStack from the AsyncStackModule.

When the send method of the (Local)AsyncStack is made async, then the program crashes on the first call to the send method. This crash appears on the same cases as mentioned above. This crash doesn't appear when ran with swift run -c release with LocalAsyncStack.

Target asyngbugtwo:
App.swift

import AsyncStackModule

@main
public struct App {
    public static func main() async {
        // var local = LocalAsyncStack<Int>()
        var module = AsyncStack<Int>()

        let upperBound = 100000
        for i in 0...upperBound {
            // await local.send(i)
            await module.send(i)
            if i % 10000 == 0 {
                print(i)
            }
        }

        var index = 0
        // while let value = await local.receive() {
        while let value = await module.receive() {
            precondition(index == upperBound - value)
            print("value", value)
            index += 1
        }
        print("Done:", index)
    }
}

LocalAsyncStack.swift

public struct LocalAsyncStack<Element>: @unchecked Sendable {
    var buffer = [Element]()

    public init() {}

    public mutating func send(_ value: Element) {
        buffer.append(value)
    }

    public mutating func receive() async -> Element? {
        buffer.popLast()
    }
}

Target AsyncStackModule:
AsyncStack.swift (identical to LocalAsyncStack)

public struct AsyncStack<Element>: @unchecked Sendable {
    var buffer = [Element]()

    public init() {}

    public mutating func send(_ value: Element) {
        buffer.append(value)
    }

    public mutating func receive() async -> Element? {
        buffer.popLast()
    }
}

Package.swift

// swift-tools-version: 5.5
import PackageDescription

let package = Package(
    name: "asyncbugtwo",
    targets: [
        .target(name: "AsyncStackModule"),
        .executableTarget(name: "asyncbugtwo", dependencies: ["AsyncStackModule"])
    ]
)

I can't reproduce this behavior on Linux or macOS.

[compnerd]

This is most likely due to the limitation of LLVM - LLVM does not support mandatory tail call optimizations on Windows frames, which means that you may hit a stack limit. I've not tested it, so this is wild speculation.

[stefanspringer1]

I am also getting a crash with too many async calls (additional to the crash reported on https://github.com/compnerd/swift-win32, echo %ERRORLEVEL% gives-1073741819). The application that crashes is an important one and is relies heavenly on async calls, so the problem makes Swift more or less useless for me on Windows.

[compnerd]

@MarSe32m could you please test with the latest snapshot? I am having trouble reproducing the crash.

[MarSe32m]

I can't reproduce this anymore, neither on latest snapshot 2023-02-02 nor version 5.7.3.
I modified the AsyncStack implementation such that it acutally suspends on calls to pop()

import Foundation

final class AsyncStack<Element>: @unchecked Sendable {
    var elements = [Element]()
    var consumers = [UnsafeContinuation<Element, Never>]()
    var lock = NSLock()

    final func push(_ element: Element) {
        lock.lock()
        defer { lock.unlock() }
        if consumers.isEmpty {
            elements.append(element)
        } else {
            let consumer = consumers.removeFirst()
            consumer.resume(returning: element)
        }
    }

    final func pop() async -> Element {
        lock.lock()
        if elements.isEmpty {
            return await withUnsafeContinuation {
                consumers.append($0)
                lock.unlock()
            }
        } else {
            let element = elements.removeLast()
            lock.unlock()
            return element
        }
    }
}

func testAsyncStack() async {
    let asyncStack = AsyncStack<Int>()
    let tasks = 100
    let iterations = 10000
    let expectedSum = (0..<tasks).map {_ in (0..<iterations).reduce(0, +) }.reduce(0, +)
    for taskId in 0..<tasks {
        Task.detached {
            for i in 0..<iterations {
                asyncStack.push(i)
            }
            print(taskId, "done")
        }
    }
    let consumerTasks = (0..<tasks).map { _ in 
        Task.detached {
            var sum = 0
            for _ in 0..<iterations {
                sum += await asyncStack.pop()
            }
            return sum
        }
    }
    print("Consuming")
    var sum = 0
    for task in consumerTasks {
        sum += await task.value
        print(sum)
    }
    print(sum, expectedSum)
    precondition(sum == expectedSum)
}

@main
struct asyncbug {
    static func main() async {
        await testAsyncStack()
    }
}

Tweaking the number of tasks and iterations will cause this to crash at different times. It happens to crash also with tasks = 1. echo %ERRORLEVEL% gives me -1073741571 --> 0xC00000FD --> STATUS_STACK_OVERFLOW.

[MarSe32m]

Sorry for bombarding with code but the following example might also be interesting since it gives me the same ERRORLEVEL as above.

func asyncStream() async {
    let iterations = 100000
    let stream = AsyncStream<Int> { continuation in 
        Task.detached {
            var sum = 0
            for i in 0..<iterations {
                let value = i
                sum += value
                continuation.yield(value)
            }
            continuation.yield(-sum)
            continuation.finish()
            print("Done yielding")
        }
    }

    var finalSum = 0
    for await value in stream {
        print("Received", value)
        finalSum += value
    }
    print("Done")
    precondition(finalSum == 0)
}

@main
struct asyncbug {
    static func main() async {
        await asyncStream()
    }
}

This crashes at around "Received 350". What makes this interesting is that if instead of calling asyncStream(), you copy the contents of the function inside main(), the program will not crash regardless of how large iterations is.

[compnerd]

@MarSe32m - awesome, thank you for the updated snippet!

[compnerd]

#63660 should address this, but at the cost of debuggability and the ability to recover stack traces once an async function is encountered.

[compnerd]

Marking this closed as #63660 is merged.