Support the multipart content type

[czechboy0]

https://github.com/OAI/OpenAPI-Specification/blob/main/versions/3.1.0.md#special-considerations-for-multipart-content

Let's do a proposal first, as there are a few ways the generated code could go here.

[czechboy0]

This one needs a bit of design, but should be possible.

A simple idea:

• generate a struct for the whole multipart payload
• each property is one content type + payload, decoded as we'd normally decode it if it were a full body
• this struct would not itself be Codable, as we'd generate custom code to decode/encode each of the properties, just like we do for type-safe properties for each operation

[czechboy0]

Additional reading: https://datatracker.ietf.org/doc/html/rfc7578

[czechboy0]

There are also headers associated with each part, not just content type + payload: https://github.com/OAI/OpenAPI-Specification/blob/main/versions/3.1.0.md#encoding-object-example

[czechboy0]

Can also be represented as an array property at the top level, at which point it could lend itself well to being a stream of values?

[czechboy0]

Should the parts be available as they become available, or all at the end in the generated struct? Seems we have to decide between more type-safety and delivering parts as they're ready instead of only returning them all at the end.

[czechboy0]

I'm working on this now, is taking a bit of time as it's a relatively large feature. A proposal with the new API/generated code is coming hopefully next week.

In the meantime, please drop here your use cases of multipart, such as:

• request body or response body?
• many parts of the same type (e.g. file), or heterogeneous, i.e. a raw log, a JSON object, and an image all in our request/response
• what are you representing with the parts? (files, images, logs, JSON blobs, ...)

Thanks! 🙏

[anthonybishopric]

Hi @czechboy0 thank you for taking on this issue, as I'm currently running into a related issue with the generated client while performing a multipart upload.

I'm writing code that updates a user's profile photo, they are submitting the photo Data after retrieving that value from their library.

 let res = try await client.post_sol_users_sol__lcub_userId_rcub__sol_photo(
                .init(path: .init(userId: user.userId),
                      headers: .init(),
                      body: .multipartForm(data) // image data here
                     )
            )

Unfortunately the server is producing an error when I attempt to complete this operation:

no multipart boundary param in Content-Type

This makes sense, as the client currently hardcodes the content type to multipart/form-data. From the relevant part of the generated client;

                    request.body = try converter.setOptionalRequestBodyAsBinary(
                        value,
                        headerFields: &request.headerFields,
                        contentType: "multipart/form-data"
                    )

Even without fully fleshing out the multipart spec as outlined above, it would be great to address this in a point release of the client.

Thank you for taking the time to work on this!

[czechboy0]

Hi @anthonybishopric,

unfortunately this is expected, as what you're sending is not a valid multipart request.

While I hope to have a proposal up for multipart support this week, and hopefully have everything landed by end of next week, here's a temporary workaround:

1. Write a client middleware that adds a constant "boundary" parameter to the content type header before it's sent to the server.
2. Change the raw body you're providing by making it look like a multipart request, i.e. boundary to start, part headers section, then your image body, then an ending boundary.

Lmk if you need more details here.

[czechboy0]

More details.

In your middleware, you'd modify the header from

content-type: multipart/form-data

to

content-type: multipart/form-data; boundary=__X_SWIFT_OPENAPI_GENERATOR_BOUNDARY__

And then the body would need to look something like:

--__X_SWIFT_OPENAPI_GENERATOR_BOUNDARY__
Content-Disposition: form-data; name="photo"

<raw photo bytes here>
--__X_SWIFT_OPENAPI_GENERATOR_BOUNDARY__--

Note that each newline needs to be a CRLF, not just a simple newline.

Hopefully this should be accepted by the server and can serve as a workaround for the new few weeks before proper multipart support lands.

[anthonybishopric]

Thanks for the suggestions @czechboy0! I found a workaround that is okay for the time being. I changed the upload parameters to accept an application/octet-stream instead and simply pass data unadorned.

      requestBody:
        content:
          application/octet-stream:
            schema:
              type: string
              format: binary

This only works because in my particular case I don't have other metadata to upload in the body, so I will keep watching progress on this issue.

[czechboy0]

Oh yeah if you only have a single payload, don't use multipart, use "single"part 🙂 (what you're doing, just one body payload).

[czechboy0]

Proposal: #369

Please, have a read and chime in with questions, comments.

[czechboy0]

https://forums.swift.org/t/proposal-soar-0009-type-safe-streaming-multipart-support/68331

[czechboy0]

All the runtime changes landed in main, will be released in 1.0.0-alpha.1 next week.

Generator changes are in a PR: #366.

[czechboy0]

Runtime changes landed, all going out next week.

[erikhric]

@czechboy0 is there a way to track upload progress of a multipart request?

[czechboy0]

@czechboy0 is there a way to track upload progress of a multipart request?

Yes, a few - using a middleware or using a wrapping async sequence. Before I recommend more details, can you elaborate? Do you want to track the upload progress of the full body (which includes potentially multiple multipart parts), or just the one part? What is the code for uploading it, how are you providing the data?

[erikhric]

I'm uploading a large video and a short text. Upload progress of the text content (description) is negligible - I don't care if it is included in overall progres.

guard let data = try? Data(contentsOf: videoURL) else { return .... }
let body = HTTPBody(data)
let multi: MultipartBody<Operations.post_sol_accounts_sol__lcub_account_rcub__sol_content_sol_basic_sol_video.Input.Body.multipartFormPayload> =  [
            .video(.init(payload: .init(body: body), filename: "video")),
            .content(.init(payload: .init(body: HTTPBody(description))))
        ] 

[czechboy0]

I haven't tested that this works, but you could use NIO's FileSystem module (https://github.com/apple/swift-nio/blob/main/Sources/NIOFileSystem/Docs.docc/NIOFileSystem.md) like this. It has the advantage of never loading the full video into memory at once, instead streaming it from disk and printing a log line on every chunk update.

import OpenAPIRuntime
import Foundation
import OpenAPIURLSession
import NIOFileSystem

let client = Client(
    serverURL: URL(string: "https://example.com/api")!,
    transport: URLSessionTransport()
)

actor ProgressLogger {
    private let totalSize: Int64
    private var accumulatedSize: Int64
    
    init(totalSize: Int64) {
        self.totalSize = totalSize
        self.accumulatedSize = 0
    }
    
    func processChunkSize(_ size: Int) {
        accumulatedSize += Int64(size)
        print("Progress: \(accumulatedSize)/\(totalSize)")
    }
}

try await FileSystem.shared.withFileHandle(forReadingAt: FilePath("/foo/video.mp4")) { read in
    let chunks = read.readChunks()
    let size = try await read.fileHandle.info().size
    let progress = ProgressLogger(totalSize: size)
    _ = try await client
        .upload(
            body: .multipartForm(
                [
                    .content(.init(payload: .init(body: "this is a description"))),
                    .video(
                        .init(
                            payload: .init(body: HTTPBody(
                                chunks
                                    .map { ArraySlice($0.readableBytesView) }
                                    .map { chunk in
                                        await progress.processChunkSize(chunk.count)
                                        return chunk
                                    },
                                length: .known(size),
                                iterationBehavior: .single
                            )),
                            filename: "video.mp4"
                        )
                    )
                ]
            )
        )
        .accepted
}

[erikhric]

I haven't tested that this works, but you could use NIO's FileSystem module (https://github.com/apple/swift-nio/blob/main/Sources/NIOFileSystem/Docs.docc/NIOFileSystem.md) like this. It has the advantage of never loading the full video into memory at once, instead streaming it from disk and printing a log line on every chunk update.

import OpenAPIRuntime
import Foundation
import OpenAPIURLSession
import NIOFileSystem

let client = Client(
    serverURL: URL(string: "https://example.com/api")!,
    transport: URLSessionTransport()
)

actor ProgressLogger {
    private let totalSize: Int64
    private var accumulatedSize: Int64
    
    init(totalSize: Int64) {
        self.totalSize = totalSize
        self.accumulatedSize = 0
    }
    
    func processChunkSize(_ size: Int) {
        accumulatedSize += Int64(size)
        print("Progress: \(accumulatedSize)/\(totalSize)")
    }
}

try await FileSystem.shared.withFileHandle(forReadingAt: FilePath("/foo/video.mp4")) { read in
    let chunks = read.readChunks()
    let size = try await read.fileHandle.info().size
    let progress = ProgressLogger(totalSize: size)
    _ = try await client
        .upload(
            body: .multipartForm(
                [
                    .content(.init(payload: .init(body: "this is a description"))),
                    .video(
                        .init(
                            payload: .init(body: HTTPBody(
                                chunks
                                    .map { ArraySlice($0.readableBytesView) }
                                    .map { chunk in
                                        await progress.processChunkSize(chunk.count)
                                        return chunk
                                    },
                                length: .known(size),
                                iterationBehavior: .single
                            )),
                            filename: "video.mp4"
                        )
                    )
                ]
            )
        )
        .accepted
}

That worked with almost no changes! Thanks a lot. This should be documented somewhere in readme 🙏