diff --git a/docs/source/using-executorch-ios.md b/docs/source/using-executorch-ios.md index 1d03284ec2c..08c862341b5 100644 --- a/docs/source/using-executorch-ios.md +++ b/docs/source/using-executorch-ios.md @@ -171,29 +171,471 @@ You can assign such a config file to your target in Xcode: ## Runtime API -Check out the [C++ Runtime API](extension-module.md) and [Tensors](extension-tensor.md) tutorials to learn more about how to load and run an exported model. It is recommended to use the C++ API for macOS or iOS, wrapped with Objective-C++ and Swift code if needed to expose it for other components. Please refer to the [Demo App](demo-apps-ios.md) as an example of such a setup. +ExecuTorch provides native Objective-C APIs, automatically bridged to Swift, for interacting with the runtime. These APIs act as wrappers around the core C++ components found in [extension/tensor](extension-tensor.md) and [extension/module](extension-module.md), offering a more idiomatic experience for Apple platform developers. -Once linked against the `executorch` runtime framework, the target can now import all ExecuTorch public headers. For example, in Objective-C++: +**Note:** These Objective-C/Swift APIs are currently experimental and subject to change. -```objectivecpp +### Importing + +Once linked against the `executorch` framework, you can import the necessary components. + +Objective-C (Objective-C++): + +```objectivec +// Import the main umbrella header for Module/Tensor/Value wrappers. #import + +// If using C++ directly alongside Objective-C++, you might still need C++ headers. #import #import ``` -Or in Swift: +Swift: + +```swift +import ExecuTorch +``` + +#### Example + +Here's a concise example demonstrating how to load a model, prepare input, run inference, and process output using the Objective-C and Swift API. Imagine you have a MobileNet v3 model (`mv3.pte`) that takes a `[1, 3, 224, 224]` float tensor as input and outputs logits. + +Objective-C: + +```objectivec +NSString *modelPath = [[NSBundle mainBundle] pathForResource:@"mv3" ofType:@"pte"]; + +// Create a module with the model file path. Nothing gets loaded into memory just yet. +ExecuTorchModule *module = [[ExecuTorchModule alloc] initWithFilePath:modelPath]; + +NSError *error; // Optional error output argument to learn about failures. + +// Force-load the program and 'forward' method. Otherwise, it's loaded at the first execution. +[module loadMethod:@"forward" error:&error]; + +float *imageBuffer = ...; // Existing image buffer. + +// Create an input tensor referencing the buffer and assuming the given shape and data type. +ExecuTorchTensor *inputTensor = [[ExecuTorchTensor alloc] initWithBytesNoCopy:imageBuffer + shape:@[@1, @3, @224, @224] + dataType:ExecuTorchDataTypeFloat]; + +// Execute the 'forward' method with the given input tensor and get output values back. +NSArray *outputs = [module forwardWithTensor:inputTensor error:&error]; + +// Get the first output value assuming it's a tensor. +ExecuTorchTensor *outputTensor = outputs.firstObject.tensor; + +// Access the output tensor data. +[outputTensor bytesWithHandler:^(const void *pointer, NSInteger count, ExecuTorchDataType dataType) { + float *logits = (float *)pointer; + // Use logits... +}]; +``` + +Swift: + +```swift +let modelPath = Bundle.main.path(forResource: "mv3", ofType: "pte")! + +// Create a module with the model file path. Nothing gets loaded into memory just yet. +let module = Module(filePath: modelPath) + +// Force-load the program and 'forward' method. Otherwise, it's loaded at the first execution. +try module.load("forward") + +let imageBuffer: UnsafeMutableRawPointer = ... // Existing image buffer + +// Create an input tensor referencing the buffer and assuming the given shape and data type. +let inputTensor = Tensor( + bytesNoCopy: imageBuffer, + shape: [1, 3, 224, 224], + dataType: .float +) + +// Execute the 'forward' method with the given input tensor and get output values back. +let outputs = try module.forward(inputTensor) + +// Get the first output value assuming it's a tensor. +if let outputTensor = outputs.first?.tensor { + // Access the output tensor data. + outputTensor.bytes { pointer, count, dataType in + // Copy the tensor data into logits array for easier access. + let logits = Array(UnsafeBufferPointer( + start: pointer.assumingMemoryBound(to: Float.self), + count: count + )) + // Use logits... + } +} +``` + +### Tensor + +The `Tensor` class (exposed as `ExecuTorchTensor` in Objective-C) represents a multi-dimensional array of elements (such as floats or ints) and includes metadata like shape (dimensions) and data type. Tensors are used to feed inputs to a model and retrieve outputs, or for any computation you need to do on raw data. You can create tensors from simple arrays of numbers, inspect their properties, read or modify their contents, and even reshape or copy them. + +#### Key Properties: + +- dataType: The element type (e.g., `.float`, `.int`, `.byte`). +- shape: An array of `NSNumber` describing the size of each dimension. +- count: The total number of elements. +- strides: The jump in memory needed to advance one element along each dimension. +- dimensionOrder: The order of dimensions in memory. +- shapeDynamism: Indicates if the tensor shape can change (`.static`, `.dynamicBound`, `.dynamicUnbound`). + +#### Initialization: + +You can create tensors in various ways: + +From existing memory buffers: +- `init(bytesNoCopy:shape:dataType:...)`: Creates a tensor that references an existing memory buffer without copying. The buffer's lifetime must exceed the tensor's. +- `init(bytes:shape:dataType:...)`: Creates a tensor by copying data from a memory buffer. + +From `NSData` / `Data`: +- `init(data:shape:dataType:...)`: Creates a tensor using an `NSData` object, referencing its bytes without copying. + +From scalar arrays: +- `init(_:shape:dataType:...)`: Creates a tensor from an array of `NSNumber` scalars. Convenience initializers exist to infer shape or data type. + +From single scalars: +- `init(_:)`, `init(_:dataType:)`, `init(float:)`, `init(int:)`, etc.: Create 0-dimensional tensors (scalars). + +Objective-C: + +```objectivec +#import + +// Create from copying bytes. +float data[] = {1.0f, 2.0f, 3.0f, 4.0f}; +NSArray *shape = @[@2, @2]; +ExecuTorchTensor *tensorFromBytes = [[ExecuTorchTensor alloc] initWithBytes:data + shape:shape + dataType:ExecuTorchDataTypeFloat]; + +// Create from scalars. +NSArray *scalars = @[@(1), @(2), @(3)]; +ExecuTorchTensor *tensorFromScalars = [[ExecuTorchTensor alloc] initWithScalars:scalars + dataType:ExecuTorchDataTypeInt]; + +// Create a float scalar tensor. +ExecuTorchTensor *scalarTensor = [[ExecuTorchTensor alloc] initWithFloat:3.14f]; +``` + +Swift: +```swift +import ExecuTorch + +// Create from existing buffer without copying. +var mutableData: [Float] = [1.0, 2.0, 3.0, 4.0] +let tensorNoCopy = mutableData.withUnsafeMutableBytes { bufferPointer in + Tensor( + bytesNoCopy: bufferPointer.baseAddress!, + shape: [2, 2], + dataType: .float + ) +} + +// Create from Data (no copy). +let data = Data(bytes: mutableData, count: mutableData.count * MemoryLayout.size) +let tensorFromData = Tensor(data: data, shape: [2, 2], dataType: .float) + +// Create from scalars (infers float type). +let tensorFromScalars = Tensor([1.0, 2.0, 3.0, 4.0], shape: [4]) + +// Create an Int scalar tensor. +let scalarTensor = Tensor(42) // Infers Int as .long data type (64-bit integer) +``` + +#### Accessing Data: + +Use `bytes(_:)` for immutable access and `mutableBytes(_:)` for mutable access to the tensor's underlying data buffer. + +Objective-C: + +```objectivec +[tensor bytesWithHandler:^(const void *pointer, NSInteger count, ExecuTorchDataType dataType) { + if (dataType == ExecuTorchDataTypeFloat) { + const float *floatPtr = (const float *)pointer; + NSLog(@"First float element: %f", floatPtr[0]); + } +}]; + +[tensor mutableBytesWithHandler:^(void *pointer, NSInteger count, ExecuTorchDataType dataType) { + if (dataType == ExecuTorchDataTypeFloat) { + float *floatPtr = (float *)pointer; + floatPtr[0] = 100.0f; // Modify the original mutableData buffer. + } +}]; +``` + +Swift: +```swift +tensor.bytes { pointer, count, dataType in + if dataType == .float { + let buffer = UnsafeBufferPointer(start: pointer.assumingMemoryBound(to: Float.self), count: count) + print("First float element: \(buffer.first ?? 0.0)") + } +} + +tensor.mutableBytes { pointer, count, dataType in + if dataType == .float { + let buffer = UnsafeMutableBufferPointer(start: pointer.assumingMemoryBound(to: Float.self), count: count) + buffer[1] = 200.0 // Modify the original mutableData buffer. + } +} +``` + +#### Resizing: + +Tensors can be resized if their underlying memory allocation allows it (typically requires ShapeDynamism other than Static or sufficient capacity). + +Objective-C: + +```objectivec +NSError *error; +BOOL success = [tensor resizeToShape:@[@4, @1] error:&error]; +if (success) { + NSLog(@"Resized shape: %@", tensor.shape); +} else { + NSLog(@"Resize failed: %@", error); +} +``` + +Swift: +```swift +do { + try tensor.resize(to: [4, 1]) + print("Resized shape: \(tensor.shape)") +} catch { + print("Resize failed: \(error)") +} +``` + +### Value + +The `Value` class (exposed as `ExecuTorchValue` in Objective-C) is a dynamic container that can hold different types of data, primarily used for model inputs and outputs. ExecuTorch methods accept and return arrays of `Value` objects. + +#### Key Properties: + +- `tag`: Indicates the type of data held (e.g., `.tensor`, `.integer`, `.string`, `.boolean`). +- `isTensor`, `isInteger`, `isString`, etc.: Boolean checks for the type. +- `tensor`, `integer`, `string`, `boolean`, `double`: Accessors for the underlying data (return `nil` or a default value if the tag doesn't match). + +#### Initialization: + +Create Value objects directly from the data they should hold. + +Objective-C: + +```objectivec +#import + +ExecuTorchTensor *tensor = [[ExecuTorchTensor alloc] initWithFloat:1.0f]; + +ExecuTorchValue *tensorValue = [[ExecuTorchValue alloc] valueWithTensor:tensor]; +ExecuTorchValue *intValue = [[ExecuTorchValue alloc] valueWithInteger:100]; +ExecuTorchValue *stringValue = [[ExecuTorchValue alloc] valueWithString:@"hello"]; +ExecuTorchValue *boolValue = [[ExecuTorchValue alloc] valueWithBoolean:YES]; +ExecuTorchValue *doubleValue = [[ExecuTorchValue alloc] valueWithDouble:3.14]; +``` + +Swift: + +```swift +import ExecuTorch + +let tensor = Tensor(2.0) + +let tensorValue = Value(tensor) +let intValue = Value(200) +let stringValue = Value("world") +let boolValue = Value(false) +let doubleValue = Value(2.718) +``` + +### Module +The `Module` class (exposed as `ExecuTorchModule` in Objective-C) represents a loaded ExecuTorch model (`.pte` file). It provides methods to load the model program and execute its internal methods (like `forward`). + +#### Initialization: + +Create a `Module` instance by providing the file path to the `.pte` model. Initialization itself is lightweight and doesn't load the program data immediately. + +Objective-C: + +```objectivec +#import + +NSString *modelPath = [[NSBundle mainBundle] pathForResource:@"model" ofType:@"pte"]; +ExecuTorchModule *module = [[ExecuTorchModule alloc] initWithFilePath:modelPath]; +// Optional: specify load mode, e.g., memory mapping. +// ExecuTorchModule *moduleMmap = [[ExecuTorchModule alloc] initWithFilePath:modelPath +// loadMode:ExecuTorchModuleLoadModeMmap]; +``` + +Swift: ```swift import ExecuTorch + +let modelPath = Bundle.main.path(forResource: "model", ofType: "pte") +let module = Module(filePath: modelPath!) +// Optional: specify load mode, e.g., memory mapping. +// let moduleMmap = Module(filePath: modelPath, loadMode: .mmap) ``` -**Note:** Importing the ExecuTorch umbrella header (or ExecuTorch module in Swift) provides access to the logging API only. You still need to import the other runtime headers explicitly as needed, e.g., `module.h`. There is no support for other runtime APIs in Objective-C or Swift beyond logging described below. +#### Loading: -**Note:** Logs are stripped in the release builds of ExecuTorch frameworks. To preserve logging, use debug builds during development. +Model loading is deferred until explicitly requested or needed for execution. While execution calls can trigger loading automatically, it's often more efficient to load methods explicitly beforehand. + +- `load()`: Loads the basic program structure. Minimal verification is used by default. +- `load(_:)`: Loads the program structure and prepares a specific method (e.g., "forward") for execution. This performs necessary setup like backend delegation and is recommended if you know which method you'll run. +- `isLoaded()` / `isLoaded(_:)`: Check loading status. + +Objective-C: + +```objectivec +NSError *error; +// Loads program and prepares 'forward' for execution. +BOOL success = [module loadMethod:@"forward" error:&error]; +if (success) { + NSLog(@"Forward method loaded: %d", [module isMethodLoaded:@"forward"]); +} else { + NSLog(@"Failed to load method: %@", error); +} +``` + +Swift: + +```swift +do { + // Loads program and prepares 'forward' for execution. + try module.load("forward") + print("Forward method loaded: \(module.isLoaded("forward"))") +} catch { + print("Failed to load method: \(error)") +} +``` + +#### Execution: + +The `Module` class offers flexible ways to execute methods within the loaded program. + +- Named Execution: You can execute any available method by name using `execute(methodName:inputs:)`. +- Forward Shortcut: For the common case of running the primary inference method, use the `forward(inputs:)` shortcut, which is equivalent to calling execute with the method name "forward". +- Input Flexibility: Inputs can be provided in several ways: + - As an array of `Value` objects. This is the most general form. + - As an array of `Tensor` objects. This is a convenience where tensors are automatically wrapped into `Value` objects. + - As a single `Value` or `Tensor` object if the method expects only one input. + - With no inputs if the method takes none. + +Outputs are always returned as an array of `Value`. + +Objective-C: + +```objectivec +ExecuTorchTensor *inputTensor1 = [[ExecuTorchTensor alloc] initWithScalars:@[@1.0f, @2.0f]]; +ExecuTorchTensor *inputTensor2 = [[ExecuTorchTensor alloc] initWithScalars:@[@3.0f, @4.0f]]; +ExecuTorchTensor *singleInputTensor = [[ExecuTorchTensor alloc] initWithFloat:5.0f]; +NSError *error; + +// Execute "forward" using the shortcut with an array of Tensors. +NSArray *outputs1 = [module forwardWithTensors:@[inputTensor1, inputTensor2] error:&error]; +if (outputs1) { + NSLog(@"Forward output count: %lu", (unsigned long)outputs1.count); +} else { + NSLog(@"Execution failed: %@", error); +} + +// Execute "forward" with a single Tensor input. +NSArray *outputs2 = [module forwardWithTensor:singleInputTensor error:&error]; +if (outputs2) { + NSLog(@"Forward single input output count: %lu", (unsigned long)outputs2.count); +} else { + NSLog(@"Execution failed: %@", error); +} + +// Execute a potentially different method by name. +NSArray *outputs3 = [module executeMethod:@"another_method" + withInput:[[ExecuTorchValue alloc] valueWithTensor:inputTensor1] + error:&error]; + +// Process outputs (assuming first output is a tensor). +if (outputs1) { + ExecuTorchValue *firstOutput = outputs1.firstObject; + if (firstOutput.isTensor) { + ExecuTorchTensor *resultTensor = firstOutput.tensorValue; + // Process resultTensor. + } +} +``` + +Swift: + +```swift +let inputTensor1 = Tensor([1.0, 2.0], dataType: .float) +let inputTensor2 = Tensor([3.0, 4.0], dataType: .float) +let singleInputTensor = Tensor([5.0], dataType: .float) + +do { + // Execute "forward" using the shortcut with an array of Tensors. + let outputs1 = try module.forward([inputTensor1, inputTensor2]) + print("Forward output count: \(outputs1.count)") + + // Execute "forward" with a single Tensor input. + let outputs2 = try module.forward(singleInputTensor) + print("Forward single input output count: \(outputs2.count)") + + // Execute a potentially different method by name. + let outputs3 = try module.execute("another_method", inputs: [Value(inputTensor1)]) + + // Process outputs (assuming first output is a tensor). + if let resultTensor = outputs1.first?.tensor { + resultTensor.bytes { ptr, count, dtype in + // Access result data. + } + } +} catch { + print("Execution failed: \(error)") +} +``` + +#### Method Names: + +You can query the available method names in the model after the program is loaded. + +Objective-C: + +```objectivec +NSError *error; + +// Note: methodNames: will load the program if not already loaded. +NSSet *names = [module methodNames:&error]; +if (names) { + NSLog(@"Available methods: %@", names); +} else { + NSLog(@"Could not get method names: %@", error); +} +``` + +Swift: + +```swift +do { + // Note: methodNames() will load the program if not already loaded. + let names = try module.methodNames() + print("Available methods: \(names)") // Output: e.g., {"forward"} +} catch { + print("Could not get method names: \(error)") +} +``` ### Logging -ExecuTorch provides extra APIs for logging in Objective-C and Swift as a lightweight wrapper of the internal ExecuTorch machinery. To use it, just import the main framework header in Objective-C. Then use the `ExecuTorchLog` interface (or the `Log` class in Swift) to subscribe your own implementation of the `ExecuTorchLogSink` protocol (or `LogSink` in Swift) to listen to log events. +ExecuTorch provides APIs for logging in Objective-C and Swift via the `ExecuTorchLog` (`Log` in Swift) singleton. You can subscribe custom log sinks conforming to the `ExecuTorchLogSink` (`LogSink` in Swift) protocol to receive internal ExecuTorch log messages. + +**Note:** Logs are stripped in the Release builds of ExecuTorch frameworks. To capture logs, link against the Debug builds (e.g., `executorch_debug`) during development. + +Objective-C: ```objectivec #import @@ -250,7 +692,7 @@ ExecuTorch provides extra APIs for logging in Objective-C and Swift as a lightwe @end ``` -Swift version: +Swift: ```swift import ExecuTorch