Build hook spec

pkgs/native_assets_builder/doc/protocol/v1.md

@@ -0,0 +1,216 @@
+Build Hook Specification
+========================
+
+Version 1: As of 2024-01-24 implemented behind `--enable-experiment=native-assets`
+
+### Concepts
+
+#### Asset
+A file that is identified by an `assetId`. There may be multiple files with the
+same `assetId` with different characteristics such as `target` or `link_mode`.
+
+Note that in v1 there are _only_ native code assets, no data assets.
+
+#### AssetId
+An asset must have an `assetId`. Dart code that uses an asset, references the
+asset using the `assetId`.
+
+A package MUST prefix all assetIds it defines with: `package:<package>/`, this
+ensures assets don't conflict between packages.
+
+Conventionally, an asset referenced from `lib/src/foo.dart` in `package:foo`
+has `assetId = 'package:foo/src/foo.dart'`.
+
+#### Target
+
+A target specifies operating system and architecture.
+
+List of `platform` and `architecture` tuples supported by the
+Dart and Flutter SDKs to be specified below:
+ * `linux_x64`
+ * TODO: enumerate full list of supported platform/architecture tuples.
+
+
+
+### `build.dart`
+A package with native assets MUST define a `build.dart` file which accepts
+a `--config` option as follows:
+
+```console
+$ dart build.dart --config <build_config.yaml>
+```
+
+The `--config` option specifies location of the `build_config.yaml` file.
+
+The `build.dart` file MUST:
+ * Read the `build_config.yaml` file,
+ * Build assets using configuration from `build_config.yaml`.
+   * If `dry_run: true` in `build_config.yaml`, then this may be skipped.
+ * Write assets into `out_dir` from `build_config.yaml`.
+   * If `dry_run: true` in `build_config.yaml`, then this may be skipped.
+   * Filename are unrelated to `assetId`.
+   * Arbitrary file names are fine, `build_output.yaml` will map `assetId` to files. 
+   * MUST avoid file name `build_output.yaml`.
+ * Write `build_output.yaml` into `out_dir` (from `build_config.yaml`).
+   * This maps `assetId` to assets previous written in `out_dir`.
+   * There may be multiple assets for a given `assetId` depending on
+     characteristics like `target`, `link_mode`, etc.
+   * If `dry_run: true` in `build_config.yaml`, the list of assets that would be
+     generated must this be enumerated. Mapping `assetId`s to non-existing files
+     is expected.
+
+
+### `build_config.yaml`
+Configuration file passed to `build.dart` as the `--config` option.
+
+```yaml
+# Build in dry-run mode.
+#
+# Running in dry-run mode `<out_dir>/build_output.yaml` must be written, but
+# the files it references need not exist.
+dry_run: true | false
+
+# Build Mode.
+#
+# A hint `build.dart` can use to determined which optimizations to enable and
+# whether or not to include debug symbols in the format relevant for the asset.
+#
+# Not provided on dry runs.
+build_mode: release | debug
+
+# Metadata as output from build.dart from direct dependencies.
+#
+# Not provided on dry runs.
+dependency_metadata:
+  # package name of direct dependency.
+  some_package_name:
+    # key value pairs.
+    some_key: some_value
+
+# Preferred link mode
+link_mode_preference: dynamic | static | prefer-dynamic | prefer-static
+
+# Path to output directory where assets should be placed.
+#
+# This is also where `build_output.yaml` should be written.
+#
+# Remark: Avoid using the name "build_output.yaml" for an asset file, this is
+#         forbidden.
+out_dir: /absolute/path/to/out_dir/
+
+# Name of the package that contains the `build.dart`
+#
+# Remark: This is entirely redundant since this is a config file specified to
+# `build.dart`, and the author of `build.dart` probably knows the name of the
+# package they are writing.
+package_name: my_package_with_native_assets
+
+# Path to root folder for the package that contains `build.dart`.
+#
+# This is useful if `build.dart` wishes to find source code files embedded in
+# its own package and compile them to an asset.
+#
+# Note that this will be most likely a path in the pub cache when the package
+# with a `build.dart` is a dependency of another package.
+package_root: /absolute/path/to/my_package_with_native_assets
+
+# Target architecture
+#
+# Combined with `target_os` this specifies the "target" for which assets
+# should be built.
+#
+# Not provided on dry runs.
+target_architecture: x64 | ia32 | arm | arm64 | riscv32 | riscv64
+
+# Target operating system
+#
+# Combined with `target_architecture` this specifies the "target" for which
+# assets should be built.
+target_os: android | ios | linux | macos | windows
+
+# Version of this file.
+version: 1.0.0
+```
+
+### `build_output.yaml`
+Mapping from `assetId`s to files created by `build.dart`.
+
+```yaml
+# The list of assets.
+#
+# In dry runs, must contain assets for each architecture for the requested os.
+assets:
+  - id: 'package:my_package_with_native_assets/src/foo.dart'
+    link_mode: dynamic | static | prefer-dynamic | prefer-static
+    path:
+      path_type: absolute | system | process | executable
+      # Only provided for path_type absolute and system.
+      #
+      # If path_type absolute: The absolute path to the file name.
+      #
+      # If path_type system: The path of the dynamic library as available on 
+      # the target machine's PATH.
+      uri: /absolute/path/to/outdir/arbitrary_filename.whatever
+    target: linux_x64
+  ...
+
+# The files used by this build.
+#
+# If any of the files in [dependencies2] are modified after [timestamp], the
+# build will be re-run.
+#
+# Not output on dry runs.
+dependencies:
+  - /absolute/path/to/my_package_with_native_assets/build.dart
+  ...
+
+# The time the build this output is for started.
+#
+# Must be before any of the files in dependencies are read.
+timestamp: 2024-01-02 17:05:35.000
+
+# Metadata usable for build.dart of packages directly depending on this package.
+#
+# Not output in dry runs.
+metadata:
+  # Key value pairs.
+  some_key: some_value
+
+# Version of this file.
+version: 1.0.0
+```
+
+
+### `.dart_tool/native_assets.yaml`
+
+This file is internal to the Dart SDK and Flutter SDK.
+It produced by `native_asset_builder` (as embedded in the SDKs).
+For each `target` it maps from `assetId` to `url`
+(with `urlKind` indicating if it's a relative or absolute path).
+
+```yaml
+# Version of this file.
+format-version: [1, 0, 0]
+
+# A mapping from target and assetId to an asset.
+native-assets:
+  # A <target>.
+  linux_x64:
+    # An <assetId>.
+    : 'package:my_package_with_native_assets/src/foo.dart'
+      # A list of path_type and optionally a path.
+      - absolute | relative | system | process | executable
+      - <url> # Only provided for absolute, relative, and system path types.
+  ...
+```
+
+This file is used by the embedded SDK to resolve `assetId`s when running in
+development mode or build a deployable application (or standalone executable).
+
+## Wishes to improve for v2
+
+* Multiple asset types. Rename `Asset` to `NativeCodeAsset` and introduce `DataAsset`s.
+* The path_types really only make sense for `link_mode: dynamic`.
+* Fix the mismatch between `target` for output but `os` and `architecture` for config.
+  * Should the architecture field of native code assets be optional and be ommitted in dry runs?
+* Introduce a `link.dart` protocol with a link input and link output.