From 3de9426d4bcd1a363d2942c9c8c961c5eb181436 Mon Sep 17 00:00:00 2001 From: Alex Crichton Date: Tue, 6 Dec 2022 12:13:23 -0800 Subject: [PATCH 01/12] Modernize the WIT.md description This commit brings the `WIT.md` file up-to-date with recent developments in WIT. I've added a new "introduction section" which is a higher-level explanation of the format than the lexical/syntactic structure. I've also edited the syntactic structure to clarify a few old references and remove `resource`, `future`, and `stream` types for now. (resources to be added soon I suspect) Additionally I've added a description of a binary format for WIT based on the component model. This binary format is intended to be the "types only" mode of the `wasm-tools component` tooling, although it doesn't match the current implementation and the `wasm-tools` repo will need to be updated. This representation, however, provides the means by which to understand what it means for a component to have the type of a `world`, namely it's a subtype of the binary representation's component type. --- design/mvp/WIT.md | 752 +++++++++++++++++++++++++++++++++++++++------- 1 file changed, 644 insertions(+), 108 deletions(-) diff --git a/design/mvp/WIT.md b/design/mvp/WIT.md index 7962d33f..070c7614 100644 --- a/design/mvp/WIT.md +++ b/design/mvp/WIT.md @@ -1,28 +1,518 @@ # The `wit` format -This is intended to document the `wit` format as it exists today. The goal is -to provide an overview to understand what features `wit` files give you and how -they're structured. This isn't intended to be a formal grammar, although it's -expected that one day we'll have a formal grammar for `wit` files. +The Wasm Interface Type (WIT) text format is a developer-friendly format to +describe the imports and exports of a component. The WIT format can be thought +of as an IDL of sorts to describe APIs that are grouped together as +[`interface`s][interfaces] inside of a [`world`][worlds]. WIT text files are +shared between producers of components and consumers of components as a format +for bindings generation in host and guest languages. The WIT text format +additionally provides a developer-friendly way to inspect an existing +component and learn about its imports and exports. + +The WIT text format uses the file extension `wit`, for example `foo.wit` is +a WIT document. The contents of a `*.wit` file must be valid utf-8 bytes. WIT +documents can contain two items at the top-level: [`interface`][interfaces] and +[`world`][worlds]. + +## WIT Interfaces +[interfaces]: #wit-interfaces + +An `interface` in WIT is a collection of [functions] and [types] which +corresponds to an instance in the component model. Interfaces can either be +imported or exported from [worlds] and represent imported and exported +instances. For example this `interface`: -If you're curious to give things a spin try out the [online -demo](https://bytecodealliance.github.io/wit-bindgen/) of `wit-bindgen` where -you can input `wit` on the left and see output of generated bindings for -languages on the right. If you're looking to start you can try out the -"markdown" output mode which generates documentation for the input document on -the left. +```wit +interface host { + log: func(msg: string) +} +``` + +represents an interface called `host` which provides one function, `log`, which +takes a single `string` argument. If this were imported into a component then it +would correspond to: + +```wasm +(component + (import "host" (insance $host + (export "log" (func (param "msg" string))) + )) + ;; ... +) +``` + +An `interface` can contain [`use`][use] statements, [type][types] definitions, +and [function][functions] definitions. For example: + +```wit +interface wasi-fs { + use { errno } from "wasi-types" + + record stat { + ino: u64, + size: u64, + // ... + } + + stat-file: func(path: string) -> result +} +``` + +More information about [`use`][use] and [types] are described below, but this +is an example of a collection of items within an `interface`. All items defined +in an `interface`, including [`use`][use] items, are considered as exports from +the interface. This means that types can further be used from the interface by +other interfaces. An interface has a single namespace which means that none of +the defined names can collide. + +A WIT document can contain any number of interfaces listed at the top-level and +in any order. The WIT validator will ensure that all references between +documents are well-formed and acyclic. + +## WIT Worlds +[worlds]: #wit-worlds + +WIT documents can contain a `world` annotation at the top-level in addition to +[`interface`][interfaces]. A world is a complete description of both imports and +exports of a component. A world can be thought of as an equivalent of a +`component` type in the component model. For example this world: + +```wit +world my-world { + import host: interface { + log: func(param: string) + } + + export run: func() +} +``` + +can be thought of as this component type: + +```wasm +(type $my-world (component + (import "host" (instance + (export "log" (func (param "param" string))) + )) + (export "run" (func)) +)) +``` + +Worlds describe a concrete component and are the basis of bindings generation. A +guest language will use a `world` to determine what functions are imported, what +they're named, and what functions are exported, in addition to their names. + +Worlds can contain any number of imports and exports, and can be either a +function or an interface. + +```wit +world wasi { + import wasi-fs: "wasi-fs" + import wasi-random: "wasi-random" + import wasi-clock: "wasi-clock" + // ... + + export command: func(args: list) +} +``` + +Additionally interfaces can be defined "inline" as a form of sugar for defining +it at the top-level + +```wit +interface out-of-line { + the-function: func() +} + +world your-world { + import out-of-line: out-of-line + // ... is equivalent to ... + import out-of-line2: interface { + the-function: func() + } +} +``` + +The name of the `import` or `export` is the name of the corresponding item in +the final component. This can be different from the [`interface`][interfaces] +name but must be a [valid identifier][identifiers]. + +There can be multiple `world` descriptions in a single WIT document. When +generating bindings from a WIT document one world must be marked as `default` or +an explicitly named world must be chosen: + +```wit +default world my-world { +} +``` + +If no `default` world is specified in the WIT document and no named world is +explicitly chosen then bindings cannot be generated. + +## WIT File Organization +[use]: #wit-file-organization + +WIT files can be organized into separate files for maintainability and +reusability. This enables a sort of module system for WIT syntax where files may +import from one another. + +> **Note**: The precise semantics of imports and how everything maps out is +> still being design. Basic filesystem-based organization works but it's +> intended to extend to URL-based organization in the near future. For example +> the strings below are intended to integrate into a registry-based workflow as +> well in addition to looking up files on the filesystem. + +Within a single WIT file the `use` statement can be used to import between +interfaces: + +```wit +interface types { + enum errno { /* ... */ } + + type size = u32 +} + +interface my-host-functions { + use { errno, size } from types +} +``` + +Here the `from` directive of the `use` is not quoted which means that it's +resolved relative to the WIT file itself. The `interface types` may come either +after or before the `use` directive's `interface`. + +The `use` directive is not allowed to create cycles and must always form an +acyclic graph of dependencies between interfaces. + +Names imported via `use` can be renamed as they're imported as well: + +```wit +interface my-host-functions { + use { errno as my-errno } from types +} +``` + +When importing or exporting an [interface][interfaces] in a [world][worlds] +unquoted names refer to [interfaces] defined elsewhere in the file: + +```wit +world my-world { + import host: host +} + +interface host { + // ... +} +``` + +The target of a `use` or the type of an import/export in a `world` may also be a +quoted string: + +```wit +interface foo { + use { /* ... */ } from "./other-file" +} + +world my-world { + import some-import: "./other-file" + import another-import: "./other-directory/other-file" +} +``` + +This quoted string form indicates that the import is located in a different WIT +file. At this time all quoted strings must start with `./` or `../` and refer to +a file relative to the location of the current WIT file. This will eventually be +expanded to allow something along the lines of `"wasi:fs"` or similar but the +precise semantics here have not been defined. + +The above directives, for example, will import from `./other-file.wit` (or +`./other-file.wit.md` as described [below][markdown]) and +`./other-directory/other-file.wit` (or `./other-directory/other-file.wit.md`). +The `.wit` extension is automatically appended to the lookup path and `.wit.md` +is tested if the `.wit` file doesn't exist. + +Additionally the above directives require that `other-file.wit` contains an +interface marked `default`. Similar to [`default` worlds][worlds] a WIT file may +have multiple [`interface`s][interfaces] inside it and the import must happen +from one of them, so `default` is used to disambiguate: + +```wit +// other-file.wit +default interface foo { + // ... +} + + +// my-file.wit +default interface foo { + use { /* ... */ } from "./other-file" +} +``` + +A `use` directive can also explicitly list the requested interface to select a +non-`default` one: + +```wit +// other-file.wit +interface foo { + // ... +} + + +// my-file.wit +default interface foo { + use { /* ... */ } from foo in "./other-file" +} +``` + +Like before within a WIT file `use` statements must be acyclic between files as +well. + +Splitting a WIT document into multiple files does not have an impact on its +final structure and it's purely a convenience to developers. Resolution of a WIT +document happens as-if everything were contained in one document (modulo +"hygienic" renaming where `use`-with-identifier still only works within one +file). This means that a WIT document can always be represented as a single +large WIT document with everything contained and separate-file organization is +not necessary. + +### Transitive imports and worlds + +A `use` statement is not implemented by copying type information around but +instead retains that it's a reference to a type defined elsewhere. This +representation is plumbed all the way through to the final component, meaning +that `use`d types have an impact on the structure of the final generated +component. + +For example this document: + +```wit +interface shared { + record metadata { + // ... + } +} + +world my-world { + import host: interface { + use { metadata } from shared + + get: func() -> metadata + } +} +``` + +would generate this component : + +```wasm +(component + (import "shared" (instance $shared + (type $metadata (record (; ... ;))) + (export "metadata" (type (eq $metadata))) + )) + (alias export $shared "metadata" (type $metadata)) + (import "host" (instance $host + (export "get" (func (result $metadata))) + )) +) +``` + +Here it can be seen that despite the `world` only listing `host` as an import +the component additionally import a `shared` instance. This is due to the fact +that the `use { ... } from shared` implicitly requires that `shared` is imported +to the component as well. + +Note that the name `"shared"` here is derived from the name of the `interface` +which can also lead to conflicts: + +```wit +// shared1.wit +default interface shared { /* ... */ } + +// shared2.wit +default interface shared { /* ... */ } + +// world.wit +world my-world { + import foo: interface { + use { a-type } from "shared1" + } + import bar: interface { + use { other-type } from "shared2" + } +} +``` + +This is an invalid WIT document due because `my-world` needs to import two +unique interfaces called `shared`. To disambiguate a manual import is required: + +``` +world my-world { + import shared1: "shared1" + import shared2: "shared1" + + import foo: interface { + use { a-type } from "shared1" + } + import bar: interface { + use { other-type } from "shared2" + } +} +``` + +For `export`ed interfaces any transitively `use`d interface is assumed to be an +import unless it's explicitly listed as an export. + +> **Note**: It's planned in the future to have "power user syntax" to configure +> this on a more fine-grained basis for exports, for example being able to +> configure that a `use`'d interface is a particular import or a particular +> export. + +## WIT Functions +[functions]: #wit-functions + +Functions are defined in an [`interface`][interfaces] or are listed as an +`import` or `export` from a [`world`][worlds]. Parameters to a function must all +be named and have unique names: + +```wit +interface foo { + a1: func() + a2: func(x: u32) + a3: func(y: u64, z: float32) +} +``` + +Functions can return at most one unnamed type: + +```wit +interface foo { + a1: func() -> u32 + a2: func() -> string +} +``` + +And functions can also return multiple types by naming them: + +```wit +interface foo { + a: func() -> (a: u32, b: float32) +} +``` + +Note that returning multiple values from a function is not equivalent to +returning a tuple of values from a function. These options are represented +distinctly in the component binary format. + +## WIT Types +[types]: #wit-types + +Types in WIT files can only be defined in [`interface`s][interfaces] at this +time. The types supported in WIT is the same set of types supported in the +component model itself: + +```wit +interface foo { + // "package of named fields" + record r { + a: u32, + b: string, + } + + // values of this type will be one of the specified cases + variant human { + baby, + child(u32), // optional type payload + adult, + } + + // similar to `variant`, but no type payloads + enum errno { + too-big, + too-small, + too-fast, + too-slow, + } + + // similar to `variant`, but doesn't require naming cases and all variants + // have a type payload -- note that this is not a C union, it still has a + // discriminant + union input { + u64, + string, + } + + // a bitflags type + flags permissions { + read, + write, + exec, + } + + // type aliases are allowed to primitive types and additionally here are some + // examples of other types + type t1 = u32 + type t2 = tuple + type t3 = string + type t4 = option + type t5 = result<_, errno> // no "ok" type + type t6 = result // no "err" type + type t7 = result // both types specified + type t8 = result // no "ok" or "err" type + type t9 = list + type t10 = t9 +} +``` + +The `record`, `variant`, `enum`, `union`, and `flags` types must all have names +associated with them. The `list`, `option`, `result`, `tuple`, and primitive +types do not need a name and can be mentioned in any context. This restriction +is in place to assist with code generation in all languages to leverage +language-builtin types where possible while accommodating types that need to be +defined within each language as well. + +## WIT Identifiers +[identifiers]: #wit-identifiers + +Identifiers in WIT documents are required to be valid component identifiers, +meaning that they're "kebab cased". This currently is restricted to ascii +characters and numbers that are `-` separated. + +For more information on this see the [binary format](./Binary.md). + +## WIT in Markdown +[markdown]: #wit-in-markdown + +The WIT text format can also additionally be parsed from markdown files with the +extension `wit.md`, for example `foo.wit.md`: + + # This would be -## Lexical structure + Some markdown text + + ```wit + // interspersed with actual `*.wit` + interface my-interface { + ``` + + ```wit + // which can be broken up between multiple blocks + } + ``` + +Triple-fence code blocks with the `wit` marker will be extracted from a markdown +file and concatenated into a single string which is then parsed as a normal +`*.wit` file. + +# Lexical structure The `wit` format is a curly-braced-based format where whitespace is optional (but -recommended). It is intended to be easily human readable and supports features -like comments, multi-line comments, and custom identifiers. A `wit` document -is parsed as a unicode string, and when stored in a file is expected to be -encoded as UTF-8. +recommended). A `wit` document is parsed as a unicode string, and when stored in +a file is expected to be encoded as utf-8. -Additionally, wit files must not contain any bidirectional override scalar values, -control codes other than newline, carriage return, and horizontal tab, or -codepoints that Unicode officially deprecates or strongly discourages. +Additionally, wit files must not contain any bidirectional override scalar +values, control codes other than newline, carriage return, and horizontal tab, +or codepoints that Unicode officially deprecates or strongly discourages. The current structure of tokens are: @@ -115,6 +605,7 @@ keyword ::= 'use' | 'world' | 'import' | 'export' + | 'default' ``` ## Top-level items @@ -154,7 +645,7 @@ Interfaces can be defined in a `wit` document. Interfaces have a name and a sequ Specifically interfaces have the structure: ```wit -interface-item ::= 'interface' id strlit? '{' interface-items* '}' +interface-item ::= 'default'? 'interface' id '{' interface-items* '}' interface-items ::= resource-item | variant-items @@ -168,12 +659,13 @@ interface-items ::= resource-item func-item ::= id ':' func-type -func-type ::= 'func' param-list '->' result-list +func-type ::= 'func' param-list result-list param-list ::= '(' named-type-list ')' -result-list ::= ty - | '(' named-type-list ')' +result-list ::= nil + | '->' ty + | '->' '(' named-type-list ')' named-type-list ::= nil | named-type ( ',' named-type )* @@ -189,13 +681,13 @@ wit documents. The structure of a use statement is: ```wit use * from other-file use { a, list, of, names } from another-file -use { name as other-name } from yet-another-file +use { name as other-name } from interface in "yet-another-file" ``` Specifically the structure of this is: ```wit -use-item ::= 'use' use-names 'from' id +use-item ::= 'use' use-names 'from' use-from use-names ::= '*' | '{' use-names-list '}' @@ -205,6 +697,10 @@ use-names-list ::= use-names-item use-names-item ::= id | id 'as' id + +use-from ::= id + | strlit + | id 'in' strlit ``` Note: Here `use-names-list?` means at least one `use-name-list` term. @@ -264,10 +760,9 @@ record-field ::= id ':' ty ### Item: `flags` (bag-of-bools) -A `flags` statement defines a new `record`-like structure where all the fields -are booleans. The `flags` type is distinct from `record` in that it typically is -represented as a bit flags representation in the canonical ABI. For the purposes -of type-checking, however, it's simply syntactic sugar for a record-of-booleans. +A `flags` represents a bitset structure with a name for each bit. The `flags` +type is represented as a bit flags representation in +the canonical ABI. ```wit flags properties { @@ -275,14 +770,6 @@ flags properties { marvel-superhero, supervillan, } - -// type-wise equivalent to: -// -// record properties { -// lego: bool, -// marvel-superhero: bool, -// supervillan: bool, -// } ``` Specifically the structure of this is: @@ -341,16 +828,6 @@ enum color { yellow, other, } - -// type-wise equivalent to: -// -// variant color { -// red, -// green, -// blue, -// yellow, -// other, -// } ``` Specifically the structure of this is: @@ -366,22 +843,14 @@ enum-cases ::= id A `union` statement defines a new type which is semantically equivalent to a `variant` where all of the cases have a payload type and the case names are -numerical. This is special-cased, however, to possibly have a different -representation in the language ABIs or have different bindings generated in for -languages. +numerical. This is special-cased, however, to have a different representation +in the language ABIs or have different bindings generated in for languages. ```wit union configuration { string, list, } - -// type-wise equivalent to: -// -// variant configuration { -// 0(string), -// 1(list), -// } ``` Specifically the structure of this is: @@ -393,42 +862,6 @@ union-cases ::= ty | ty ',' union-cases? ``` -## Item: `resource` - -Resources represent a value that has a hidden representation not known to the -outside world. This means that the resource is operated on through a "handle" (a -pointer of sorts). Resources also have ownership associated with them and -languages will have to manage the lifetime of resources manually (they're -similar to file descriptors). - -Resources can also optionally have functions defined within them which adds an -implicit "self" argument as the first argument to each function of the same type -of the including resource, unless the function is flagged as `static`. - -```wit -resource file-descriptor - -resource request { - static new: func() -> request - - body: func() -> future> - headers: func() -> list -} -``` - -Specifically resources have the structure: - -```wit -resource-item ::= 'resource' id resource-contents - -resource-contents ::= nil - | '{' resource-defs '}' - -resource-defs ::= resource-def resource-defs? - -resource-def ::= 'static'? func-item -``` - ## Types As mentioned previously the intention of `wit` is to allow defining types @@ -455,8 +888,6 @@ ty ::= 'u8' | 'u16' | 'u32' | 'u64' | list | option | result - | future - | stream | id tuple ::= 'tuple' '<' tuple-list '>' @@ -471,14 +902,6 @@ result ::= 'result' '<' ty ',' ty '>' | 'result' '<' '_' ',' ty '>' | 'result' '<' ty '>' | 'result' - -future ::= 'future' '<' ty '>' - | 'future' - -stream ::= 'stream' '<' ty ',' ty '>' - | 'stream' '<' '_' ',' ty '>' - | 'stream' '<' ty '>' - | 'stream' ``` The `tuple` type is semantically equivalent to a `record` with numerical fields, @@ -577,9 +1000,122 @@ record bar2 { } ``` -The intention of `wit` is that it maps down to interface types, so the goal of -name resolution is to effectively create the type section of a wasm module using -interface types. The restrictions about self-referential types and such come -from how types can be defined in the interface types section. Additionally -definitions of named types such as `record foo { ... }` are intended to map -roughly to declarations in the type section of new types. +# Binary Format + +In addition to a textual format WIT files can also be encoded to a binary +format. The binary format is itself a WebAssembly component and is chosen to be +a more stable artifact over time than the text format in case text format +updates are required. Additionally being represented by the component binary +format means that it's guaranteed that all features of WIT are representable in +the component model. + +The binary format for a WIT document can also be thought of in a loose sense as +a fully-resolved and indexed representation of a WIT document. Constructs such +as `use` are preserved but all name resolution has been boiled away to index +references. Additionally the transitive import set is all finalized as well. + +An example of the binary format is that this document: + +```wit +interface host { + log: func(arg: string) +} + +world the-world { + import host: host + export run: func() +} +``` + +would correspond to: + +```wasm +(component + (type $wit (instance + (type $host (instance + (export "log" (func (param "arg" string))) + )) + (export "host" (type $host)) + + (type $the-world (component + (import "host" (instance (type $host))) + (export "run" (func)) + )) + (export "the-world" (type $the-world)) + )) + (export "wit" (type $wit)) +) +``` + +Here it can be seen that the entire document itself is represented as an +`instance` type with named exports. Within this `instance` type each export is +either itself an `instance` or a `component` with `instance`s corresponding to +an `interface` in WIT and `component`s corresponding to `world`s. The outer +component has a single export named `wit` which points to the instance type that +describes the document. + +Types defined within a WIT interface are additionally defined within the wasm +`instance`. Additionally `use` of types between interfaces is encoded with +aliases between them. + +```wit +interface shared { + record metadata { + // ... + } +} + +world the-world { + import host: interface { + use { metadata } from shared + + get: func() -> metadata + } + + export db: interface { + use { metadata } from shared + + get: func(data: metadata) -> string + } +} +``` + +would correspond to: + +```wasm +(component + (type $wit (instance + (type $shared (instance + (type $metadata (record (; ... ;))) + (export "metadata" (type (eq $metadata))) + )) + (export $shared "shared" (type $shared)) + (alias export $shared "metadata" (type $outer-metadata)) + + (type $host (instance + (export $metadata "metadata" (type $outer-metadata)) + (export "get" (func (result $metadata))) + )) + (export $host "host" (type $host)) + + (type $db (instance + (export $metadata "metadata" (type $outer-metadata)) + (export "get" (func (param "data" $metadata) (result string))) + )) + (export $db "db" (type $db)) + + (type $the-world (component + (import "shared" (instance (type $shared))) + (import "host" (instance (type $host))) + (export "db" (instance (type $db))) + )) + (export "the-world" (type $the-world)) + )) + (export "wit" (type $wit)) +) +``` + +A `world` in a WIT document describes a concrete component, and is represented +in this binary representation as a `component` type (e.g. `$the-world` above). +Tooling which creates a WebAssembly component from a `world` is expected to +create a component that is a subtype of this type. From e425309daecc2b86b600ede4ed06694473aaffcc Mon Sep 17 00:00:00 2001 From: Alex Crichton Date: Wed, 7 Dec 2022 08:05:18 -0800 Subject: [PATCH 02/12] Add `in` keyword --- design/mvp/WIT.md | 1 + 1 file changed, 1 insertion(+) diff --git a/design/mvp/WIT.md b/design/mvp/WIT.md index 070c7614..af244288 100644 --- a/design/mvp/WIT.md +++ b/design/mvp/WIT.md @@ -606,6 +606,7 @@ keyword ::= 'use' | 'import' | 'export' | 'default' + | 'in' ``` ## Top-level items From 6b2b01cf7ba3da9b82eccad498df1bf9145f8e3c Mon Sep 17 00:00:00 2001 From: Alex Crichton Date: Wed, 7 Dec 2022 08:05:31 -0800 Subject: [PATCH 03/12] Update world syntax: * Allow preceding `default` keyword * Remove `export foo: u32` * Specify more support in `interface-type` to include all the `use-from` possibilities such as `id`, `strlit`, and `id 'in' strlit` --- design/mvp/WIT.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/design/mvp/WIT.md b/design/mvp/WIT.md index af244288..2b3ccc9d 100644 --- a/design/mvp/WIT.md +++ b/design/mvp/WIT.md @@ -627,16 +627,17 @@ Worlds define a [componenttype](https://github.com/WebAssembly/component-model/b Concretely, the structure of a world is: ```wit -world-item ::= 'world' id '{' world-items* '}' +world-item ::= 'default'? 'world' id '{' world-items* '}' world-items ::= export-item | import-item export-item ::= 'export' id ':' extern-type import-item ::= 'import' id ':' extern-type -extern-type ::= ty | func-type | interface-type +extern-type ::= func-type | interface-type interface-type ::= 'interface' '{' interface-items* '}' + | use-from ``` ## Item: `interface` From 95a21b44c55ef9e40313633de904f737381a8b51 Mon Sep 17 00:00:00 2001 From: Alex Crichton Date: Wed, 7 Dec 2022 08:07:01 -0800 Subject: [PATCH 04/12] Fix some typos --- design/mvp/WIT.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/design/mvp/WIT.md b/design/mvp/WIT.md index 2b3ccc9d..9cceaab8 100644 --- a/design/mvp/WIT.md +++ b/design/mvp/WIT.md @@ -34,7 +34,7 @@ would correspond to: ```wasm (component - (import "host" (insance $host + (import "host" (instance $host (export "log" (func (param "msg" string))) )) ;; ... @@ -157,7 +157,7 @@ reusability. This enables a sort of module system for WIT syntax where files may import from one another. > **Note**: The precise semantics of imports and how everything maps out is -> still being design. Basic filesystem-based organization works but it's +> still being designed. Basic filesystem-based organization works but it's > intended to extend to URL-based organization in the near future. For example > the strings below are intended to integrate into a registry-based workflow as > well in addition to looking up files on the filesystem. @@ -343,7 +343,7 @@ world my-world { } ``` -This is an invalid WIT document due because `my-world` needs to import two +This is an invalid WIT document because `my-world` needs to import two unique interfaces called `shared`. To disambiguate a manual import is required: ``` From 50dda7544c87572928cffcc396eb9c49cb56b4af Mon Sep 17 00:00:00 2001 From: Alex Crichton Date: Fri, 9 Dec 2022 10:38:13 -0800 Subject: [PATCH 05/12] Update with recent discussion from Luke * The top-level item is now a "wit package" * A package is a collection of documents, and documents are a file * `use` can happen between interfaces of a file, between interfaces across documents in a package, and across packages * Documents are always a flat list of interfaces/worlds/etc * The binary encoding now is updated to accommodate restrictions for resources, this new flat list, and such. --- design/mvp/WIT.md | 567 +++++++++++++++++++++++++++++++--------------- 1 file changed, 389 insertions(+), 178 deletions(-) diff --git a/design/mvp/WIT.md b/design/mvp/WIT.md index 9cceaab8..1cce261c 100644 --- a/design/mvp/WIT.md +++ b/design/mvp/WIT.md @@ -1,26 +1,43 @@ # The `wit` format -The Wasm Interface Type (WIT) text format is a developer-friendly format to -describe the imports and exports of a component. The WIT format can be thought -of as an IDL of sorts to describe APIs that are grouped together as -[`interface`s][interfaces] inside of a [`world`][worlds]. WIT text files are -shared between producers of components and consumers of components as a format -for bindings generation in host and guest languages. The WIT text format -additionally provides a developer-friendly way to inspect an existing -component and learn about its imports and exports. - -The WIT text format uses the file extension `wit`, for example `foo.wit` is -a WIT document. The contents of a `*.wit` file must be valid utf-8 bytes. WIT -documents can contain two items at the top-level: [`interface`][interfaces] and -[`world`][worlds]. +The Wasm Interface Type (WIT) format is an [IDL] to provide tooling for the +[WebAssembly Component Model][components] in two primary ways: + +* WIT is a developer-friendly format to describe the imports and exports to a + component. It is easy to read and write and provides the foundational basis + for producing components from guest languages as well as consuming components + in host languages. + +* WIT packages are the basis of sharing types and definitions in an ecosystem of + components. Authors can import types from other WIT packages when generating a + component, publish a WIT package representing a host embedding, or collaborate + on a WIT definition of a shared set of APIs between platforms. + +A WIT package is a collection of WIT documents. Each WIT document is defined in +a file that uses the file extension `wit`, for example `foo.wit`, and is encoded +as valid utf-8 bytes. Each WIT document contains a collection of +[`interface`s][interfaces] and [`world`s][worlds]. Types can be imported from +sibling documents (files) within a package and additionally from other packages +through a URLs. + +This document will go through the purpose of the syntactic constructs of a WIT +document, a pseudo-formal [grammar specification][lexical-structure], and +additionally a specification of the [binary format][binary-format] of a WIT +package suitable for distribution. + +[IDL]: https://en.wikipedia.org/wiki/Interface_description_language +[components]: https://github.com/webassembly/component-model ## WIT Interfaces [interfaces]: #wit-interfaces -An `interface` in WIT is a collection of [functions] and [types] which -corresponds to an instance in the component model. Interfaces can either be -imported or exported from [worlds] and represent imported and exported -instances. For example this `interface`: +The concept of an "interface" is central in WIT as a collection of [functions] +and [types]. An interface can be thought of as an instance in the WebAssembly +Component Model, for example a unit of functionality imported from the host or +implemented by a component for consumption on a host. All functions and types +belong to an interface. + +An example of an interface is: ```wit interface host { @@ -46,7 +63,7 @@ and [function][functions] definitions. For example: ```wit interface wasi-fs { - use { errno } from "wasi-types" + use { errno } from .types record stat { ino: u64, @@ -67,7 +84,20 @@ the defined names can collide. A WIT document can contain any number of interfaces listed at the top-level and in any order. The WIT validator will ensure that all references between -documents are well-formed and acyclic. +interfaces are well-formed and acyclic. + +An interface may optionally be listed as the `default` of a document. For +example: + +```wit +default interface types { + // ... +} +``` + +This will come up later when describing [`use` statements][use] and indicates +that when a document is imported from the `types` name here, for example, does +not need to be specified as it's the `default`. ## WIT Worlds [worlds]: #wit-worlds @@ -116,6 +146,10 @@ world wasi { } ``` +An imported or exported interface corresponds to an imported or exported +instance in the component model. Functions are equivalent to bare component +functions. + Additionally interfaces can be defined "inline" as a form of sugar for defining it at the top-level @@ -149,21 +183,17 @@ default world my-world { If no `default` world is specified in the WIT document and no named world is explicitly chosen then bindings cannot be generated. -## WIT File Organization -[use]: #wit-file-organization +## WIT Packages and `use` +[use]: #wit-packages-and-use -WIT files can be organized into separate files for maintainability and -reusability. This enables a sort of module system for WIT syntax where files may -import from one another. +A WIT package represents a unit of distribution that can be published to a +registry, for example, and used by other WIT packages and documents. WIT +packages are a flat list of documents, defined in `*.wit` files. The current +thinking for a convention is that projects will have a `wit` folder where all +`wit/*.wit` files within are members of a WIT package. -> **Note**: The precise semantics of imports and how everything maps out is -> still being designed. Basic filesystem-based organization works but it's -> intended to extend to URL-based organization in the near future. For example -> the strings below are intended to integrate into a registry-based workflow as -> well in addition to looking up files on the filesystem. - -Within a single WIT file the `use` statement can be used to import between -interfaces: +Within a single WIT document (file) a `use` statement can be used to import +between interfaces, for example: ```wit interface types { @@ -177,12 +207,10 @@ interface my-host-functions { } ``` -Here the `from` directive of the `use` is not quoted which means that it's -resolved relative to the WIT file itself. The `interface types` may come either -after or before the `use` directive's `interface`. - -The `use` directive is not allowed to create cycles and must always form an -acyclic graph of dependencies between interfaces. +Here the destination of the `from` is resolved to the `types` interface defined +locally within the file. The interface `types` may come either after or before +the `use` directive's `interface`. Interfaces linked with `use` are not allowed +to be cyclic. Names imported via `use` can be renamed as they're imported as well: @@ -192,90 +220,95 @@ interface my-host-functions { } ``` -When importing or exporting an [interface][interfaces] in a [world][worlds] -unquoted names refer to [interfaces] defined elsewhere in the file: +Documents in a WIT package may also import from each other, for example the +above can be rewritten as: ```wit -world my-world { - import host: host +// types.wit +default interface types { + enum errno { /* ... */ } + + type size = u32 } -interface host { - // ... +// host.wit +interface my-host-functions { + use { errno, size } from .types } ``` -The target of a `use` or the type of an import/export in a `world` may also be a -quoted string: +Note the `.` in the `.types` destination of the `from` here, which indicates +that a sibling document is being imported from. Additionally note the usage of +`default interface` in the `types.wit` file which simplifies the `from` +directive of the `use`. WIT documents can also import from any interface defined +within another document, however: ```wit -interface foo { - use { /* ... */ } from "./other-file" +// types.wit +default interface types { /* .. */ } + +interface more-types { + type another-type = string } -world my-world { - import some-import: "./other-file" - import another-import: "./other-directory/other-file" +// host.wit +interface my-host-functions { + use { another-type } from more-types in .types } ``` -This quoted string form indicates that the import is located in a different WIT -file. At this time all quoted strings must start with `./` or `../` and refer to -a file relative to the location of the current WIT file. This will eventually be -expanded to allow something along the lines of `"wasi:fs"` or similar but the -precise semantics here have not been defined. - -The above directives, for example, will import from `./other-file.wit` (or -`./other-file.wit.md` as described [below][markdown]) and -`./other-directory/other-file.wit` (or `./other-directory/other-file.wit.md`). -The `.wit` extension is automatically appended to the lookup path and `.wit.md` -is tested if the `.wit` file doesn't exist. - -Additionally the above directives require that `other-file.wit` contains an -interface marked `default`. Similar to [`default` worlds][worlds] a WIT file may -have multiple [`interface`s][interfaces] inside it and the import must happen -from one of them, so `default` is used to disambiguate: +Here the `more-types in ...` indicates that a specific non-`default` interface +is being chosen to import from. Documents in a WIT package must be named after a +[valid identifier][identifiers] and be unique within the package. Documents +cannot contain cycles between them as well with `use` statements. + +When importing or exporting an [interface][interfaces] in a [world][worlds] +the same syntax is used after the `:` as what's after a `from` in a `use`: ```wit -// other-file.wit -default interface foo { - // ... +world my-world { + import host: host + import other-functionality: .sibling-file + import more-functionality: specific-interface in .sibling-file } - -// my-file.wit -default interface foo { - use { /* ... */ } from "./other-file" +interface host { + // ... } ``` -A `use` directive can also explicitly list the requested interface to select a -non-`default` one: +The target of a `use` or the type of an import/export in a `world` may also be a +quoted string: ```wit -// other-file.wit interface foo { - // ... + use { /* ... */ } from "registry:package/document" } +``` +This quoted string form indicates that the import is located in a different WIT +package. Locating a package is a higher-level tooling concern than the WIT +specification and is expected to be arbitrated by a registry, for example, with +integration into per-language tooling. A parser for WIT would need to be +configured where this package lives to resolve the interface `foo`. + +Import strings must be valid URLs. An import string additionally refers to a +specific document within a package. In the above example `registry:package` can +be thought of as the base URL for what's being imported, while `/document` is +selecting the WIT document called `document`. + +> **Note**: Integration of URLs into WIT and packages is still pretty early days +> and the specifics here may change. -// my-file.wit +Like with importing from sibling packages a `use` directive can also explicitly +list the requested interface to select a non-`default` one: + +```wit default interface foo { - use { /* ... */ } from foo in "./other-file" + use { /* ... */ } from bar in "registry:package/document } ``` -Like before within a WIT file `use` statements must be acyclic between files as -well. - -Splitting a WIT document into multiple files does not have an impact on its -final structure and it's purely a convenience to developers. Resolution of a WIT -document happens as-if everything were contained in one document (modulo -"hygienic" renaming where `use`-with-identifier still only works within one -file). This means that a WIT document can always be represented as a single -large WIT document with everything contained and separate-file organization is -not necessary. - ### Transitive imports and worlds A `use` statement is not implemented by copying type information around but @@ -302,7 +335,7 @@ world my-world { } ``` -would generate this component : +would generate this component: ```wasm (component @@ -335,10 +368,10 @@ default interface shared { /* ... */ } // world.wit world my-world { import foo: interface { - use { a-type } from "shared1" + use { a-type } from .shared1 } import bar: interface { - use { other-type } from "shared2" + use { other-type } from .shared2 } } ``` @@ -348,14 +381,14 @@ unique interfaces called `shared`. To disambiguate a manual import is required: ``` world my-world { - import shared1: "shared1" - import shared2: "shared1" + import shared1: .shared1 + import shared2: .shared2 import foo: interface { - use { a-type } from "shared1" + use { a-type } from .shared1 } import bar: interface { - use { other-type } from "shared2" + use { other-type } from .shared2 } } ``` @@ -505,6 +538,7 @@ file and concatenated into a single string which is then parsed as a normal `*.wit` file. # Lexical structure +[lexical-structure]: #lexical-structure The `wit` format is a curly-braced-based format where whitespace is optional (but recommended). A `wit` document is parsed as a unicode string, and when stored in @@ -629,7 +663,7 @@ Concretely, the structure of a world is: ```wit world-item ::= 'default'? 'world' id '{' world-items* '}' -world-items ::= export-item | import-item +world-items ::= export-item | import-item | use-item | typedef-item export-item ::= 'export' id ':' extern-type import-item ::= 'import' id ':' extern-type @@ -640,25 +674,31 @@ interface-type ::= 'interface' '{' interface-items* '}' | use-from ``` +Note that worlds can import types and define their own types to be exported +from the root of a component and used within functions imported and exported. + ## Item: `interface` -Interfaces can be defined in a `wit` document. Interfaces have a name and a sequence of items and functions. +Interfaces can be defined in a `wit` document. Interfaces have a name and a +sequence of items and functions. Specifically interfaces have the structure: ```wit interface-item ::= 'default'? 'interface' id '{' interface-items* '}' -interface-items ::= resource-item - | variant-items - | record-item - | union-items - | flags-items - | enum-items - | type-item +interface-items ::= typedef-item | use-item | func-item +typedef-item ::= resource-item + | variant-items + | record-item + | union-items + | flags-items + | enum-items + | type-item + func-item ::= id ':' func-type func-type ::= 'func' param-list result-list @@ -681,18 +721,14 @@ A `use` statement enables importing type or resource definitions from other wit documents. The structure of a use statement is: ```wit -use * from other-file -use { a, list, of, names } from another-file -use { name as other-name } from interface in "yet-another-file" +use { a, list, of, names } from another-interface +use { name as other-name } from interface in "a:separate/package" ``` Specifically the structure of this is: ```wit -use-item ::= 'use' use-names 'from' use-from - -use-names ::= '*' - | '{' use-names-list '}' +use-item ::= 'use' '{' use-names-list '}' 'from' use-from use-names-list ::= use-names-item | use-names-item ',' use-names-list? @@ -701,6 +737,8 @@ use-names-item ::= id | id 'as' id use-from ::= id + | '.' id + | id in '.' id | strlit | id 'in' strlit ``` @@ -1003,13 +1041,26 @@ record bar2 { ``` # Binary Format +[binary-format]: #binary-format -In addition to a textual format WIT files can also be encoded to a binary -format. The binary format is itself a WebAssembly component and is chosen to be -a more stable artifact over time than the text format in case text format -updates are required. Additionally being represented by the component binary -format means that it's guaranteed that all features of WIT are representable in -the component model. +In addition to a textual format WIT packages can also be encoded to a binary +format. The binary format for WIT is represented as a WebAssembly Component +binary with a specific structure. The purpose of the WIT binary format is to: + +* Provide a way to clearly define what an `interface` and a `world` map to in + the component model. For example a host is said to implement an `interface` if + it provides a subtype of the `instance` type defined in a component. Similarly + a component is said to implement a `world` if the component's type is a + subtype of the `world`'s type. + +* Provide a means by which the textual format of WIT can be evolved while + minimizing impact to the ecosystem. Similar to how the WebAssembly text format + has changed over time it's envisioned that it may be necessary to change the + WIT format for new features as new component model features are implemented. + The binary format provides a clear delineation of what to ubiquitously consume + without having to be so concerned with text format parsers. + +* A binary format is intended to be more efficient parse and store. The binary format for a WIT document can also be thought of in a loose sense as a fully-resolved and indexed representation of a WIT document. Constructs such @@ -1019,13 +1070,42 @@ references. Additionally the transitive import set is all finalized as well. An example of the binary format is that this document: ```wit -interface host { +// host.wit +interface console { log: func(arg: string) } +``` -world the-world { - import host: host - export run: func() +would correspond to: + +```wasm +(component + (type (export "host") (component + (type $console (instance + (export "log" (func (param "arg" string))) + )) + (export "console" (instance (type $console))) + )) +) +``` + +Here it can be seen how an `interface` corresponds to an `instance` in the +component model. Note that the WIT document is encoded entirely within the type +section of a component and does not actually represent any concrete instances. +This is done to implement `use` statements: + +```wit +// host.wit +interface types { + enum level { + info, + debug, + } +} + +interface console { + use { level } from types + log: func(level: level, msg: string) } ``` @@ -1033,91 +1113,222 @@ would correspond to: ```wasm (component - (type $wit (instance - (type $host (instance - (export "log" (func (param "arg" string))) + (type (export "host") (component + (type $types (instance + (export "enum" (type (enum "info" "debug"))) )) - (export "host" (type $host)) + (export $types "types" (instance (type $types))) + (alias export $types "level" (type $level)) + (type $console (instance + (alias outer 1 $level (type $level')) + (export "log" (func (param "level" $level') (param "msg" string))) + )) + (export "console" (instance (type $console))) + )) +) +``` + +Here the `alias` of `"level"` indicates that the exact type is being used from a +different interface. +A `world` is represented as a component type. + +```wit +// host.wit +world the-world { + export test: func() + export run: func() +} +``` + +would correspond to: + +```wasm +(component + (type (export "host") (component (type $the-world (component - (import "host" (instance (type $host))) + (import "test" (func)) (export "run" (func)) )) (export "the-world" (type $the-world)) )) - (export "wit" (type $wit)) ) ``` -Here it can be seen that the entire document itself is represented as an -`instance` type with named exports. Within this `instance` type each export is -either itself an `instance` or a `component` with `instance`s corresponding to -an `interface` in WIT and `component`s corresponding to `world`s. The outer -component has a single export named `wit` which points to the instance type that -describes the document. +Component types in the WebAssembly component binary format cannot close over +outer instances so interfaces referred to by a component are redefined, at least +the parts needed, within the component: -Types defined within a WIT interface are additionally defined within the wasm -`instance`. Additionally `use` of types between interfaces is encoded with -aliases between them. ```wit -interface shared { - record metadata { - // ... - } +// host.wit +world the-world { + import console: console } -world the-world { - import host: interface { - use { metadata } from shared +interface console { + log: func(arg: string) +} +``` - get: func() -> metadata - } +would correspond to: - export db: interface { - use { metadata } from shared +```wasm +(component + (type (export "host") (component + (type $console (instance + (export "log" (func (param "arg" string))) + )) + (export "console" (instance (type $console))) + (type $the-world (component + (import "console" (instance + (export "log" (func (param "arg" string))) + )) + )) + (export "the-world" (type $the-world)) + )) +) +``` - get: func(data: metadata) -> string - } -} +Each WIT document in a package becomes a top-level export of the component: + +```wit +// foo.wit +interface foo {} + +// bar.wit +interface bar {} ``` would correspond to: ```wasm (component - (type $wit (instance - (type $shared (instance - (type $metadata (record (; ... ;))) - (export "metadata" (type (eq $metadata))) + (type (export "foo") (component + (type $foo (instance + )) + (export "foo" (instance (type $foo))) + )) + (type (export "bar") (component + (type $bar (instance )) - (export $shared "shared" (type $shared)) - (alias export $shared "metadata" (type $outer-metadata)) + (export "bar" (instance (type $bar))) + )) +) +``` + +Imports of packages via a URL are encoded as imports to the outermost component +type as well. + +```wit +// foo.wit +interface foo { + use { some-type } from "a:url/types" +} +``` - (type $host (instance - (export $metadata "metadata" (type $outer-metadata)) - (export "get" (func (result $metadata))) +would correspond to: + +```wasm +(component + (type (export "foo") (component + (import "types" "a:url/types" (instance $types + (type $some-type ...) + (export "some-type" (type $some-type)) + )) + (alias export $types "some-type" (type $some-type)) + (type $foo (instance + (alias outer 1 $some-type (type $some-type')) + (export "some-type" (type $some-type')) )) - (export $host "host" (type $host)) + (export "foo" (instance (type $foo))) + )) +) +``` - (type $db (instance - (export $metadata "metadata" (type $outer-metadata)) - (export "get" (func (param "data" $metadata) (result string))) +putting all of this together an example of development of the `wasi-http` +package would be: + +```wit +// wasi-http repo + +// wit/types.wit +default interface types { + resource request { ... } + resource response { ... } +} + +// wit/handler.wit +default interface handler { + use { request, response } from .types + handle: func(request) -> response +} + +// wit/proxy.wit +default world proxy { + import console: "wasi:logging/backend" + import origin: .handler + export handler: .handler +} +``` + +and its corresponding binary encoding would be: + +```wasm +(component + ;; corresponds to `wit/types.wit` + (type (export "types") (component + (export "types" (instance + (export "request" (type (sub resource))) + (export "response" (type (sub resource))) + )) + )) + ;; corresponds to `wit/handler.wit` + (type (export "handler") (component + ;; interfaces not required in a document are imported here. The name "types" + ;; with no URL refers to the `types` document in this package. + (import "types" (instance $types + (export "request" (type (sub resource))) + (export "response" (type (sub resource))) )) - (export $db "db" (type $db)) - (type $the-world (component - (import "shared" (instance (type $shared))) - (import "host" (instance (type $host))) - (export "db" (instance (type $db))) + ;; aliases represent `use` from the imported document + (alias export $types "request" (type $request)) + (alias export $types "response" (type $response)) + (export "handler" (instance + (export $request' "request" (type $request)) + (export $response' "response" (type $response)) + (export "handle" (func (param (own $request')) (result (own $response')))) + )) + )) + ;; corresponds to `wit/proxy.wit` + (type (export "proxy") (component + (export "proxy" (component + ;; This world transitively depends on "types" so it's listed as an + ;; import. + (import "types" (instance $types + (export "request" (type (sub resource))) + (export "response" (type (sub resource))) + )) + (alias export $types "request" (type $request)) + (alias export $types "response" (type $response)) + + ;; This is filled in with the contents of what `wasi:logging/backend` + ;; resolved to + (import "console" "wasi:logging/backend" (instance + ... + )) + (import "origin" (instance + (export $request' "request" (type (eq $request))) + (export $response' "response" (type (eq $response))) + (export "handle" (func (param (own $request)) (result (own $response)))) + )) + (export "handler" (instance $handler + (export $request' "request" (type (eq $request))) + (export $response' "response" (type (eq $response))) + (export "handle" (func (param (own $request')) (result (own $response')))) + )) )) - (export "the-world" (type $the-world)) )) - (export "wit" (type $wit)) ) ``` - -A `world` in a WIT document describes a concrete component, and is represented -in this binary representation as a `component` type (e.g. `$the-world` above). -Tooling which creates a WebAssembly component from a `world` is expected to -create a component that is a subtype of this type. From ff8bd9ea88e1c4039d556c35e6f4119f38aca88d Mon Sep 17 00:00:00 2001 From: Alex Crichton Date: Fri, 9 Dec 2022 10:40:09 -0800 Subject: [PATCH 06/12] Remove markdown section --- design/mvp/WIT.md | 24 ------------------------ 1 file changed, 24 deletions(-) diff --git a/design/mvp/WIT.md b/design/mvp/WIT.md index 1cce261c..c49a5bb0 100644 --- a/design/mvp/WIT.md +++ b/design/mvp/WIT.md @@ -513,30 +513,6 @@ characters and numbers that are `-` separated. For more information on this see the [binary format](./Binary.md). -## WIT in Markdown -[markdown]: #wit-in-markdown - -The WIT text format can also additionally be parsed from markdown files with the -extension `wit.md`, for example `foo.wit.md`: - - # This would be - - Some markdown text - - ```wit - // interspersed with actual `*.wit` - interface my-interface { - ``` - - ```wit - // which can be broken up between multiple blocks - } - ``` - -Triple-fence code blocks with the `wit` marker will be extracted from a markdown -file and concatenated into a single string which is then parsed as a normal -`*.wit` file. - # Lexical structure [lexical-structure]: #lexical-structure From 98f348fb932a816feca074fb8321293407b8cd97 Mon Sep 17 00:00:00 2001 From: Alex Crichton Date: Mon, 12 Dec 2022 12:24:56 -0800 Subject: [PATCH 07/12] Use a different syntax for `use` * Remove strings as targets-for-`use` and instead always use from identifiers. This pushes resolution and such up a layer where it's going to be happening anyway between packages. * Move the module that's being used from to the start of `use` to better assist with auto-completion in IDEs possibly in the future. This way we as humans will type `use foo.bar.{` and an IDE should have enough context by that point to auto-complete what to use. * Add a `pkg` and `self` path anchor which paths start from to indicate that they're rooted in the current package or current document. * Continue to use `.` as a separator between items in a `use` statement. --- design/mvp/WIT.md | 198 +++++++++++++++++++++++++++------------------- 1 file changed, 117 insertions(+), 81 deletions(-) diff --git a/design/mvp/WIT.md b/design/mvp/WIT.md index c49a5bb0..043e6f29 100644 --- a/design/mvp/WIT.md +++ b/design/mvp/WIT.md @@ -63,7 +63,7 @@ and [function][functions] definitions. For example: ```wit interface wasi-fs { - use { errno } from .types + use pkg.types.{errno} record stat { ino: u64, @@ -137,9 +137,9 @@ function or an interface. ```wit world wasi { - import wasi-fs: "wasi-fs" - import wasi-random: "wasi-random" - import wasi-clock: "wasi-clock" + import wasi-fs: wasi.fs + import wasi-random: wasi.random + import wasi-clock: wasi.clock // ... export command: func(args: list) @@ -159,8 +159,8 @@ interface out-of-line { } world your-world { - import out-of-line: out-of-line - // ... is equivalent to ... + import out-of-line: self.out-of-line + // ... is roughtly equivalent to ... import out-of-line2: interface { the-function: func() } @@ -177,6 +177,7 @@ an explicitly named world must be chosen: ```wit default world my-world { + // ... } ``` @@ -203,20 +204,21 @@ interface types { } interface my-host-functions { - use { errno, size } from types + use self.types.{errno, size} } ``` -Here the destination of the `from` is resolved to the `types` interface defined -locally within the file. The interface `types` may come either after or before -the `use` directive's `interface`. Interfaces linked with `use` are not allowed -to be cyclic. +Here the `use` starts with `self` which indicates that something from the +current document is being used. Then the interface named `types` is listed, +followed by a list of type names to use from the interface. The interface +`types` may textually come either after or before the `use` directive's +`interface`. Interfaces linked with `use` are not allowed to be cyclic. Names imported via `use` can be renamed as they're imported as well: ```wit interface my-host-functions { - use { errno as my-errno } from types + use self.types.{errno as my-errno} } ``` @@ -233,14 +235,14 @@ default interface types { // host.wit interface my-host-functions { - use { errno, size } from .types + use pkg.types.{errno, size} } ``` -Note the `.` in the `.types` destination of the `from` here, which indicates -that a sibling document is being imported from. Additionally note the usage of -`default interface` in the `types.wit` file which simplifies the `from` -directive of the `use`. WIT documents can also import from any interface defined +The `pkg` keyword indicates that the `use` statement starts at the package root, +as opposed to the `self` keyword described above starting in the local document. +Additionally note the usage of `default interface` in the `types.wit` file which +simplifies the `use`. WIT documents can also import from any interface defined within another document, however: ```wit @@ -253,23 +255,23 @@ interface more-types { // host.wit interface my-host-functions { - use { another-type } from more-types in .types + use pkg.types.more-types.{another-type} } ``` -Here the `more-types in ...` indicates that a specific non-`default` interface -is being chosen to import from. Documents in a WIT package must be named after a -[valid identifier][identifiers] and be unique within the package. Documents -cannot contain cycles between them as well with `use` statements. +Here `more-types in the `use` path indicates that it's the specific interface +being referenced. Documents in a WIT package must be named after a [valid +identifier][identifiers] and be unique within the package. Documents cannot +contain cycles between them as well with `use` statements. When importing or exporting an [interface][interfaces] in a [world][worlds] -the same syntax is used after the `:` as what's after a `from` in a `use`: +the same syntax is used after the `:` as `use`: ```wit world my-world { - import host: host - import other-functionality: .sibling-file - import more-functionality: specific-interface in .sibling-file + import host: self.host + import other-functionality: pkg.sibling-file + import more-functionality: pkg.sibling-file.specific-interface } interface host { @@ -277,37 +279,74 @@ interface host { } ``` -The target of a `use` or the type of an import/export in a `world` may also be a -quoted string: +The `use` statement so far has always started with `self` or `pkg`, but it may +also start with any valid identifier: ```wit interface foo { - use { /* ... */ } from "registry:package/document" + use package.other-document.{some-type} } ``` -This quoted string form indicates that the import is located in a different WIT -package. Locating a package is a higher-level tooling concern than the WIT -specification and is expected to be arbitrated by a registry, for example, with -integration into per-language tooling. A parser for WIT would need to be -configured where this package lives to resolve the interface `foo`. - -Import strings must be valid URLs. An import string additionally refers to a -specific document within a package. In the above example `registry:package` can -be thought of as the base URL for what's being imported, while `/document` is -selecting the WIT document called `document`. - -> **Note**: Integration of URLs into WIT and packages is still pretty early days -> and the specifics here may change. - -Like with importing from sibling packages a `use` directive can also explicitly -list the requested interface to select a non-`default` one: - -```wit -default interface foo { - use { /* ... */ } from bar in "registry:package/document -} -``` +This form indicates that the identifier `package` corresponds to some externally +specified package. Resolution of this WIT document requires the name `package` +to be provided via external configuration. For example a registry might indicate +that `package` was downloaded to a particular path at a particular version. The +name `package` could also perhaps be another package defined in the local +project which is configured via bindings generation. + +This syntax allows `use` paths starting with any identifier other than `self` +and `pkg` to create a set of names that the document needs to be resolved +correctly. These names are considered the dependency packages of the current +package being parsed. Note that the set of names here are local to this package +do not need to conflict with dependency names in other packages. This enables +each package to be resolved entirely separately and, if necessary, the name +`foo` could mean different things to different packages. + +> **Note**: The tooling for and mechanism for precisly how these external names +> are defined is not specified here. This is something that will be iterated on +> to create documentation of the tooling in question and community standards +> about how best to do this. +> +> As an example, however, imagine a CLI tool to generate C bindings for creating +> a component that takes, as input, the world to generate. +> +> generate-c-bindings pkg.my-component +> +> The specification of `pkg` here indicates that a locally-written WIT file +> is being consumed. This could look inside of a `wit` folder in the current +> directory for a package to parse as a set of WIT files. Inside this package +> the `my-component` document would be selected and it would be expected to have +> a `default world`. +> +> Similarly an invocation such as: +> +> generate-c-bindings pkg.my-component --extern wasi=./wit/wasi +> +> Could indicate that the identifier `wasi` within `my-component` would be +> resolved to a WIT package located in the `./wit/wasi` folder. Instead of +> `./wit/wasi` it could even pass `./wit/wasi.wasm` which would be the binary +> encoding of the `wasi` package. This sort of filesystem-based heuristic could +> be applied by the `generate-c-bindings` tool as well. +> +> Alternatively there could also be a tool which takes a configuration file: +> +> ```toml +> [wit-dependencies] +> wasi = "1.0" +> ``` +> +> Where running `wit-registry` (a hypothetical CLI command) would parse this +> file, resolve it fully, and download the `wasi` package and place it somewhere +> along with perhaps a `resolve.json` file. This `resolve.json` could then be +> consumed by `generate-c-bindings` as an additional method for defining the +> dependency graph of components. +> +> Note that these scenarios are all hypothetical at this time and are the rough +> direction that tooling is expected to go in. This will evolve over time but is +> intended to set the stage for "what actually happens when I refer to a +> dependency package" where the high-level idea is that it's a concern external +> to the WIT package itself and resolved by higher-level tooling. ### Transitive imports and worlds @@ -328,7 +367,7 @@ interface shared { world my-world { import host: interface { - use { metadata } from shared + use self.shared.{metadata}) get: func() -> metadata } @@ -368,10 +407,10 @@ default interface shared { /* ... */ } // world.wit world my-world { import foo: interface { - use { a-type } from .shared1 + use pkg.shared1.{a-type} } import bar: interface { - use { other-type } from .shared2 + use pkg.shared2.{other-type} } } ``` @@ -381,14 +420,14 @@ unique interfaces called `shared`. To disambiguate a manual import is required: ``` world my-world { - import shared1: .shared1 - import shared2: .shared2 + import shared1: pkg.shared1 + import shared2: pkg.shared2 import foo: interface { - use { a-type } from .shared1 + use pkg.shared1.{a-type} } import bar: interface { - use { other-type } from .shared2 + use pkg.shared2.{other-type} } } ``` @@ -532,7 +571,6 @@ token ::= whitespace | operator | keyword | identifier - | strlit ``` Whitespace and comments are ignored when parsing structures defined elsewhere @@ -606,7 +644,6 @@ keyword ::= 'use' | 'list' | 'result' | 'as' - | 'from' | 'static' | 'interface' | 'tuple' @@ -616,7 +653,6 @@ keyword ::= 'use' | 'import' | 'export' | 'default' - | 'in' ``` ## Top-level items @@ -647,7 +683,7 @@ import-item ::= 'import' id ':' extern-type extern-type ::= func-type | interface-type interface-type ::= 'interface' '{' interface-items* '}' - | use-from + | use-path ``` Note that worlds can import types and define their own types to be exported @@ -697,14 +733,15 @@ A `use` statement enables importing type or resource definitions from other wit documents. The structure of a use statement is: ```wit -use { a, list, of, names } from another-interface -use { name as other-name } from interface in "a:separate/package" +use self.interface.{a, list, of, names} +use pkg.document.some-type +use my-dependency.document.other-type ``` Specifically the structure of this is: ```wit -use-item ::= 'use' '{' use-names-list '}' 'from' use-from +use-item ::= 'use' use-path '.' '{' use-names-list '}' use-names-list ::= use-names-item | use-names-item ',' use-names-list? @@ -712,11 +749,7 @@ use-names-list ::= use-names-item use-names-item ::= id | id 'as' id -use-from ::= id - | '.' id - | id in '.' id - | strlit - | id 'in' strlit +use-path ::= id ('.' id)* ``` Note: Here `use-names-list?` means at least one `use-name-list` term. @@ -1080,7 +1113,7 @@ interface types { } interface console { - use { level } from types + use self.types.{level} log: func(level: level, msg: string) } ``` @@ -1193,13 +1226,13 @@ would correspond to: ) ``` -Imports of packages via a URL are encoded as imports to the outermost component +Imports of packages are encoded as imports to the outermost component type as well. ```wit // foo.wit interface foo { - use { some-type } from "a:url/types" + use registry-package.types.{some-type} } ``` @@ -1208,7 +1241,7 @@ would correspond to: ```wasm (component (type (export "foo") (component - (import "types" "a:url/types" (instance $types + (import "types" "URL" (instance $types (type $some-type ...) (export "some-type" (type $some-type)) )) @@ -1222,7 +1255,10 @@ would correspond to: ) ``` -putting all of this together an example of development of the `wasi-http` +Note that `URL` would be provided by external tooling providing the definition +of `registry-package` here as well. + +Putting all of this together an example of development of the `wasi-http` package would be: ```wit @@ -1236,15 +1272,15 @@ default interface types { // wit/handler.wit default interface handler { - use { request, response } from .types + use pkg.types.{request, response} handle: func(request) -> response } // wit/proxy.wit default world proxy { - import console: "wasi:logging/backend" - import origin: .handler - export handler: .handler + import console: wasi-logging.backend + import origin: pkg.handler + export handler: pkg.handler } ``` @@ -1289,9 +1325,9 @@ and its corresponding binary encoding would be: (alias export $types "request" (type $request)) (alias export $types "response" (type $response)) - ;; This is filled in with the contents of what `wasi:logging/backend` + ;; This is filled in with the contents of what `wasi-logging.backend` ;; resolved to - (import "console" "wasi:logging/backend" (instance + (import "console" "URL-for-wasi-logging.backend" (instance ... )) (import "origin" (instance From 08e7f69f89747cb10d9a74756480b9997a857500 Mon Sep 17 00:00:00 2001 From: Alex Crichton Date: Tue, 13 Dec 2022 07:35:37 -0800 Subject: [PATCH 08/12] Typos --- design/mvp/WIT.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/design/mvp/WIT.md b/design/mvp/WIT.md index 043e6f29..fc8438e9 100644 --- a/design/mvp/WIT.md +++ b/design/mvp/WIT.md @@ -160,7 +160,7 @@ interface out-of-line { world your-world { import out-of-line: self.out-of-line - // ... is roughtly equivalent to ... + // ... is roughly equivalent to ... import out-of-line2: interface { the-function: func() } @@ -259,7 +259,7 @@ interface my-host-functions { } ``` -Here `more-types in the `use` path indicates that it's the specific interface +Here `more-types` in the `use` path indicates that it's the specific interface being referenced. Documents in a WIT package must be named after a [valid identifier][identifiers] and be unique within the package. Documents cannot contain cycles between them as well with `use` statements. @@ -303,7 +303,7 @@ do not need to conflict with dependency names in other packages. This enables each package to be resolved entirely separately and, if necessary, the name `foo` could mean different things to different packages. -> **Note**: The tooling for and mechanism for precisly how these external names +> **Note**: The tooling for and mechanism for precisely how these external names > are defined is not specified here. This is something that will be iterated on > to create documentation of the tooling in question and community standards > about how best to do this. From f53ffb39e8e0b74a7d688208d9bde9afc4852982 Mon Sep 17 00:00:00 2001 From: Alex Crichton Date: Wed, 14 Dec 2022 07:32:32 -0800 Subject: [PATCH 09/12] Fix some typos --- design/mvp/WIT.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/design/mvp/WIT.md b/design/mvp/WIT.md index fc8438e9..17ab3c70 100644 --- a/design/mvp/WIT.md +++ b/design/mvp/WIT.md @@ -390,9 +390,9 @@ would generate this component: ``` Here it can be seen that despite the `world` only listing `host` as an import -the component additionally import a `shared` instance. This is due to the fact +the component additionally imports a `shared` instance. This is due to the fact that the `use { ... } from shared` implicitly requires that `shared` is imported -to the component as well. +into the component as well. Note that the name `"shared"` here is derived from the name of the `interface` which can also lead to conflicts: @@ -1124,7 +1124,7 @@ would correspond to: (component (type (export "host") (component (type $types (instance - (export "enum" (type (enum "info" "debug"))) + (export "level" (type (enum "info" "debug"))) )) (export $types "types" (instance (type $types))) (alias export $types "level" (type $level)) From 819cce718e3e304bf19d448089058d74ee0e8e6a Mon Sep 17 00:00:00 2001 From: Alex Crichton Date: Mon, 9 Jan 2023 20:47:30 -0800 Subject: [PATCH 10/12] Update examples with output of current implementation Reflects the progress made on bytecodealliance/wasm-tools#867 --- design/mvp/WIT.md | 58 +++++++++++++++++++++++++---------------------- 1 file changed, 31 insertions(+), 27 deletions(-) diff --git a/design/mvp/WIT.md b/design/mvp/WIT.md index 17ab3c70..0a6a5151 100644 --- a/design/mvp/WIT.md +++ b/design/mvp/WIT.md @@ -1124,12 +1124,13 @@ would correspond to: (component (type (export "host") (component (type $types (instance - (export "level" (type (enum "info" "debug"))) + (type $level (enum "info" "debug")) + (export "level" (type (eq $level))) )) (export $types "types" (instance (type $types))) (alias export $types "level" (type $level)) (type $console (instance - (alias outer 1 $level (type $level')) + (export $level' "level" (type (eq $level))) (export "log" (func (param "level" $level') (param "msg" string))) )) (export "console" (instance (type $console))) @@ -1155,11 +1156,10 @@ would correspond to: ```wasm (component (type (export "host") (component - (type $the-world (component - (import "test" (func)) + (export "the-world" (component + (export "test" (func)) (export "run" (func)) )) - (export "the-world" (type $the-world)) )) ) ``` @@ -1189,12 +1189,11 @@ would correspond to: (export "log" (func (param "arg" string))) )) (export "console" (instance (type $console))) - (type $the-world (component + (export "the-world" (component (import "console" (instance (export "log" (func (param "arg" string))) )) )) - (export "the-world" (type $the-world)) )) ) ``` @@ -1243,12 +1242,11 @@ would correspond to: (type (export "foo") (component (import "types" "URL" (instance $types (type $some-type ...) - (export "some-type" (type $some-type)) + (export "some-type" (type (eq $some-type))) )) (alias export $types "some-type" (type $some-type)) (type $foo (instance - (alias outer 1 $some-type (type $some-type')) - (export "some-type" (type $some-type')) + (export "some-type" (type (eq $some-type))) )) (export "foo" (instance (type $foo))) )) @@ -1291,26 +1289,30 @@ and its corresponding binary encoding would be: ;; corresponds to `wit/types.wit` (type (export "types") (component (export "types" (instance - (export "request" (type (sub resource))) - (export "response" (type (sub resource))) + (type $request (record)) + (type $response (record)) + (export "request" (type (eq $request))) + (export "response" (type (eq $response))) )) )) ;; corresponds to `wit/handler.wit` (type (export "handler") (component ;; interfaces not required in a document are imported here. The name "types" ;; with no URL refers to the `types` document in this package. - (import "types" (instance $types - (export "request" (type (sub resource))) - (export "response" (type (sub resource))) + (import "types" "pkg:/types/types" (instance $types + (type $request (record)) + (type $response (record)) + (export "request" (type (eq $request))) + (export "response" (type (eq $response))) )) ;; aliases represent `use` from the imported document (alias export $types "request" (type $request)) (alias export $types "response" (type $response)) (export "handler" (instance - (export $request' "request" (type $request)) - (export $response' "response" (type $response)) - (export "handle" (func (param (own $request')) (result (own $response')))) + (export $request' "request" (type (eq $request))) + (export $response' "response" (type (eq $response))) + (export "handle" (func (param "request" $request') (result $response'))) )) )) ;; corresponds to `wit/proxy.wit` @@ -1318,27 +1320,29 @@ and its corresponding binary encoding would be: (export "proxy" (component ;; This world transitively depends on "types" so it's listed as an ;; import. - (import "types" (instance $types - (export "request" (type (sub resource))) - (export "response" (type (sub resource))) + (import "types" "pkg:/types/types" (instance $types + (type $request (record)) + (type $response (record)) + (export "request" (type (eq $request))) + (export "response" (type (eq $response))) )) (alias export $types "request" (type $request)) (alias export $types "response" (type $response)) ;; This is filled in with the contents of what `wasi-logging.backend` ;; resolved to - (import "console" "URL-for-wasi-logging.backend" (instance - ... + (import "console" "dep:/foo/bar/baz" (instance + ;; ... )) - (import "origin" (instance + (import "origin" "pkg:/handler/handler" (instance (export $request' "request" (type (eq $request))) (export $response' "response" (type (eq $response))) - (export "handle" (func (param (own $request)) (result (own $response)))) + (export "handle" (func (param "request" $request') (result $response'))) )) - (export "handler" (instance $handler + (export "handler" "pkg:/handler/handler" (instance (export $request' "request" (type (eq $request))) (export $response' "response" (type (eq $response))) - (export "handle" (func (param (own $request')) (result (own $response')))) + (export "handle" (func (param "request" $request') (result $response'))) )) )) )) From cb922fb505a8ed5f49ab1116033876de7e18745f Mon Sep 17 00:00:00 2001 From: Alex Crichton Date: Wed, 18 Jan 2023 07:16:49 -0800 Subject: [PATCH 11/12] Review comments --- design/mvp/WIT.md | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/design/mvp/WIT.md b/design/mvp/WIT.md index ece5ed64..031a6104 100644 --- a/design/mvp/WIT.md +++ b/design/mvp/WIT.md @@ -102,10 +102,11 @@ not need to be specified as it's the `default`. ## WIT Worlds [worlds]: #wit-worlds -WIT documents can contain a `world` annotation at the top-level in addition to -[`interface`][interfaces]. A world is a complete description of both imports and -exports of a component. A world can be thought of as an equivalent of a -`component` type in the component model. For example this world: +WIT documents can contain a `world` definition at the top-level in addition to +[`interface`][interfaces] definitions. A world is a complete description of +both imports and exports of a component. A world can be thought of as an +equivalent of a `component` type in the component model. For example this +world: ```wit world my-world { @@ -136,13 +137,13 @@ Worlds can contain any number of imports and exports, and can be either a function or an interface. ```wit -world wasi { - import wasi-fs: wasi.fs - import wasi-random: wasi.random - import wasi-clock: wasi.clock +world command { + import fs: wasi-fs.fs + import random: wasi-random.random + import clock: wasi-clock.clock // ... - export command: func(args: list) + export main: func(args: list) } ``` From af420065176ec20626095b42fc8463c2b3f1a27c Mon Sep 17 00:00:00 2001 From: Alex Crichton Date: Wed, 1 Feb 2023 08:47:32 -0800 Subject: [PATCH 12/12] Fix a typo --- design/mvp/WIT.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/design/mvp/WIT.md b/design/mvp/WIT.md index 031a6104..20b456c7 100644 --- a/design/mvp/WIT.md +++ b/design/mvp/WIT.md @@ -368,7 +368,7 @@ interface shared { world my-world { import host: interface { - use self.shared.{metadata}) + use self.shared.{metadata} get: func() -> metadata }