Take another step towards interface types (#395)

* Take another step towards interface types

This commit is another large refactoring on the tail of #391 which is
intended to help march one step closer to using pure interface types for
specifying WASI and using `*.witx` files. Contained here is a large
amount of refactoring of the `*.witx` files, the internals of the `witx`
crate, and how code generators are supposed to work.

At the `*.witx` level, some notable changes have been made:

* All functions returning results now return `(expected ok? (error
  $errno))`. This helps signify that the intention of all functions is
  that they return a packaged value which is either a successful result
  or the erroneous error code on failure. ABI-wise nothing has changed,
  but this is intended to model more closely what the interface-types
  API of this would be.

* The `flags` type in `*.witx` now desugars as a struct-of-bools rather
  than simply an `int`. This doesn't have any effect on the ABI today,
  but it does change the AST and type-level representation of these types.

All existing `*.witx` files in this repository have been updated for all
phases with these new forms. To reiterate, though, no details of the ABI
have changed (or at least not intentionally). Everything should still be
ABI-compatible with before.

Under the hood for code generators this is a very large breaking change.
The intention of this commit is to start moving code generators relying
on `witx` into the direction that we'll end up with interface types.
Namely the `coretypes` module is entirely replaced with a new `abi`
module with the intention of handling lots more high-level details of
translation than the previous module did.

The `InterfaceFunc` type now sports two new methods: `call_wasm` and
`call_interface`. These two methods represent the two halves of
lifting/lowering operations performed by code generators. The
`call_wasm` function takes interface types values (or whatever they are
represented as in the source language) and converts them to wasm types
to call a wasm import. The wasm import's return values are then
"deserialized" back into the language's interface types values too. The
`call_interface` function is the opposite, it assumes that you're coming
from wasm values and calling, e.g., a host function with interface values.

The `abi` module is refactored with an `Instruction` `enum` which lists
out what are the current set of "adapter instructions". These adapter
instructions are intended to somewhat model the eventual idea of adapter
functions and their instructions, but for now they're pretty specific to
WASI as-is today. All instructions currently model what WASI currently
does, sometimes in very specific manners. It's expected that the
`Instruction` type will be in flux as we tweak the ABI of WASI over
time, but the current set of instructions will likely only be expanded
because of maintaining compatibility with the current snapshot.

The expected usage of `Instruction` is that code generators will
implement how to generate code for each `Instruction`. This should be a
very low-level operation (generating code per instruction) and should be
in theory quite easy to implement. Not all code generators need to
implement all instructions depending on their use case. Additionally
WASI as-is today doesn't always exercise all types of instructions.

The next steps after a PR like this include starting to define a new ABI
for WASI which supports all types in all positions rather than the
current ABI which supports only a limited subset of types in some
positions. The intention is that this new ABI may add a new instruction
or two but will generally not be a large breaking change for all
existing code generators. Some new additions are expected but other than
that existing code generators updated to use this PR will require little
effort to support new ABIs.

* Add more documentation and some more tests for shorthand syntax

* Bump to 0.9.0 and bump wast dependency

* Support variants as arguments

Same as records, they use pointers

* Fix a typo

Author: alexcrichton

phases/ephemeral/docs.md

@@ -195,7 +195,7 @@
 
 ;;; File descriptor rights, determining which actions may be performed.
 (typename $rights
-  (flags (@witx bitflags u64)
+  (flags (@witx repr u64)
     ;;; The right to invoke `fd_datasync`.
     ;;
     ;;; If `path_open` is set, includes the right to invoke
@@ -392,7 +392,7 @@
 
 ;;; File descriptor flags.
 (typename $fdflags
-  (flags (@witx bitflags u16)
+  (flags (@witx repr u16)
     ;;; Append mode: Data written to the file is always appended to the file's end.
     $append
     ;;; Write according to synchronized I/O data integrity completion. Only the data stored in the file is synchronized.
@@ -429,7 +429,7 @@
 
 ;;; Which file time attributes to adjust.
 (typename $fstflags
-  (flags (@witx bitflags u16)
+  (flags (@witx repr u16)
     ;;; Adjust the last data access timestamp to the value stored in `filestat::atim`.
     $atim
     ;;; Adjust the last data access timestamp to the time of clock `clockid::realtime`.
@@ -443,15 +443,15 @@
 
 ;;; Flags determining the method of how paths are resolved.
 (typename $lookupflags
-  (flags (@witx bitflags u32)
+  (flags (@witx repr u32)
     ;;; As long as the resolved path corresponds to a symbolic link, it is expanded.
     $symlink_follow
   )
 )
 
 ;;; Open flags used by `path_open`.
 (typename $oflags
-  (flags (@witx bitflags u16)
+  (flags (@witx repr u16)
     ;;; Create file if it does not exist.
     $create
     ;;; Fail if not a directory.
@@ -470,7 +470,7 @@
 ;;; file in a filesystem, and don't fully reflect all the conditions
 ;;; which determine whether a given WASI program can access the file.
 (typename $permissions
-  (flags (@witx bitflags u8)
+  (flags (@witx repr u8)
     ;;; For files, permission to read the file.
     ;;; For directories, permission to do `readdir` and access files
     ;;; within the directory.
@@ -543,7 +543,7 @@
 ;;; The state of the file descriptor subscribed to with
 ;;; `eventtype::fd_read` or `eventtype::fd_write`.
 (typename $eventrwflags
-  (flags (@witx bitflags u16)
+  (flags (@witx repr u16)
     ;;; The peer of this socket has closed or disconnected.
     $fd_readwrite_hangup
   )
@@ -584,7 +584,7 @@
 ;;; Flags determining how to interpret the timestamp provided in
 ;;; `subscription_clock::timeout`.
 (typename $subclockflags
-  (flags (@witx bitflags u16)
+  (flags (@witx repr u16)
     ;;; If set, treat the timestamp provided in
     ;;; `subscription_clock::timeout` as an absolute timestamp of clock
     ;;; `subscription_clock::id`. If clear, treat the timestamp
@@ -643,7 +643,7 @@
 
 ;;; Flags provided to `sock_recv`.
 (typename $riflags
-  (flags (@witx bitflags u16)
+  (flags (@witx repr u16)
     ;;; Returns the message without removing it from the socket's receive queue.
     $recv_peek
     ;;; On byte-stream sockets, block until the full amount of data can be returned.
@@ -653,7 +653,7 @@
 
 ;;; Flags returned by `sock_recv`.
 (typename $roflags
-  (flags (@witx bitflags u16)
+  (flags (@witx repr u16)
     ;;; Returned by `sock_recv`: Message data has been truncated.
     $recv_data_truncated
   )
@@ -665,7 +665,7 @@
 
 ;;; Which channels on a socket to shut down.
 (typename $sdflags
-  (flags (@witx bitflags u8)
+  (flags (@witx repr u8)
     ;;; Disables further receive operations.
     $rd
     ;;; Disables further send operations.

phases/ephemeral/witx/typenames.witx

@@ -14,15 +14,13 @@
   (@interface func (export "get")
     (param $argv (@witx pointer (@witx pointer (@witx char8))))
     (param $argv_buf (@witx pointer (@witx char8)))
-    (result $error $errno)
+    (result $error (expected (error $errno)))
   )
 
   ;;; Return command-line argument data sizes.
   (@interface func (export "sizes_get")
-    (result $error $errno)
-    ;;; The number of arguments.
-    (result $argc $size)
-    ;;; The size of the argument string data.
-    (result $argv_buf_size $size)
+    ;;; Returns the number of arguments and the size of the argument string
+    ;;; data, or an error.
+    (result $error (expected (tuple $size $size) (error $errno)))
   )
 )

phases/ephemeral/witx/wasi_ephemeral_args.witx

@@ -18,9 +18,8 @@
   (@interface func (export "res_get")
     ;;; The clock for which to return the resolution.
     (param $id $clockid)
-    (result $error $errno)
     ;;; The resolution of the clock.
-    (result $resolution $timestamp)
+    (result $error (expected $timestamp (error $errno)))
   )
 
   ;;; Return the time value of a clock.
@@ -30,8 +29,7 @@
     (param $id $clockid)
     ;;; The maximum lag (exclusive) that the returned time value may have, compared to its actual value.
     (param $precision $timestamp)
-    (result $error $errno)
     ;;; The time value of the clock.
-    (result $time $timestamp)
+    (result $error (expected $timestamp (error $errno)))
   )
 )

phases/ephemeral/witx/wasi_ephemeral_clock.witx

@@ -14,15 +14,13 @@
   (@interface func (export "get")
     (param $environ (@witx pointer (@witx pointer (@witx char8))))
     (param $environ_buf (@witx pointer (@witx char8)))
-    (result $error $errno)
+    (result $error (expected (error $errno)))
   )
 
   ;;; Return environment variable data sizes.
   (@interface func (export "sizes_get")
-    (result $error $errno)
-    ;;; The number of environment variable arguments.
-    (result $environc $size)
-    ;;; The size of the environment variable data.
-    (result $environ_buf_size $size)
+    ;;; Returns the number of environment variable arguments and the size of the
+    ;;; environment variable data.
+    (result $error (expected (tuple $size $size) (error $errno)))
   )
 )

phases/ephemeral/witx/wasi_ephemeral_environ.witx

@@ -21,7 +21,7 @@
     (param $len $filesize)
     ;;; The advice.
     (param $advice $advice)
-    (result $error $errno)
+    (result $error (expected (error $errno)))
   )
 
   ;;; Force the allocation of space in a file.
@@ -32,30 +32,29 @@
     (param $offset $filesize)
     ;;; The length of the area that is allocated.
     (param $len $filesize)
-    (result $error $errno)
+    (result $error (expected (error $errno)))
   )
 
   ;;; Close a file descriptor.
   ;;; Note: This is similar to `close` in POSIX.
   (@interface func (export "close")
     (param $fd $fd)
-    (result $error $errno)
+    (result $error (expected (error $errno)))
   )
 
   ;;; Synchronize the data of a file to disk.
   ;;; Note: This is similar to `fdatasync` in POSIX.
   (@interface func (export "datasync")
     (param $fd $fd)
-    (result $error $errno)
+    (result $error (expected (error $errno)))
   )
 
   ;;; Get the attributes of a file descriptor.
   ;;; Note: This returns similar flags to `fsync(fd, F_GETFL)` in POSIX, as well as additional fields.
   (@interface func (export "fdstat_get")
     (param $fd $fd)
-    (result $error $errno)
     ;;; The buffer where the file descriptor's attributes are stored.
-    (result $stat $fdstat)
+    (result $error (expected $fdstat (error $errno)))
   )
 
   ;;; Adjust the flags associated with a file descriptor.
@@ -64,7 +63,7 @@
     (param $fd $fd)
     ;;; The desired values of the file descriptor flags.
     (param $flags $fdflags)
-    (result $error $errno)
+    (result $error (expected (error $errno)))
   )
 
   ;;; Adjust the rights associated with a file descriptor.
@@ -74,15 +73,14 @@
     ;;; The desired rights of the file descriptor.
     (param $fs_rights_base $rights)
     (param $fs_rights_inheriting $rights)
-    (result $error $errno)
+    (result $error (expected (error $errno)))
   )
 
   ;;; Return the attributes of an open file.
   (@interface func (export "filestat_get")
     (param $fd $fd)
-    (result $error $errno)
     ;;; The buffer where the file's attributes are stored.
-    (result $buf $filestat)
+    (result $error (expected $filestat (error $errno)))
   )
 
   ;;; Adjust the size of an open file. If this increases the file's size, the extra bytes are filled with zeros.
@@ -91,7 +89,7 @@
     (param $fd $fd)
     ;;; The desired file size.
     (param $size $filesize)
-    (result $error $errno)
+    (result $error (expected (error $errno)))
   )
 
   ;;; Adjust the timestamps of an open file or directory.
@@ -104,7 +102,7 @@
     (param $mtim $timestamp)
     ;;; A bitmask indicating which timestamps to adjust.
     (param $fst_flags $fstflags)
-    (result $error $errno)
+    (result $error (expected (error $errno)))
   )
 
   ;;; Set the permissions of a file or directory.
@@ -123,7 +121,7 @@
     (param $fd $fd)
     ;;; The permissions associated with the file.
     (param $permissions $permissions)
-    (result $error $errno)
+    (result $error (expected (error $errno)))
   )
 
   ;;; Read from a file descriptor, without using and updating the file descriptor's offset.
@@ -134,17 +132,15 @@
     (param $iovs $iovec_array)
     ;;; The offset within the file at which to read.
     (param $offset $filesize)
-    (result $error $errno)
     ;;; The number of bytes read.
-    (result $nread $size)
+    (result $error (expected $size (error $errno)))
   )
 
   ;;; Return a description of the given preopened file descriptor.
   (@interface func (export "prestat_get")
     (param $fd $fd)
-    (result $error $errno)
     ;;; The buffer where the description is stored.
-    (result $buf $prestat)
+    (result $error (expected $prestat (error $errno)))
   )
 
   ;;; Return a description of the given preopened file descriptor.
@@ -153,7 +149,7 @@
     ;;; A buffer into which to write the preopened directory name.
     (param $path (@witx pointer (@witx char8)))
     (param $path_len $size)
-    (result $error $errno)
+    (result $error (expected (error $errno)))
   )
 
   ;;; Write to a file descriptor, without using and updating the file descriptor's offset.
@@ -168,9 +164,8 @@
     (param $iovs $ciovec_array)
     ;;; The offset within the file at which to write.
     (param $offset $filesize)
-    (result $error $errno)
     ;;; The number of bytes written.
-    (result $nwritten $size)
+    (result $error (expected $size (error $errno)))
   )
 
   ;;; Read from a file descriptor.
@@ -179,9 +174,8 @@
     (param $fd $fd)
     ;;; List of scatter/gather vectors to which to store data.
     (param $iovs $iovec_array)
-    (result $error $errno)
     ;;; The number of bytes read.
-    (result $nread $size)
+    (result $error (expected $size (error $errno)))
   )
 
   ;;; Read directory entries from a directory.
@@ -201,9 +195,8 @@
     (param $buf_len $size)
     ;;; The location within the directory to start reading
     (param $cookie $dircookie)
-    (result $error $errno)
     ;;; The number of bytes stored in the read buffer. If less than the size of the read buffer, the end of the directory has been reached.
-    (result $bufused $size)
+    (result $error (expected $size (error $errno)))
   )
 
   ;;; Atomically replace a file descriptor by renumbering another file descriptor.
@@ -220,7 +213,7 @@
     (param $fd $fd)
     ;;; The file descriptor to overwrite.
     (param $to $fd)
-    (result $error $errno)
+    (result $error (expected (error $errno)))
   )
 
   ;;; Move the offset of a file descriptor.
@@ -231,25 +224,23 @@
     (param $offset $filedelta)
     ;;; The base from which the offset is relative.
     (param $whence $whence)
-    (result $error $errno)
     ;;; The new offset of the file descriptor, relative to the start of the file.
-    (result $newoffset $filesize)
+    (result $error (expected $filesize (error $errno)))
   )
 
   ;;; Synchronize the data and metadata of a file to disk.
   ;;; Note: This is similar to `fsync` in POSIX.
   (@interface func (export "sync")
     (param $fd $fd)
-    (result $error $errno)
+    (result $error (expected (error $errno)))
   )
 
   ;;; Return the current offset of a file descriptor.
   ;;; Note: This is similar to `lseek(fd, 0, SEEK_CUR)` in POSIX.
   (@interface func (export "tell")
     (param $fd $fd)
-    (result $error $errno)
     ;;; The current offset of the file descriptor, relative to the start of the file.
-    (result $offset $filesize)
+    (result $error (expected $filesize (error $errno)))
   )
 
   ;;; Write to a file descriptor.
@@ -262,8 +253,7 @@
     (param $fd $fd)
     ;;; List of scatter/gather vectors from which to retrieve data.
     (param $iovs $ciovec_array)
-    (result $error $errno)
     ;;; The number of bytes written.
-    (result $nwritten $size)
+    (result $error (expected $size (error $errno)))
   )
 )