Merge pull request WebAssembly#324 from WebAssembly/merge-funcref3

Merge upstream funcref + main repo

Author: rossberg

.github/workflows/main.yml

@@ -21,6 +21,17 @@ jobs:
       - run: opam install --yes ocamlbuild.0.14.0
       - run: cd interpreter && opam exec make all
 
+  ref-interpreter-js-library:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - name: Setup OCaml
+        uses: ocaml/setup-ocaml@v2
+        with:
+          ocaml-compiler: 4.13.x
+      - run: opam install --yes ocamlbuild.0.14.0 ocamlfind.1.9.5 js_of_ocaml.4.0.0 js_of_ocaml-ppx.4.0.0
+      - run: cd interpreter && opam exec make wast.js
+
   build-js-api-spec:
     runs-on: ubuntu-latest
     steps:

document/core/appendix/algorithm.rst

@@ -21,7 +21,10 @@ The algorithm is expressed in typed pseudo code whose semantics is intended to b
 Data Structures
 ~~~~~~~~~~~~~~~
 
-Types are representable as a set of enumerations.
+Types
+.....
+
+Value types are representable as a set of enumerations.
 
 .. code-block:: pseudo
 
@@ -34,11 +37,11 @@ Types are representable as a set of enumerations.
    func is_num(t : val_type) : bool =
      return t = I32 || t = I64 || t = F32 || t = F64 || t = Bot
 
-   func is_vec(t : val_type | Unknown) : bool =
-     return t = V128 || t = Unknown
+   func is_vec(t : val_type) : bool =
+     return t = V128 || t = Bot
 
-   func is_ref(t : val_type | Unknown) : bool =
-     return t = not (is_num t || is_vec t) || t = Bot
+   func is_ref(t : val_type) : bool =
+     return not (is_num t || is_vec t) || t = Bot
 
 Equivalence and subtyping checks can be defined on these types.
 
@@ -65,36 +68,64 @@ Equivalence and subtyping checks can be defined on these types.
      return matches_heap(t1.heap, t2.heap) && matches_null(t1.null, t2.null)
 
    func matches(t1 : val_type, t2 : val_type) : bool =
-     return
-       (is_num t1 && is_num t2 & t1 = t2) ||
-       (is_ref t1 && is_ref t2 & matches_ref(t1, t2)) ||
-       t1 = Bot
+     switch (t1, t2)
+       case (Ref(_), Ref(_))
+         return matches_ref(t1, t2)
+       case (Bot, _)
+         return true
+       case (_, _)
+         return t1 = t2
+
+Context
+.......
+
+Validation requires a :ref:`context <context>` for checking uses of :ref:`indices <syntax-index>`.
+For the purpose of presenting the algorithm, it is maintained in a set of global variables:
+
+.. code-block:: pseudo
+
+   var locals : array(val_type)
+   var locals_init : array(bool)
+   var globals : array(global_type)
+   var funcs : array(func_type)
+   var tables : array(table_type)
+   var mems : array(mem_type)
+
+This assumes suitable representations for the various :ref:`types <syntax-type>` besides :code:`val_type`, which are omitted here.
+
+For locals, there is an additional array recording the initialization status of each local.
+
+Stacks
+......
 
-The algorithm uses two separate stacks: the *value stack* and the *control stack*.
-The former tracks the :ref:`types <syntax-valtype>` of operand values on the :ref:`stack <stack>`,
-the latter surrounding :ref:`structured control instructions <syntax-instr-control>` and their associated :ref:`blocks <syntax-instr-control>`.
+The algorithm uses three separate stacks: the *value stack*, the *control stack*, and the *initialization stack*.
+The value stack tracks the :ref:`types <syntax-valtype>` of operand values on the :ref:`stack <stack>`.
+The control stack tracks surrounding :ref:`structured control instructions <syntax-instr-control>` and their associated :ref:`blocks <syntax-instr-control>`.
+The initialization stack records all :ref:`locals <syntax-local>` that have been initialized since the beginning of the function.
 
 .. code-block:: pseudo
 
    type val_stack = stack(val_type)
+   type init_stack = stack(u32)
 
    type ctrl_stack = stack(ctrl_frame)
    type ctrl_frame = {
      opcode : opcode
      start_types : list(val_type)
      end_types : list(val_type)
-     height : nat
+     val_height : nat
+     init_height : nat
      unreachable : bool
    }
 
-For each value, the value stack records its :ref:`value type <syntax-valtype>`. 
-For each entered block, the control stack records a *control frame* with the originating opcode, the types on the top of the operand stack at the start and end of the block (used to check its result as well as branches), the height of the operand stack at the start of the block (used to check that operands do not underflow the current block), and a flag recording whether the remainder of the block is unreachable (used to handle :ref:`stack-polymorphic <polymorphism>` typing after branches).
+For each entered block, the control stack records a *control frame* with the originating opcode, the types on the top of the operand stack at the start and end of the block (used to check its result as well as branches), the height of the operand stack at the start of the block (used to check that operands do not underflow the current block), the height of the initialization stack at the start of the block (used to reset initialization status at the end of the block), and a flag recording whether the remainder of the block is unreachable (used to handle :ref:`stack-polymorphic <polymorphism>` typing after branches).
 
-For the purpose of presenting the algorithm, the operand and control stacks are simply maintained as global variables:
+For the purpose of presenting the algorithm, these stacks are simply maintained as global variables:
 
 .. code-block:: pseudo
 
    var vals : val_stack
+   var inits : init_stack
    var ctrls : ctrl_stack
 
 However, these variables are not manipulated directly by the main checking function, but through a set of auxiliary functions:
@@ -122,7 +153,7 @@ However, these variables are not manipulated directly by the main checking funct
    func pop_ref() : ref_type =
      let actual = pop_val()
      error_if(not is_ref(actual))
-     if actual = Bot then return Ref(Bot, false)
+     if (actual = Bot) return Ref(Bot, false)
      return actual
 
    func push_vals(types : list(val_type)) = foreach (t in types) push_val(t)
@@ -149,25 +180,49 @@ Finally, there are accumulative functions for pushing or popping multiple operan
    so that, e.g., :code:`ctrls[0]` accesses the element pushed last.
 
 
+The initialization stack and the initialization status of locals is manipulated through the following functions:
+
+.. code-block:: pseudo
+
+   func get_local(idx : u32) =
+     error_if(not locals_init[idx])
+
+   func set_local(idx : u32) =
+     if (not locals_init[idx])
+       inits.push(idx)
+       locals_init[idx] := true
+
+   func reset_locals(height : nat) =
+     while (inits.size() > height)
+       locals_init[inits.pop()] := false
+
+Getting a local verifies that it is known to be initialized.
+When a local is set that was not set already,
+then its initialization status is updated and the change is recorded in the initialization stack.
+Thus, the initialization status of all locals can be reset to a previous state by denoting a specific height in the initialization stack.
+
+The size of the initialization stack is bounded by the number of (non-defaultable) locals in a function, so can be preallocated by an algorithm.
+
 The control stack is likewise manipulated through auxiliary functions:
 
 .. code-block:: pseudo
 
    func push_ctrl(opcode : opcode, in : list(val_type), out : list(val_type)) =
-     let frame = ctrl_frame(opcode, in, out, vals.size(), false)
+     let frame = ctrl_frame(opcode, in, out, vals.size(), inits.size(), false)
      ctrls.push(frame)
      push_vals(in)
 
    func pop_ctrl() : ctrl_frame =
      error_if(ctrls.is_empty())
      let frame = ctrls[0]
      pop_vals(frame.end_types)
-     error_if(vals.size() =/= frame.height)
+     error_if(vals.size() =/= frame.val_height)
+     reset_locals(frame.init_height)
      ctrls.pop()
      return frame
 
    func label_types(frame : ctrl_frame) : list(val_types) =
-     return (if frame.opcode == loop then frame.start_types else frame.end_types)
+     return (if (frame.opcode = loop) frame.start_types else frame.end_types)
 
    func unreachable() =
      vals.resize(ctrls[0].height)
@@ -179,18 +234,22 @@ It allocates a new frame record recording them along with the current height of
 Popping a frame first checks that the control stack is not empty.
 It then verifies that the operand stack contains the right types of values expected at the end of the exited block and pops them off the operand stack.
 Afterwards, it checks that the stack has shrunk back to its initial height.
+Finally, it undoes all changes to the initialization status of locals that happend inside the block.
 
 The type of the :ref:`label <syntax-label>` associated with a control frame is either that of the stack at the start or the end of the frame, determined by the opcode that it originates from.
 
 Finally, the current frame can be marked as unreachable.
 In that case, all existing operand types are purged from the value stack, in order to allow for the :ref:`stack-polymorphism <polymorphism>` logic in :code:`pop_val` to take effect.
+Because every function has an implicit outermost label that corresponds to an implicit block frame,
+it is an invariant of the validation algorithm that there always is at least one frame on the control stack when validating an instruction, and hence, `ctrls[0]` is always defined.
 
 .. note::
    Even with the unreachable flag set, consecutive operands are still pushed to and popped from the operand stack.
    That is necessary to detect invalid :ref:`examples <polymorphism>` like :math:`(\UNREACHABLE~(\I32.\CONST)~\I64.\ADD)`.
    However, a polymorphic stack cannot underflow, but instead generates :code:`Bot` types as needed.
 
 
+
 .. index:: opcode
 
 Validation of Opcode Sequences
@@ -199,10 +258,6 @@ Validation of Opcode Sequences
 The following function shows the validation of a number of representative instructions that manipulate the stack.
 Other instructions are checked in a similar manner.
 
-.. note::
-   Various instructions not shown here will additionally require the presence of a validation :ref:`context <context>` for checking uses of :ref:`indices <syntax-index>`.
-   That is an easy addition and therefore omitted from this presentation.
-
 .. code-block:: pseudo
 
    func validate(opcode) =
@@ -221,7 +276,7 @@ Other instructions are checked in a similar manner.
          let t2 = pop_val()
          error_if(not (is_num(t1) && is_num(t2) || is_vec(t1) && is_vec(t2)))
          error_if(t1 =/= t2 && t1 =/= Bot && t2 =/= Bot)
-         push_val(if (t1 == Bot) t2 else t1)
+         push_val(if (t1 = Bot) t2 else t1)
 
        case (select t)
          pop_val(I32)
@@ -237,6 +292,14 @@ Other instructions are checked in a similar manner.
          let rt = pop_ref()
          push_val(Ref(rt.heap, false))
 
+       case (local.get x)
+         get_local(x)
+         push_val(locals[x])
+
+       case (local.set x)
+         pop_val(locals[x])
+         set_local(x)
+
        case (unreachable)
          unreachable()
 
@@ -295,7 +358,7 @@ Other instructions are checked in a similar manner.
          let rt = pop_ref()
          if (rt.heap =/= Bot)
            error_if(not is_def(rt.heap))
-           let ft = lookup_func_type(rt.heap.idx)  // TODO
+           let ft = funcs[rt.heap.idx]
            pop_vals(ft.params)
            push_vals(ft.results)
 

document/core/appendix/changes.rst

@@ -53,7 +53,7 @@ Added |FUNCREF| and |EXTERNREF| as new value types and respective instructions [
 
 * New :ref:`reference instructions <syntax-instr-ref>`: |REFNULL|, |REFFUNC|, |REFISNULL|
 
-* Enrich :ref:`parametric instruction <syntax-instr-parametric>`: |SELECT| with optional type immediate
+* Extended :ref:`parametric instruction <syntax-instr-parametric>`: |SELECT| with optional type immediate
 
 * New :ref:`declarative <syntax-elemmode>` form of :ref:`element segment <syntax-elem>`
 
@@ -138,6 +138,33 @@ Added vector type and instructions that manipulate multiple numeric values in pa
 * New injection/projection :ref:`vector instructions <syntax-instr-vec>`: :math:`\K{i}\!N\!\K{x}\!M\!\K{.splat}`, :math:`\K{f}\!N\!\K{x}\!M\!\K{.splat}`, :math:`\K{i}\!N\!\K{x}\!M\!\K{.bitmask}`
 
 
+Release 2.?
+~~~~~~~~~~~
+
+.. index:: reference, reference type, heap type, value type, local, local type, instruction, instruction type, table, function, function type, matching, subtyping
+
+Typeful References
+..................
+
+Added more precise types for references [#proposal-typedref]_.
+
+* New generalised form of :ref:`reference types <syntax-reftype>`: :math:`(\REF~\NULL^?~\heaptype)`
+
+* New class of :ref:`heap types <syntax-heaptype>`: |FUNC|, |EXTERN|, :math:`\typeidx`
+
+* Basic :ref:`subtyping <match>` on :ref:`reference <match-reftype>` and :ref:`value <match-valtype>` types
+
+* New :ref:`reference instructions <syntax-instr-ref>`: |REFASNONNULL|, |BRONNULL|, |BRONNONNULL|
+
+* New :ref:`control instruction <syntax-instr-control>`: |CALLREF|
+
+* Refined typing of :ref:`reference instruction <syntax-instr-ref>` |REFFUNC| with more precise result type
+
+* Refined typing of :ref:`local instructions <valid-instr-variable>` and :ref:`instruction sequences <valid-instr-seq>` to track the :ref:`initialization status <syntax-init>` of :ref:`locals <syntax-local>` with non-:ref:`defaultable <valid-defaultable>` type
+
+* Extended :ref:`table definitions <syntax-table>` with optional initializer expression
+
+
 .. [#proposal-signext]
    https://github.com/WebAssembly/spec/tree/main/proposals/sign-extension-ops/
 
@@ -155,3 +182,6 @@ Added vector type and instructions that manipulate multiple numeric values in pa
 
 .. [#proposal-vectype]
    https://github.com/WebAssembly/spec/tree/main/proposals/simd/
+
+.. [#proposal-typedref]
+   https://github.com/WebAssembly/spec/tree/main/proposals/function-references/