Finalizing the instruction set

[tlively]

It's been a while since we introduced the criteria in #203 for evaluating new proposed instructions. Since then, we have made significant progress on the spec tests, text, and interpreter, to the point where finalizing the instruction set will soon be the main phase 4 blocker.

We should continue evaluating currently proposed instructions using the previous criteria and we should continue to be open to bug fixes and improvements to our existing instructions, but I propose that effective immediately (upon stakeholder agreement) we start labeling all new proposed instructions as post-mvp. Does that sound reasonable to everyone?

[lars-t-hansen]

I think this is a good idea, but as part of this I think we should also come up with and make public an overall strategy for how incremental extensions will be made to the (fixed-width) instruction set. Having such a strategy will make it easier to draw the line for the MVP.

[tlively]

@lars-t-hansen, that's a good idea. My assumption was that we would lean heavily on the normal proposal process for follow-on extensions, but the more we sketch out a framework for how we will evaluate and group post-MVP instructions now, the more streamlined that those follow-on proposals will be. Did you have specific thoughts about what our strategy should be?

I think that one of the things that has worked out really well in this proposal has been having clear technical guidelines (e.g. portability, determinism) to focus development and prevent feature creep. Follow-on extensions to this proposal (i.e. not radically different proposals like scalable vectors or machine-specific instructions) will be easier to scope if they each focus on doing one thing well, either by focusing on a particular use case (e.g. neural networks) or a single instruction family (e.g. fma).

[lars-t-hansen]

@tlively, I think grouping instructions thematically is going to be very helpful and I support that. We need to be careful about how we define the groups so that we don't get into another (bulk memory, reference types, threads) quagmire, but maybe that's not too hard with these low-level instructions.

Part of our SIMD strategy will likely have to do with dealing with machine specific corner cases, though. IIRC both arm64 and x64 have support for 8-bit operations but differ in some cases. Also IIRC, the QFMA class of instructions had similar issues. These are desirable instruction (groups) and we should probably be talking about principles for handling cross-architecture incompatibilities (beyond NaN cases) so that we can ship this type of functionality.

[binji]

These are desirable instruction (groups) and we should probably be talking about principles for handling cross-architecture incompatibilities (beyond NaN cases) so that we can ship this type of functionality.

Right, I think we need to start talking again about conditional compilation for these sorts of features -- either as a generic feature (https://github.com/webassembly/conditional-sections) or as something specific to SIMD.

[Maratyszcza]

I think we need to set a date for finalizing the instruction set, but this date should be in the future, not imminent. Right now the SIMD specification is missing whole classes of instructions (e.g. #348), and finalizing this early would make it unusable for a set of use-cases.

IMO a good criterion for finalizing the ISA would be to check that all instructions from ARM NEON and x86 SSE4.1 either:

1. Have a close equivalent in WAsm SIMD, or
2. Can't be specified in a portable and efficient way across architectures

[penzn]

IMO a good criterion for finalizing the ISA would be to check that all instructions from ARM NEON and x86 SSE4.1

Adding equivalents of "missing" hardware instructions does not guarantee users would be able to leverage them. In particular, I'd like to avoid the situation when we are going to add some amount of instructions which all map to existing hardware ops, but overall the speedup vs scalar is around 1x. Also, there are other desired properties aside from mapping of hardware instructions - code size is probably the most important criteria.

If we are serious about evaluating whether or not we can add any more instructions we need to have a set of actual, not micro, benchmarks linked from this repo, which we can test for both speed and size of generated binaries. If we can't come up with something like this, I'd rather freeze the set against adding new operations (fixing already added ops is OK).

[tlively]

Getting full coverage of portable native instructions is an explicit non-goal of this proposal. That's why we've always had criteria demanding demonstrated speedups in real-world use cases. @Maratyszcza, how far in the future would you want the cutoff date to be?

[Maratyszcza]

IMO, we need to hold until at least end of the year, to make sure SIMD specification works for multiple classes of applications.

I'm concerned that SIMD specification might be too tied to the small number of applications ported so far, and missing important instructions needed for other applications. Checking instructions present in SSE4.1 and NEON is a good exercise in looking for missing instructions, as the designers of native SIMD instructions sets already had to make similar decisions on which instructions to include.

[binji]

IMO, we need to hold until at least end of the year, to make sure SIMD specification works for multiple classes of applications.

Are there specific teams that we're waiting for? I'm curious why an additional 3 months is necessary now, since SIMD has been available experimentally for a year+ now.

Checking instructions present in SSE4.1 and NEON is a good exercise in looking for missing instructions, as the designers of native SIMD instructions sets already had to make similar decisions on which instructions to include.

Agreed, but there's no requirement that these instructions go into the SIMD MVP. We already know that there are instructions that will never be included in the SIMD MVP (e.g. anything with non-determinism), so I think it's reasonable to hold off on other instructions too, knowing that they can be bucketed into the next round of instructions.

[Maratyszcza]

Are there specific teams that we're waiting for? I'm curious why an additional 3 months is necessary now, since SIMD has been available experimentally for a year+ now.

Origin Trial is active since July 14, 2020 release of Chrome 84, just over two months. AFAICT, many teams were reluctant to port code to WebAssembly SIMD before OT as there wasn't any path to production. Some types of applications we haven't seen yet are:

• Scientific computing applications, in particular using double-precision and mixed-precision (missing whole sets of instructions, see Double-precision SIMD conversions #348)
• Hashing and cryptography (e.g. missing carry-less multiplication, unclear how well WAsm SIMD works for bitslicing algorithms)
• Algorithms using Hamming weight (unclear if existing dynamic shuffle instruction is sufficient, or we need dedicated Population Count instructions)

Besides, I have several instruction proposals in review (#124, #127, #237, #247, #290, #350, #352), and I'd like to see them accepted/rejected based on performance evaluation. I think three months would be enough to finish it.

Agreed, but there's no requirement that these instructions go into the SIMD MVP. We already know that there are instructions that will never be included in the SIMD MVP (e.g. anything with non-determinism), so I think it's reasonable to hold off on other instructions too, knowing that they can be bucketed into the next round of instructions.

WebAssembly doesn't have any mechanism for runtime feature detection, and thus software must threat different WAsm extensions as separate architectures (e.g. TensorFlow.js has to recompile 3 times for WAsm MVP, WAsm SIMD, WAsm SIMD+Threads). Supporting both WAsm SIMD and WAsm SIMD2 is not like supporting SSE2 and SSE4, it is like supporting both x86 and ARM. Because of the resulting deployment pains, I want WAsm SIMD MVP to be sufficiently feature-complete that most applications could target it.

[binji]

Some types of applications we haven't seen yet are ...

Thanks, this is helpful. I'm curious if we know whether anyone is actively working on porting for these applications, however. I wouldn't want to wait 3 months and be in the same place we are now.

Besides, I have several instruction proposals in review (#124, #127, #237, #247, #290, #350, #352), and I'd like to see them accepted/rejected based on performance evaluation. I think three months would be enough to finish it.

Some of these are older proposals, and I agree they should be resolved. Some of them are much newer, however. I'm a little concerned that we will keep finding another few instructions to add and not be able to move forward with the MVP.

WebAssembly doesn't have any mechanism for runtime feature detection, and thus software must threat different WAsm extensions as separate architectures

Agreed, this is an issue @lars-t-hansen brought up above. I think we should seriously consider adding an explicit runtime SIMD feature detection instruction for this purpose, so we don't have to increase the size of the support matrix you mention.

[tlively]

I think we should seriously consider adding an explicit runtime SIMD feature detection instruction for this purpose, so we don't have to increase the size of the support matrix you mention.

See #356!

[jan-wassenberg]

Good that Marat mentions crypto - AES would probably be helpful, it's very widely used.

PPC and x86 have straightforward instructions but unfortunately ARM splits the operation differently: instead of ShiftRows, SubBytes, MixColumns, XOR (x86 AESENC), they have ShiftRows+SubBytes+XOR(AESE) and MixColumns (AESMC), and the AESE and AESMC should be consecutive to enable fusing. Anyone interested in defining a way to bridge that gap?

[Maratyszcza]

Cryptographic primitives are already exposed through Web Cryptography API, so I see little value in having AES instructions in the spec: they would be nice for parallel RNG, but given that multithreaded WAsm applications are uncommon, it wouldn’t have much usage. I was more thinking about carryless multiplication, which has use-cases in hashing beyond cryptography.