Investigate possible performance wins with TextEncoder#encodeInto

[alexcrichton]

There's some discussion starting here about how we can probably improve the current logic of using encodeInto through some more clever usage and possibly some magic numbers. We should take a look into this! Ideally we'd also take a look at actual performance numbers when doing so.

[hsivonen]

I put a slightly polished restatement of my previous comment onto MDN.

It would be nice to copy and paste the code resulting from fixing this issue into the "Examples" section of the MDN article.

[alexcrichton]

Oh thanks for the link @hsivonen!

[RReverser]

Note that another alternative that might be worth considering and measuring is calculating UTF-8 byte length upfront.

I've already added usage of Buffer.byteLength(...) in #1391 for Node.js, but for other targets we can do something like:

let size = arg.length;
for (let i = 0; i < arg.length; i++) {
    let code = arg.charCodeAt(i);
    if (code > 0x7f) size++;
    if (code > 0x7ff) size++;
    if (code >= 0xD800 && code <= 0xDBFF) i++; // high surrogate
}

After that, we can allocate the right size right away.

[hsivonen]

Note that another alternative that might be worth considering and measuring is calculating UTF-8 byte length upfront.

Gecko used to do this for its internal UTF-16 to UTF-8 conversions, but doing imprecise allocations was better for performance. (With jemalloc, allocations are imprecise anyway. With a precise allocator, it might be more interesting to attempt to do precise allocations.)

[RReverser]

With jemalloc, allocations are imprecise anyway.

Does WASM in Rust use jemalloc? AFAIK Rust switched from it to the system allocator by default, but now that I think about it, I'm not sure which allocator is considered "system" and included in the WASM target.

Although maybe most people use wee_alloc instead of the default one? Worth investigating.

[alexcrichton]

The wasm target does not use jemalloc, it uses a port of dlmalloc.

[RReverser]

Good to know. So, point stands - probably worth optimising for it and/or wee_alloc if we do try to take allocator characteristics into account.

Alternatively, I'd rely on bucket allocators to already round up any allocations to the bucket size and then realloc that stays within the bucket is essentially no-op, so we probably don't need to try and do that manually.

[alexcrichton]

This was done originally in #1414 and we can always follow up with further improvements if necessary!