syscall/js: increase performance of Call, Invoke, and New by not allowing new slices to escape onto the heap

[finnbear]

What version of Go are you using (go version)?

$ go version
go version go1.14.4 linux/amd64

Does this issue reproduce with the latest release?

Yes, in go1.15beta1

What did you do?

I used js.Call, js.New, and js.Invoke with multiple arguments, hundreds of times per frame, 60 frames per second in order to implement my WebGL game.

What did you expect to see?

The above functions are absolutely essential for getting work done in js/wasm, as they are analogous to a simple function call. They should be optimized as such. No excess go heap memory should be allocated, no extra go garbage collection. After implementing a hack to fix the issue I will go on to describe (see below), here is a CPU profile of memory allocation:

Importantly, memory allocation is 8% of the total time and does not include makeArgs allocating any slices that end up on the heap

What did you see instead?

2+ new slices allocated on the heap per call to one of the above three functions, as the first few lines of the makeArgs function...

go/src/syscall/js/js.go

Lines 361 to 363 in 60f7876

	func makeArgs(args []interface{}) ([]Value, []ref) {
	argVals := make([]Value, len(args))
	argRefs := make([]ref, len(args))

...possibly are assumed to escape to the heap via...

go/src/syscall/js/js.go

Lines 405 to 406 in 60f7876

	func valueCall(v ref, m string, args []ref) (ref, bool)

...or valueNew/valueInvoke, or if go's escape analysis optimization doesn't work on the caller's stack.

A CPU profile shows the significant penalty associated with allocating too many slices (mallocgc now accounts for over 20%), and that doesn't even include the extra garbage collector load that claims a few FPS.

I thought of adding //go:noescape before each of the above and potentially other functions implemented in javascript, but I didn't get any improvement right away. Only my hack to makeArgs worked consistently:

const maxArgs = 9 // A better hack/fix would grow the slices automatically
var argVals = make([]Value, 0, maxArgs)
var argRefs = make([]ref, 0, maxArgs)

func makeArgs(args []interface{}) ([]Value, []ref) {
	for i, _ := range argVals {
		argVals[i] = Value{}
	}
	for i, _ := range argRefs { // I don't think this is strictly necessary
		argRefs[i] = 0
	}

	argVals = argVals[:len(args)]
	argRefs = argRefs[:len(args)]

	for i, arg := range args {
		v := ValueOf(arg)
		argVals[i] = v
		argRefs[i] = v.ref
	}
	return argVals, argRefs
}

Another potential solution, building on the above hack, would be to make makeArgs take slices, to put the values/refs in, as arguments (instead of returning new slices). The caller could deal with efficiently handling the memory. Maybe this could help go's escape analysis, in conjunction with //go:noescape.

Side note: Ideally, a solution would address the heap allocation of an interface for certain arguments. Hundreds of thousands get allocated before the garbage collector runs, causing the garbage collector to take 12-20ms.

[cagedmantis]

/cc @neelance @cherrymui @hajimehoshi

[neelance]

@cherrymui Would this be a good case for sync.Pool? Would it be okay to restrict the maximum number of arguments so we can have a pool of slices with a fixed size?

[finnbear]

I've formulated 3 fully complete solutions to the problem:

1. Using sync/pool as suggested by @neelance: master...finnbear:makeargs-syncpool

• Performance: ~50% improvement to time spent in malloc related to makeArgs
• Garbage collection: Some (mainly interfaces)
• Benefits: Using sync/pool is standard, safe, and future-proof (multithreaded WASM)
• Issues: Interface allocations undo half of the performance gain of not allocating slices, doesn't help performance for higher than X arguments (X = 16 in my example)

2. Using a slice as a pool: master...finnbear:makeargs-slicepool

• Performance: ~99.5% improvement to time spent in malloc related to makeArgs
• Garbage collection: Very little
• Benefits: Near perfect performance and is safe
• Issues: Doesn't help peformance for higher than X arguments (X = 16 in my example), and not future-proof if multiple threads are to be supported.

3. Using a single slice (like my original hack): master...finnbear:makeargs-singleslice

• Performance: ~99.999% improvement to time spent in malloc related to makeArgs
• Garbage collection: Almost none
• Benefits: Perfect performance, simple, probably safe
• Issues: I logically came to the conclusion that this is 100% safe but it may not seem that way (in the case of re-entrant recursive calls). It is also not future proof if multiple threads are to be supported.
  • In my understanding, JS/WASM uses a single, unified call tree
  • I don't think it's possible for execution to re-enter the code at a position that is unsafe

My suggestion is for you to:

1. Investigate and implement solution 3
2. Investigate and implement solution 2
3. Try to get //go:noescape to work (I could not)
4. Investigate and implement solution 1
5. Do nothing

For what it's worth, I have validated all of the above solutions in my WebAssembly game to get an idea of their performance, and observed no crashing/panicking/issues.

@neelance @cherrymui @hajimehoshi

[finnbear]

I've been looking for a solution to this ever since I filed the issue, and ended up with a patch that solves the problem (both the slice problem and the interface escaping to heap problem) for me. Specifically, heap allocations per frame went from ~1500-40,000 (almost all related to syscall/js) to ~60-300 (which are unrelated to syscall/js). Note that this makes ValueOf unsafe to call except during Call, Invoke, and New. If this was being integrated into the standard library, one solution would be to leave ValueOf as-is and make a second, unexported version with the noescape change.

patch -u $GOROOT/src/syscall/js/js.go -i see_below.patch

--- js.go.dist	2021-01-23 15:50:00.931644132 -0800
+++ js.go	2021-01-23 17:31:54.938784167 -0800
@@ -145,6 +145,11 @@
 	return valueGlobal
 }
 
+func noescape(foo interface{}) interface{} {
+	bar := uintptr(unsafe.Pointer(&foo))
+	return *((*interface{})(unsafe.Pointer(bar ^ 0)))
+}
+
 // ValueOf returns x as a JavaScript value:
 //
 //  | Go                     | JavaScript             |
@@ -159,7 +164,8 @@
 //  | map[string]interface{} | new object             |
 //
 // Panics if x is not one of the expected types.
-func ValueOf(x interface{}) Value {
+func ValueOf(a interface{}) Value {
+	x := noescape(a)
 	switch x := x.(type) {
 	case Value: // should precede Wrapper to avoid a loop
 		return x
@@ -215,11 +221,12 @@
 			o.Set(k, v)
 		}
 		return o
-	default:
-		panic("ValueOf: invalid value")
 	}
+	runtime.KeepAlive(a)
+	panic("ValueOf: invalid value")
 }
 
+//go:noescape
 func stringVal(x string) ref
 
 // Type represents the JavaScript type of a Value.
@@ -303,6 +310,7 @@
 	return r
 }
 
+//go:noescape
 func valueGet(v ref, p string) ref
 
 // Set sets the JavaScript property p of value v to ValueOf(x).
@@ -317,6 +325,7 @@
 	runtime.KeepAlive(xv)
 }
 
+//go:noescape
 func valueSet(v ref, p string, x ref)
 
 // Delete deletes the JavaScript property p of value v.
@@ -329,6 +338,7 @@
 	runtime.KeepAlive(v)
 }
 
+//go:noescape
 func valueDelete(v ref, p string)
 
 // Index returns JavaScript index i of value v.
@@ -342,6 +352,7 @@
 	return r
 }
 
+//go:noescape
 func valueIndex(v ref, i int) ref
 
 // SetIndex sets the JavaScript index i of value v to ValueOf(x).
@@ -356,11 +367,24 @@
 	runtime.KeepAlive(xv)
 }
 
+//go:noescape
 func valueSetIndex(v ref, i int, x ref)
 
-func makeArgs(args []interface{}) ([]Value, []ref) {
-	argVals := make([]Value, len(args))
-	argRefs := make([]ref, len(args))
+var (
+	argValsSlice []Value
+	argRefsSlice []ref
+)
+
+func makeArgs(args []interface{}) (argVals []Value, argRefs []ref) {
+	for i, _ := range argValsSlice {
+		argValsSlice[i] = Value{}
+	}
+	if len(args) > cap(argValsSlice) {
+		argValsSlice = make([]Value, 0, len(args))
+		argRefsSlice = make([]ref, 0, len(args))
+	}
+	argVals = argValsSlice[:len(args)]
+	argRefs = argRefsSlice[:len(args)]
 	for i, arg := range args {
 		v := ValueOf(arg)
 		argVals[i] = v
@@ -380,6 +404,7 @@
 	return r
 }
 
+//go:noescape
 func valueLength(v ref) int
 
 // Call does a JavaScript call to the method m of value v with the given arguments.
@@ -402,6 +427,7 @@
 	return makeValue(res)
 }
 
+//go:noescape
 func valueCall(v ref, m string, args []ref) (ref, bool)
 
 // Invoke does a JavaScript call of the value v with the given arguments.
@@ -421,6 +447,7 @@
 	return makeValue(res)
 }
 
+//go:noescape
 func valueInvoke(v ref, args []ref) (ref, bool)
 
 // New uses JavaScript's "new" operator with value v as constructor and the given arguments.
@@ -440,6 +467,7 @@
 	return makeValue(res)
 }
 
+//go:noescape
 func valueNew(v ref, args []ref) (ref, bool)
 
 func (v Value) isNumber() bool {
@@ -539,8 +567,10 @@
 	return string(b)
 }
 
+//go:noescape
 func valuePrepareString(v ref) (ref, int)
 
+//go:noescape
 func valueLoadString(v ref, b []byte)
 
 // InstanceOf reports whether v is an instance of type t according to JavaScript's instanceof operator.
@@ -551,6 +581,7 @@
 	return r
 }
 
+//go:noescape
 func valueInstanceOf(v ref, t ref) bool
 
 // A ValueError occurs when a Value method is invoked on
@@ -577,6 +608,7 @@
 	return n
 }
 
+//go:noescape
 func copyBytesToGo(dst []byte, src ref) (int, bool)
 
 // CopyBytesToJS copies bytes from src to dst.
@@ -591,4 +623,5 @@
 	return n
 }
 
+//go:noescape
 func copyBytesToJS(dst ref, src []byte) (int, bool)

[hajimehoshi]

I'm excited about this suggestion and change, @finnbear!

[hajimehoshi]

Issues: I logically came to the conclusion that this is 100% safe but it may not seem that way (in the case of re-entrant recursive calls). It is also not future proof if multiple threads are to be supported.

I believe so too: the argument slice is used only when a callback is invoked from the Go side, and this invoking should never happen recursively. @neelance what do you think?

I don't have any insights about go:noescape, but do we really need them?

[neelance]

The documentation of go:noescape says:

The //go:noescape directive must be followed by a function declaration without a body (meaning that the function has an implementation not written in Go). It specifies that the function does not allow any of the pointers passed as arguments to escape into the heap or into the values returned from the function. This information can be used during the compiler's escape analysis of Go code calling the function.

If I understand this correctly, then //go:noescape makes no difference for func valueIndex(v ref, i int) ref since neither ref nor int are pointers.

For a case like func copyBytesToGo(dst []byte, src ref) (int, bool) it should be fine to apply //go:noescape because the pointer of []byte is only accessed inside of copyBytesToGo and not stored anywhere else.

It might also apply to string parameters.

I don't understand what's going on with x := noescape(a) but my gut feeling is bad. 😄
Please investigate which lines exactly are responsible for moving the value to the heap.