crypto/rand: Reader is internally buffered inconsistently on different platforms

[takeyourhatoff]

Please answer these questions before submitting your issue. Thanks!

1. What version of Go are you using (go version)?
   1.6.3
2. What operating system and processor architecture are you using (go env)?
   GOARCH="amd64"
   GOBIN=""
   GOEXE=""
   GOHOSTARCH="amd64"
   GOHOSTOS="linux"
   GOOS="linux"
   GOPATH="/home/tyho/go"
   GORACE=""
   GOROOT="/usr/lib/golang"
   GOTOOLDIR="/usr/lib/golang/pkg/tool/linux_amd64"
   GO15VENDOREXPERIMENT="1"
   CC="gcc"
   GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0"
   CXX="g++"
   CGO_ENABLED="1"
3. What did you expect to see?
   rand.Reader to consistently internally buffer, or not internally buffer calls to Read regardless of platform.
4. What did you see instead?
   On generic Unix, including old versions of Linux (without a getrandom syscall), rand.Reader reads from /dev/urandom, and buffers calls to Read using the bufio package. On modern Linux however, rand.Read requests are implemented using the new getrandom syscall, but the calls are not buffered.
5. What did you do?
   I tested to see if internally buffering output on platforms with the getrandom syscall available would make a significant difference to the performance of small 16B reads:

https://play.golang.org/p/80fyLlBFQb

On my machine, internally buffering the output of Read doubled performance compared with raw Reads. Tv` suggested that the unbuffered Reads may be quicker when multithreaded due to bufio adding a mutex. He wrote a benchmark to test this theory but it turned out not to be true:

https://play.golang.org/p/3sO47iNhK5

[ianlancetaylor]

Dup of #16124 ?

[josharian]

Not a dup, I think. That was math/rand; this is crypto/rand.

[agl]

Not buffering is actually a nice feature of the getrandom code. Although Go programs don't fork, thus eliminating one of the classic sources of problems, these days there's hibernation, VM migrations etc.

What's the situation where the syscall performance is a problem? (Is it CBC-mode in TLS again?)

[CAFxX]

Still applies to 1.12 and tip. See #33256 for benchmarks showing order-of-magnitude differences for small reads via a bufio.Reader.

Although Go programs don't fork, thus eliminating one of the classic sources of problems, these days there's hibernation, VM migrations etc.

Don't these apply equally to the entropy pool in the kernel? Why is it a concern only for entropy buffered in process? (Also note that the /dev/urandom fallback already buffers.)

What's the situation where the syscall performance is a problem?

We ran into this in the profile of a workload where we need to generate tokens for authentication (unrelated to TLS).

[gopherbot]

Change https://golang.org/cl/331269 mentions this issue: crypto/rand: buffer the entropy obtained by calling unix.GetRandom

[magical]

I'd like to mildly object to CL 331269. Not buffering has nice security properties that agl mentioned above, getentropy should be fairly fast, and in cases where the syscall overhead is actually noticable it should be simple for the user to wrap crypto/rand.Reader in a bufio.Reader (which is essentially what the CL does). The user is in a better position to know when buffers can be shared, too. By introducing a global buffer, the CL introduces contention where there needn't be any.

[andrewchambers]

I think its better to never buffer it, then clients can decide to buffer it without accidentally double buffering on some platforms.

[ianlancetaylor]

CC @golang/security

Doesn't look like this is happening for 1.19. Moving milestone to backlog. Please recategorize if appropriate.

[magical]

I think this issue is fixed/superseded by the pair of commits:

• d68a8d0 (which added buffering for getentropy/getrandom)
• 3ae414c (which removed all buffering)

[rolandshoemaker]

Yes, I believe this was fixed (and is included in 1.19.)