fs.readFile does not partition reads - threadpool exhaustion #17047
Description
Version: Everything
Platform: All
Subsystem: FS
Description
The node master, and all previous releases AFAIK, implement fs.readFile as a call to stat, followed by a request to read the entire file based on the size reported by stat.
Why is this bad?
The effect is to place on the libuv threadpool a "giant" read request, occupying the libuv thread until it completes. While readFile certainly requires buffering the entire file contents, it can partition the read into smaller buffers (as is done on other read paths) to avoid threadpool squatting.
If the file is relatively large or stored on a slow medium, reading the entire file in one shot seems particularly harmful, and seems to present a possible DoS vector.
Downsides to partitioning?
I don't think partitioning the read like this raises any additional risk of read-write races on the FS. If the application is concurrently readFile'ing and modifying the file, it will already see funny behavior. Though libuv uses preadv where available, this doesn't guarantee read atomicity in the presence of concurrent writes.
Related
It might be that writeFile has similar behavior. I didn't check.
PR
I have a patch that partitions readFile. I'm happy to submit a PR if there's interest.