Skip to content

Optimize file splatting on Windows, Darwin and Haiku #24784

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

ilmaria
Copy link
Contributor

@ilmaria ilmaria commented Aug 10, 2025

Fixes #24783

@ilmaria ilmaria force-pushed the optimize-windows-file-splatting branch from 39cd230 to b2c861c Compare August 10, 2025 20:47
@ilmaria
Copy link
Contributor Author

ilmaria commented Aug 11, 2025

I don't know what is causing the test failures. Locally testing, this seemd to work on x86_64-windows and x86_64-linux.

@andrewrk
Copy link
Member

looks like something that was fixed by b9a6dae so you should be able to rebase to solve it

@ilmaria ilmaria force-pushed the optimize-windows-file-splatting branch 2 times, most recently from 7974103 to 72a1806 Compare August 11, 2025 14:45
@ilmaria ilmaria changed the title Optimize file splatting on Windows Optimize file splatting on Windows, Darwin and Haiku Aug 11, 2025
@ilmaria
Copy link
Contributor Author

ilmaria commented Aug 11, 2025

Darwin and Haiku also had the same issue as Windows because posix.pwritev actually uses non-vectored pwrite for these platforms.

@ilmaria ilmaria force-pushed the optimize-windows-file-splatting branch from 9968971 to 701bf82 Compare August 12, 2025 14:20
@ilmaria
Copy link
Contributor Author

ilmaria commented Aug 12, 2025

I now realized that testing this through File.Writer's public API is not actually sensible since we cannot guarantee that a single write call will write all given bytes. If we wanted to test this splat optimization, we would need to test that File.Writer internally avoids calling write with small buffers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

File.Writer splatByteAll with high splat value is slow on Windows
2 participants