preadv2 not found during linking

[NobodyXu]

Ref: rust-lang/cc-rs#1039

On older glibc version, the preadv2 symbol isn't available.

I think it might makes sense to use syscall directly to avoid this link-time issue.

[NobodyXu]

Switching to using syscall would also enable it to be used on musl.

[NobodyXu]

From preadv64v2.c and preadv2 from glibc:

ssize_t result = SYSCALL_CANCEL(preadv2, fd, vector, count,
				   LO_HI_LONG(offset), flags);

[NobodyXu]

For LO_HI_LONG:

// x86_64
#define LO_HI_LONG(val) (val), 0

// x32
#define LO_HI_LONG(val) (val)

// Other
/* Provide a macro to pass the off{64}_t argument on p{readv,writev}{64}.  */
#define LO_HI_LONG(val) \
 (long) (val), \
 (long) (((uint64_t) (val)) >> 32)

I scrapped it from https://elixir.bootlin.com/glibc/glibc-2.39/source/sysdeps/unix/sysv/linux/sysdep.h#L94 , https://elixir.bootlin.com/glibc/glibc-2.39/source/sysdeps/unix/sysv/linux/x86_64/sysdep.h#L386 , and https://elixir.bootlin.com/glibc/glibc-2.39/source/sysdeps/unix/sysv/linux/x86_64/x32/sysdep.h#L27

[weihanglo]

Thanks for the investigation.

Before implementing this, I would suggest we re-evaluate the value we gain from this Linux specific optimization. If it is pretty beneficial, then we can consider adding it, otherwise this might introduce a bit too many complexity and compatibility issues, and I'd rather remove preadv2.

[weihanglo]

By re-evaluations, I mean something like listing out in which situation preadv2 would help, and how common they are, as well as looking into the performance side like benchmarking, especially real-world benchmarks.

[ofek]

Can we please start off by reverting so that builds everywhere start passing once again?

[NobodyXu]

By re-evaluations, I mean something like listing out in which situation preadv2 would help, and how common they are,

preadv2 would help in situations where procfs isn't present and jobserver is given a pipe instead of fifo.

It's possible that some dev envs block access to procfs due to safety/sandboxing reason.

cc @the8472 since he might have more knowledge of this and how to handle it

Mote that preadv2 is currently disabled on musl due to musl not supporting it yet.

[weihanglo]

It's possible that some dev envs block access to procfs due to safety/sandboxing reason.

Is jobserver completely unusable without this optimization for this situation? That being said, if a simple syscall can resolve the issue without breaking old kernel/glibc, I am pretty okay with it. We already have similar work like:

jobserver-rs/src/unix.rs

Lines 70 to 72 in 7b96b41

	static PIPE2_AVAILABLE: AtomicBool = AtomicBool::new(true);
	if PIPE2_AVAILABLE.load(Ordering::SeqCst) {
	match libc::syscall(libc::SYS_pipe2, pipes.as_mut_ptr(), libc::O_CLOEXEC) {

[the8472]

preadv2 also helps in the sense that the FD does not have to be set to O_NONBLOCK which allows the acquire path continue to use blocking reads which are more efficient on pipes than the poll + non-blocking read dance.

That being said, if a simple syscall can resolve the issue without breaking old kernel/glibc, I am pretty okay with it. We already have similar work like:

jobserver-rs/src/unix.rs

Lines 70 to 72 in 7b96b41

	static PIPE2_AVAILABLE: AtomicBool = AtomicBool::new(true);
	if PIPE2_AVAILABLE.load(Ordering::SeqCst) {
	match libc::syscall(libc::SYS_pipe2, pipes.as_mut_ptr(), libc::O_CLOEXEC) {

Yes, the other calls should have used the same pattern.