Skip to content

Conversation

RalfJung
Copy link
Member

@RalfJung RalfJung commented Mar 4, 2021

@thomcc pointed out that the current docs leave it kind of unclear how one can satisfy the "no wrapping around isize or the address space" requirement of offset_from, so make the docs clearer about that.

FWIW, I don't think I entirely agree with that second paragraph about large objects (that I left mostly unchanged here). LLVM, to my knowledge, fundamentally assumes that all allocations fit into an isize::MAX. So in that sense creating a larger allocation is simply UB. I would expect a guarantee that Rust heap allocation methods will never return allocations larger than isize::MAX (or rather, Rust heap allocation methods should require that the Layout is no larger than isize::MAX). However, I cannot find any such requirement documented currently. Large allocations are not mentioned at all in the allocator docs, which is quite surprising -- even if we say that such allocations are not insta-UB (which I think is incompatible with LLVM), they are still extremely footgunny since ptr::offset/ptr::add do not support offsetting by more than isize::MAX bytes.

Furthermore, the allocator docs don't even say anything about allocations wrapping around the address space. But that is certainly something allocators must ensure never happens; we cannot expect clients to defend against this.

Cc @rust-lang/wg-allocators

@rust-highfive
Copy link
Contributor

r? @dtolnay

(rust-highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Mar 4, 2021
Copy link
Member

@dtolnay dtolnay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with that fixed.

@RalfJung
Copy link
Member Author

RalfJung commented Mar 4, 2021

Cc @rust-lang/lang I think this is already an established guarantee that "two pointers are within some allocated object of Rust type T" implies that they are no more than isize::MAX bytes apart and do not wrap around the address space (at least there's plenty of code out there relying on this) -- but pinging you nevertheless to make sure there's no objections here.

/// For instance, no known 64-bit platform can ever serve a request
/// for 2<sup>63</sup> bytes due to page-table limitations or splitting the address space.
/// However, some 32-bit and 16-bit platforms may successfully serve a request for
/// more than `isize::MAX` bytes with things like Physical Address
/// Extension. As such, memory acquired directly from allocators or memory
/// mapped files *may* be too large to handle with this function.
/// (Note that [`offset`] and [`add`] also have a similar limitation and hence cannot be used on
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the link for offset defined somewhere?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it already exists and is further up.

@dtolnay
Copy link
Member

dtolnay commented Mar 6, 2021

@bors r+ rollup

@bors
Copy link
Collaborator

bors commented Mar 6, 2021

📌 Commit ebe0407 has been approved by dtolnay

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 6, 2021
bors added a commit to rust-lang-ci/rust that referenced this pull request Mar 7, 2021
Rollup of 13 pull requests

Successful merges:

 - rust-lang#77916 (Change built-in kernel targets to be os = none throughout)
 - rust-lang#82130 (Make some Option, Result methods unstably const)
 - rust-lang#82292 (Prevent specialized ZipImpl from calling `__iterator_get_unchecked` twice with the same index)
 - rust-lang#82402 (Remove RefCell around `module_trait_cache`)
 - rust-lang#82592 (Improve transmute docs with further clarifications)
 - rust-lang#82651 (Cleanup rustdoc warnings)
 - rust-lang#82720 (Fix diagnostic suggests adding type `[type error]`)
 - rust-lang#82751 (improve offset_from docs)
 - rust-lang#82793 (Move some tests to more suitable subdirs)
 - rust-lang#82803 (rustdoc: Add an unstable option to print all unversioned files)
 - rust-lang#82808 (Sync rustc_codegen_cranelift)
 - rust-lang#82822 (Fix typo)
 - rust-lang#82837 (tweak MaybeUninit docs)

Failed merges:

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit 05a2366 into rust-lang:master Mar 7, 2021
@rustbot rustbot added this to the 1.52.0 milestone Mar 7, 2021
@RalfJung RalfJung deleted the offset_from branch March 7, 2021 10:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants