Allocator confusion when a crate replaces the global allocator and also captures a backtrace

[saethlin]

dhat fails under Miri, but I suspect this is some kind of convoluted Miri bug/edge case. It looks to me like dhat is freeing an allocation produced by the interpreter itself inside Miri's backtrace shim, and since Miri and dhat are using different allocators, Miri is correctly detecting a bug in... itself???

cargo miri test tracking the right allocation reports this:

test main ... note: tracking was triggered
   --> /home/ben/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.63/src/backtrace/miri.rs:57:37
    |
57  |     let frame: MiriFrame = unsafe { miri_resolve_frame(ptr as *mut (), 0) };
    |                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ created allocation with id 96235
    |
    = note: inside `backtrace::backtrace::miri::resolve_addr` at /home/ben/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.63/src/backtrace/miri.rs:57:37
    = note: inside `backtrace::backtrace::miri::trace_unsynchronized::<&mut [closure@dhat::new_backtrace_inner::{closure#0}]>` at /home/ben/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.63/src/backtrace/miri.rs:67:21
    = note: inside `backtrace::backtrace::miri::trace::<&mut [closure@dhat::new_backtrace_inner::{closure#0}]>` at /home/ben/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.63/src/backtrace/miri.rs:51:14
    = note: inside `backtrace::backtrace::trace_unsynchronized::<[closure@dhat::new_backtrace_inner::{closure#0}]>` at /home/ben/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.63/src/backtrace/mod.rs:66:5
    = note: inside `backtrace::backtrace::trace::<[closure@dhat::new_backtrace_inner::{closure#0}]>` at /home/ben/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.63/src/backtrace/mod.rs:53:14
    = note: inside `dhat::new_backtrace_inner` at /tmp/dhat-0.3.0/src/lib.rs:1183:5
    = note: inside `dhat::Globals::new` at /tmp/dhat-0.3.0/src/lib.rs:508:23
    = note: inside `dhat::ProfilerBuilder::build` at /tmp/dhat-0.3.0/src/lib.rs:1130:41
note: inside `main` at tests/ad-hoc.rs:23:13
   --> tests/ad-hoc.rs:23:13
    |
23  | /             dhat::Profiler::builder()
24  | |                 .ad_hoc()
25  | |                 .trim_backtraces(Some(usize::MAX))
26  | |                 .eprint_json()
27  | |                 .build(),
    | |________________________^
note: inside closure at tests/ad-hoc.rs:15:1
   --> tests/ad-hoc.rs:15:1
    |
14  |   #[test]
    |   ------- in this procedural macro expansion
15  | / fn main() {
16  | |     use serde_json::Value::{self, *};
17  | |
18  | |     // Ignored because profiling hasn't started yet.
...   |
164 | |     let _profiler = dhat::Profiler::new_heap();
165 | | }
    | |_^
    = note: this note originates in the attribute macro `test` (in Nightly builds, run with -Z macro-backtrace for more info)

error: Undefined Behavior: deallocating alloc96235, which is Rust heap memory, using C heap deallocation operation
   --> /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/sys/unix/alloc.rs:42:9
    |
42  |         libc::free(ptr as *mut libc::c_void)
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ deallocating alloc96235, which is Rust heap memory, using C heap deallocation operation
    |
    = help: this indicates a bug in the program: it performed an invalid operation, and caused Undefined Behavior
    = help: see https://doc.rust-lang.org/nightly/reference/behavior-considered-undefined.html for further information
            
    = note: inside `std::sys::unix::alloc::<impl std::alloc::GlobalAlloc for std::alloc::System>::dealloc` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/sys/unix/alloc.rs:42:9
    = note: inside `<dhat::Alloc as std::alloc::GlobalAlloc>::dealloc` at /tmp/dhat-0.3.0/src/lib.rs:1282:13
note: inside `_::__rg_dealloc` at tests/ad-hoc.rs:2:15
   --> tests/ad-hoc.rs:2:15
    |
1   | #[global_allocator]
    | ------------------- in this procedural macro expansion
2   | static ALLOC: dhat::Alloc = dhat::Alloc;
    |               ^^^^^^^^^^^
    = note: inside `std::alloc::dealloc` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/alloc/src/alloc.rs:105:14
    = note: inside `<std::alloc::Global as std::alloc::Allocator>::deallocate` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/alloc/src/alloc.rs:242:22
    = note: inside `alloc::alloc::box_free::<[u8], std::alloc::Global>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/alloc/src/alloc.rs:340:9
    = note: inside `std::ptr::drop_in_place::<std::boxed::Box<[u8]>> - shim(Some(std::boxed::Box<[u8]>))` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:188:1
    = note: inside `std::ptr::drop_in_place::<backtrace::backtrace::miri::MiriFrame> - shim(Some(backtrace::backtrace::miri::MiriFrame))` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:188:1
    = note: inside `std::ptr::drop_in_place::<backtrace::backtrace::miri::Frame> - shim(Some(backtrace::backtrace::miri::Frame))` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:188:1
    = note: inside `std::ptr::drop_in_place::<backtrace::backtrace::Frame> - shim(Some(backtrace::backtrace::Frame))` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:188:1
    = note: inside `backtrace::backtrace::miri::trace_unsynchronized::<&mut [closure@dhat::new_backtrace_inner::{closure#0}]>` at /home/ben/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.63/src/backtrace/miri.rs:68:43
    = note: inside `backtrace::backtrace::miri::trace::<&mut [closure@dhat::new_backtrace_inner::{closure#0}]>` at /home/ben/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.63/src/backtrace/miri.rs:51:14
    = note: inside `backtrace::backtrace::trace_unsynchronized::<[closure@dhat::new_backtrace_inner::{closure#0}]>` at /home/ben/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.63/src/backtrace/mod.rs:66:5
    = note: inside `backtrace::backtrace::trace::<[closure@dhat::new_backtrace_inner::{closure#0}]>` at /home/ben/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.63/src/backtrace/mod.rs:53:14
    = note: inside `dhat::new_backtrace_inner` at /tmp/dhat-0.3.0/src/lib.rs:1183:5
    = note: inside `dhat::Globals::new` at /tmp/dhat-0.3.0/src/lib.rs:508:23
    = note: inside `dhat::ProfilerBuilder::build` at /tmp/dhat-0.3.0/src/lib.rs:1130:41
note: inside `main` at tests/ad-hoc.rs:23:13
   --> tests/ad-hoc.rs:23:13
    |
23  | /             dhat::Profiler::builder()
24  | |                 .ad_hoc()
25  | |                 .trim_backtraces(Some(usize::MAX))
26  | |                 .eprint_json()
27  | |                 .build(),
    | |________________________^
note: inside closure at tests/ad-hoc.rs:15:1
   --> tests/ad-hoc.rs:15:1
    |
14  |   #[test]
    |   ------- in this procedural macro expansion
15  | / fn main() {
16  | |     use serde_json::Value::{self, *};
17  | |
18  | |     // Ignored because profiling hasn't started yet.
...   |
164 | |     let _profiler = dhat::Profiler::new_heap();
165 | | }
    | |_^
    = note: this error originates in the attribute macro `global_allocator` (in Nightly builds, run with -Z macro-backtrace for more info)

note: some details are omitted, run with `MIRIFLAGS=-Zmiri-backtrace=full` for a verbose backtrace

error: aborting due to previous error

error: test failed, to rerun pass '--test ad-hoc'

[bjorn3]

Will be fixed by #1975

[RalfJung]

Next step is to update the backtrace crate to use the new APIs Miri provides now.

[RalfJung]

@drmeepster do you plan to continue pushing this through?

[beepster4096]

I already have the PR open, rust-lang/backtrace-rs#462

[RalfJung]

rust-lang/rust#97207 landed, so this should be fixed now. However, we still need a test. Could someone write one? :D