Make backtraces work with #[global_allocator]
Currently, backtraces break when the global allocator is overridden because the allocator will attempt to deallocate memory allocated directly by Miri.
~~This PR fixes that by using a new memory kind and providing a function to deallocate it. We can't call the custom allocator to allocate because it's not possible to call a function in the middle of a shim.~~
This PR fixes that by adding a new version of the backtrace API accessible by setting `flags` to 1. Existing code still functions.
backtrace-rs PR: rust-lang/backtrace-rs#462
Fixes https://github.com/rust-lang/miri/issues/1996
Add a lot more information to SB fatal errors
In fatal errors, this clarifies the difference between a tag not being present in the borrow stack at all, and the tag being present but granting SRO. It also introduces a little notation for memory ranges so we can mention to the user that the span may point to code that operates on multiple memory locations, but we are reporting an error at a particular offset.
This also gets rid of the unqualified phrase "the borrow stack" in errors, and clarifies that it is the borrow stack _for some location_.
The crate `pdqselect` v0.1.1:
Before:
```
2103 | unsafe { copy_nonoverlapping(src, dst, count) }
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ no item granting read access to tag <2357> at alloc1029 found in borrow stack.
```
After:
```
2103 | unsafe { copy_nonoverlapping(src, dst, count) }
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| attempting a read access using <2357> at alloc1029[0x0], but that tag does not exist in the borrow stack for this location
| this error occurs as part of an access at alloc1029[0x0..0x4]
```
And the crate `half` v1.8.2
Before:
```
131 | unsafe { &mut *ptr::slice_from_raw_parts_mut(data, len) }
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ trying to reborrow for Unique at alloc1051, but parent tag <2091> does not have an appropriate item in the borrow stack
```
After:
```
131 | unsafe { &mut *ptr::slice_from_raw_parts_mut(data, len) }
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| trying to reborrow <2091> for Unique permission at alloc1051[0x0], but that tag only grants SharedReadOnly permission for this location
| this error occurs as part of a reborrow at alloc1051[0x0..0x6]
```
This tries to clarify exactly why an access is not valid by printing
what memory range the access was over, which in combination with
tag-tracking may help a user figure out the source of the problem.
add write_int_fields to replace write_packed_immediates
This avoids having to explicitly list the types of all fields -- we derive them from the type of the struct instead.
Also add write_int_fields_named, to give the fields by name instead of ordered by index.
Allow varargs for libc::open when it is allowed by the second argument
This PR allows `libc::open` to be called using two or three arguments as defined in https://man7.org/linux/man-pages/man2/open.2.html
The presence of the third argument depends on the value of the second argument. If the second argument dictates that the third argument is *required* miri will emit an error if the argument is missing. If the second argument does *not* require a third argument, then the argument is ignored and passed as 0 internally (it would be ignored by libc anyway)
add flag to forward specific env vars (while isolation remains enabled)
The flag is called `-Zmiri-env-forward=<var>`, but I am open to bikeshedding. ;)
Prune backtraces similar to RUST_BACKTRACE=1 logic
This removes the majority of output from `cargo miri run` and `cargo miri test` in common usage.
~~I've copied the logic almost directly from `std`:
3b186511f6/library/std/src/sys_common/backtrace.rs (L76-L77~~)
~~It might be nice to have the "some details were omitted" note and a fallback to a setting where we print everything just in case this logic goes sideways, but~~
~~1. I'm not sure where to put the note~~
~~2. `MIRI_BACKTRACE`, `RUST_BACKTRACE`, and `RUSTC_CTFE_BACKTRACE` already do something else. Should we repurpose or add on to the semantics of `MIRI_BACKTRACE`?~~
---
Based on this tiny silly crate:
```rust
fn main() {
some_function();
}
fn some_function() {
unsafe {
let _x: &u8 = core::mem::transmute(1usize);
}
}
#[cfg(test)]
mod tests {
#[test]
fn it_works() {
unsafe {
let _x: &'static u8 = core::mem::transmute(1usize);
}
}
}
```
`cargo miri run`:
Before:
```
Finished dev [unoptimized + debuginfo] target(s) in 0.10s
Running `/home/ben/.cargo/bin/cargo-miri target/miri/x86_64-unknown-linux-gnu/debug/scratch`
error: Undefined Behavior: type validation failed: encountered a dangling reference (address 0x1 is unallocated)
--> src/main.rs:7:23
|
7 | let _x: &u8 = core::mem::transmute(1usize);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ type validation failed: encountered a dangling reference (address 0x1 is unallocated)
|
= help: this indicates a bug in the program: it performed an invalid operation, and caused Undefined Behavior
= help: see https://doc.rust-lang.org/nightly/reference/behavior-considered-undefined.html for further information
= note: inside `some_function` at src/main.rs:7:23
note: inside `main` at src/main.rs:2:5
--> src/main.rs:2:5
|
2 | some_function();
| ^^^^^^^^^^^^^^^
= note: inside `<fn() as std::ops::FnOnce<()>>::call_once - shim(fn())` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
= note: inside `std::sys_common::backtrace::__rust_begin_short_backtrace::<fn(), ()>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/sys_common/backtrace.rs:122:18
= note: inside closure at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/rt.rs:145:18
= note: inside `std::ops::function::impls::<impl std::ops::FnOnce<()> for &dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe>::call_once` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/ops/function.rs:259:13
= note: inside `std::panicking::r#try::do_call::<&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe, i32>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panicking.rs:492:40
= note: inside `std::panicking::r#try::<i32, &dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panicking.rs:456:19
= note: inside `std::panic::catch_unwind::<&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe, i32>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panic.rs:137:14
= note: inside closure at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/rt.rs:128:48
= note: inside `std::panicking::r#try::do_call::<[closure@std::rt::lang_start_internal::{closure#2}], isize>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panicking.rs:492:40
= note: inside `std::panicking::r#try::<isize, [closure@std::rt::lang_start_internal::{closure#2}]>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panicking.rs:456:19
= note: inside `std::panic::catch_unwind::<[closure@std::rt::lang_start_internal::{closure#2}], isize>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panic.rs:137:14
= note: inside `std::rt::lang_start_internal` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/rt.rs:128:20
= note: inside `std::rt::lang_start::<()>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/rt.rs:144:17
error: aborting due to previous error
```
After:
``` Finished dev [unoptimized + debuginfo] target(s) in 0.10s
Running `/home/ben/.cargo/bin/cargo-miri target/miri/x86_64-unknown-linux-gnu/debug/scratch`
error: Undefined Behavior: type validation failed: encountered a dangling reference (address 0x1 is unallocated)
--> src/main.rs:7:23
|
7 | let _x: &u8 = core::mem::transmute(1usize);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ type validation failed: encountered a dangling reference (address 0x1 is unallocated)
|
= help: this indicates a bug in the program: it performed an invalid operation, and caused Undefined Behavior
= help: see https://doc.rust-lang.org/nightly/reference/behavior-considered-undefined.html for further information
= note: inside `some_function` at src/main.rs:7:23
note: inside `main` at src/main.rs:2:5
--> src/main.rs:2:5
|
2 | some_function();
| ^^^^^^^^^^^^^^^
note: Some details are omitted, run with `MIRIFLAGS=-Zmiri-backtrace=full` for a verbose backtrace.
error: aborting due to previous error
```
`cargo miri test`
Before:
```
Finished test [unoptimized + debuginfo] target(s) in 0.00s
Running unittests (target/miri/x86_64-unknown-linux-gnu/debug/deps/scratch-9d7717efc37bb64c)
running 1 test
test tests::it_works ... error: Undefined Behavior: type validation failed: encountered a dangling reference (address 0x1 is unallocated)
--> src/main.rs:16:35
|
16 | let _x: &'static u8 = core::mem::transmute(1usize);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ type validation failed: encountered a dangling reference (address 0x1 is unallocated)
|
= help: this indicates a bug in the program: it performed an invalid operation, and caused Undefined Behavior
= help: see https://doc.rust-lang.org/nightly/reference/behavior-considered-undefined.html for further information
= note: inside `tests::it_works` at src/main.rs:16:35
note: inside closure at src/main.rs:14:5
--> src/main.rs:14:5
|
13 | #[test]
| ------- in this procedural macro expansion
14 | / fn it_works() {
15 | | unsafe {
16 | | let _x: &'static u8 = core::mem::transmute(1usize);
17 | | }
18 | | }
| |_____^
= note: inside `<[closure@src/main.rs:14:5: 18:6] as std::ops::FnOnce<()>>::call_once - shim` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
= note: inside `<fn() as std::ops::FnOnce<()>>::call_once - shim(fn())` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
= note: inside `tests::test::__rust_begin_short_backtrace::<fn()>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/test/src/lib.rs:575:5
= note: inside closure at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/test/src/lib.rs:566:30
= note: inside `<[closure@tests::test::run_test::{closure#1}] as std::ops::FnOnce<()>>::call_once - shim(vtable)` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
= note: inside `<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send> as std::ops::FnOnce<()>>::call_once` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/alloc/src/boxed.rs:1854:9
= note: inside `<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>> as std::ops::FnOnce<()>>::call_once` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/panic/unwind_safe.rs:271:9
= note: inside `std::panicking::r#try::do_call::<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>>, ()>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panicking.rs:492:40
= note: inside `std::panicking::r#try::<(), std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>>>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panicking.rs:456:19
= note: inside `std::panic::catch_unwind::<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>>, ()>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panic.rs:137:14
= note: inside `tests::test::run_test_in_process` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/test/src/lib.rs:598:18
= note: inside closure at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/test/src/lib.rs:492:39
= note: inside `tests::test::run_test::run_test_inner` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/test/src/lib.rs:530:13
= note: inside `tests::test::run_test` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/test/src/lib.rs:562:28
= note: inside `tests::test::run_tests::<[closure@tests::test::run_tests_console::{closure#2}]>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/test/src/lib.rs:305:17
= note: inside `tests::test::run_tests_console` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/test/src/console.rs:290:5
= note: inside `tests::test::test_main` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/test/src/lib.rs:116:15
= note: inside `tests::test::test_main_static` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/test/src/lib.rs:135:5
= note: inside `main`
= note: inside `<fn() as std::ops::FnOnce<()>>::call_once - shim(fn())` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
= note: inside `std::sys_common::backtrace::__rust_begin_short_backtrace::<fn(), ()>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/sys_common/backtrace.rs:122:18
= note: inside closure at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/rt.rs:145:18
= note: inside `std::ops::function::impls::<impl std::ops::FnOnce<()> for &dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe>::call_once` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/ops/function.rs:259:13
= note: inside `std::panicking::r#try::do_call::<&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe, i32>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panicking.rs:492:40
= note: inside `std::panicking::r#try::<i32, &dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panicking.rs:456:19
= note: inside `std::panic::catch_unwind::<&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe, i32>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panic.rs:137:14
= note: inside closure at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/rt.rs:128:48
= note: inside `std::panicking::r#try::do_call::<[closure@std::rt::lang_start_internal::{closure#2}], isize>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panicking.rs:492:40
= note: inside `std::panicking::r#try::<isize, [closure@std::rt::lang_start_internal::{closure#2}]>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panicking.rs:456:19
= note: inside `std::panic::catch_unwind::<[closure@std::rt::lang_start_internal::{closure#2}], isize>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/panic.rs:137:14
= note: inside `std::rt::lang_start_internal` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/rt.rs:128:20
= note: inside `std::rt::lang_start::<()>` at /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/std/src/rt.rs:144:17
= note: this error originates in the attribute macro `test` (in Nightly builds, run with -Z macro-backtrace for more info)
error: aborting due to previous error
error: test failed, to rerun pass '--bin scratch'
```
After:
```
Finished test [unoptimized + debuginfo] target(s) in 0.00s
Running unittests (target/miri/x86_64-unknown-linux-gnu/debug/deps/scratch-9d7717efc37bb64c)
running 1 test
test tests::it_works ... error: Undefined Behavior: type validation failed: encountered a dangling reference (address 0x1 is unallocated)
--> src/main.rs:16:35
|
16 | let _x: &'static u8 = core::mem::transmute(1usize);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ type validation failed: encountered a dangling reference (address 0x1 is unallocated)
|
= help: this indicates a bug in the program: it performed an invalid operation, and caused Undefined Behavior
= help: see https://doc.rust-lang.org/nightly/reference/behavior-considered-undefined.html for further information
= note: inside `tests::it_works` at src/main.rs:16:35
note: inside closure at src/main.rs:14:5
--> src/main.rs:14:5
|
13 | #[test]
| ------- in this procedural macro expansion
14 | / fn it_works() {
15 | | unsafe {
16 | | let _x: &'static u8 = core::mem::transmute(1usize);
17 | | }
18 | | }
| |_____^
= note: this error originates in the attribute macro `test` (in Nightly builds, run with -Z macro-backtrace for more info)
note: Some details are omitted, run with `MIRIFLAGS=-Zmiri-backtrace=full` for a verbose backtrace.
error: aborting due to previous error
error: test failed, to rerun pass '--bin scratch'
```
Previously, Miri would always print a backtrace including all frames
when encountering an error. This adds -Zmiri-backtrace which defaults
to 1, internally called BacktraceStyle::Short. By default, backtraces
are pruned to start at __rust_begin_short_backtrace, similar to std.
Then we also remove non-local frames from the bottom of the trace.
This cleans up the last one or two shims outside main or a test.
Users can opt out of pruning by setting -Zmiri-backtrace=full, and will
be automatically opted out if there are no local frames because that
means the reported error is likely in the Rust runtime, which this
pruning is crafted to remove.
