make Miri's scheduler proper round-robin
When thread N blocks or yields, we activate thread N+1 next, rather than always activating thread 0. This should guarantee that as long as all threads regularly yield, each thread eventually takes a step again.
Fixes the "multiple loops that yield playing ping-pong" part of https://github.com/rust-lang/miri/issues/1388.
`@cbeuw` I hope this doesn't screw up the scheduler-dependent tests you are adding in your PR.
ui_test tweaks
- support multiple filters
- make `./miri check` also cover ui_test
- Run opt-level=4 tests again, but only the "run" tests
r? `@oli-obk`
Save a created event for zero-size reborrows
Currently, we don't save a created event for zero-sized reborrows. Attempting to use something from a zero-sized reborrow is surprisingly common, for example on `minimal-lexical==0.2.1` we previously just emit this:
```
Undefined Behavior: attempting a write access using <187021> at alloc72933[0x0], but that tag does not exist in the borrow stack for this location
--> /root/rust/library/core/src/ptr/mod.rs:1287:9
|
1287 | copy_nonoverlapping(&src as *const T, dst, 1);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| attempting a write access using <187021> at alloc72933[0x0], but that tag does not exist in the borrow stack for this location
| this error occurs as part of an access at alloc72933[0x0..0x8]
|
= help: this indicates a potential bug in the program: it performed an invalid operation, but the rules it violated are still experimental
= help: see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md for further information
= note: inside `std::ptr::write::<u64>` at /root/rust/library/core/src/ptr/mod.rs:1287:9
note: inside `minimal_lexical::stackvec::StackVec::push_unchecked` at /root/build/src/stackvec.rs:82:13
--> /root/build/src/stackvec.rs:82:13
|
82 | ptr::write(self.as_mut_ptr().add(self.len()), value);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
... backtrace continues...
```
Which leaves us with the question "where did we make this pointer?" because for every other diagnostic you get a "was created by" note, so I suspect people might be tempted to think there is a Miri bug here. I certainly was.
---
This code duplication is so awful, I'm going to take a look at cleaning it up later. The fact that `ptr_get_alloc_id` can fail in this situation makes things annoying.
Add support for _COARSE clocks
Original idea does not work, so I'm just going to try expanding support to include the `_COARSE` clocks.
The original motivation for this PR is that the test suite for the crate [`minstant`](https://crates.io/crates/minstant) reports UB, because it tries to use a clock type Miri didn't support, but never checked for an error code and so just used the uninit `libc::timespec`. So, that's technically a bug in `minstant`, but outside of Miri you'd have to be using an incredibly old Linux to ever see an `EINVAL` so the more helpful thing for Miri to do is behave like a newer Linux.
So now we don't detect UB in `minstant`, but we have a test failure:
```
failures:
---- src/instant.rs - instant::Instant::as_unix_nanos (line 150) stdout ----
Test executable failed (exit status: 101).
stderr:
thread 'main' panicked at 'assertion failed: (instant.as_unix_nanos(&anchor) as i64 - expected as i64).abs() < 1_000_000', src/instant.rs:11:1
```
I'm having trouble getting my head around the code in `minstant` that's involved in this test, but as far as I can tell from the man pages, these `_COARSE` clocks meet the requirements.
Closes https://github.com/rust-lang/miri/issues/1983 at least as best as I can.
tweak new test suite output
- Make the entire "## Running ui tests ..." green, including the target.
- Fix double-space in `testname.rs .. ok`.
- Make the final summary a bit more like compiletest-rs, in particular the newlines around it
- Use the term "ignored" consistently, rather than "skipped"
r? `@oli-obk`
Add a command line flag to avoid printing to stdout and stderr
This is practical for tests that don't actually care about the output and thus don't want it intermingled with miri's warnings, errors or ICEs
fixes#2083
Replace unneeded use of `ref` in favor of "match ergonomics"
The signature of `check_shim` is very amenable to this.
```rust
fn check_shim<'a, const N: usize>(…) -> InterpResult<'tcx, &'a [OpTy<'tcx, Tag>; N]>
```
Instead of:
```rust
let &[ref ptr, ref flags] = this.check_shim(…)?;
```
we can write it just as:
```rust
let [ptr, flags] = this.check_shim(…)?;
```
add -Zmiri-strict-provenance
This implements [strict provenance](https://github.com/rust-lang/rust/issues/95228) in Miri. The only change is that casting an integer to a pointer does not even attempt to produce a good provenance for the given address; instead, it always uses the invalid provenance. This stricter than even `-Zmiri-tag-raw-pointers` in that it also rejects the following example (which does not even involve Stacked Borrows):
```rust
fn main() {
let x = 22;
let ptr = &x as *const _ as *const u8;
let roundtrip = ptr as usize as *const u8;
let _ = unsafe { roundtrip.offset(1) };
}
```
The new flag also implies `-Zmiri-tag-raw-pointers` since the only reason one would *not* want to tag raw pointers is to support ptr-int-ptr roundtrips.
Note that the flag does *not* check against ptr-to-int *transmutes*; that still requires `-Zmiri-check-number-validity`. You can also check for strict provenance *without* Stacked Borrows by adding `-Zmiri-disable-stacked-borrows`.
The new "Miri hard mode" flags for maximal checking are `-Zmiri-strict-provenance -Zmiri-check-number-validity`. (Add `-Zmiri-symbolic-alignment-check` if you feel extra spicy today.)
Make backtraces work with #[global_allocator]
Currently, backtraces break when the global allocator is overridden because the allocator will attempt to deallocate memory allocated directly by Miri.
~~This PR fixes that by using a new memory kind and providing a function to deallocate it. We can't call the custom allocator to allocate because it's not possible to call a function in the middle of a shim.~~
This PR fixes that by adding a new version of the backtrace API accessible by setting `flags` to 1. Existing code still functions.
backtrace-rs PR: rust-lang/backtrace-rs#462
Fixes https://github.com/rust-lang/miri/issues/1996
Add a lot more information to SB fatal errors
In fatal errors, this clarifies the difference between a tag not being present in the borrow stack at all, and the tag being present but granting SRO. It also introduces a little notation for memory ranges so we can mention to the user that the span may point to code that operates on multiple memory locations, but we are reporting an error at a particular offset.
This also gets rid of the unqualified phrase "the borrow stack" in errors, and clarifies that it is the borrow stack _for some location_.
The crate `pdqselect` v0.1.1:
Before:
```
2103 | unsafe { copy_nonoverlapping(src, dst, count) }
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ no item granting read access to tag <2357> at alloc1029 found in borrow stack.
```
After:
```
2103 | unsafe { copy_nonoverlapping(src, dst, count) }
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| attempting a read access using <2357> at alloc1029[0x0], but that tag does not exist in the borrow stack for this location
| this error occurs as part of an access at alloc1029[0x0..0x4]
```
And the crate `half` v1.8.2
Before:
```
131 | unsafe { &mut *ptr::slice_from_raw_parts_mut(data, len) }
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ trying to reborrow for Unique at alloc1051, but parent tag <2091> does not have an appropriate item in the borrow stack
```
After:
```
131 | unsafe { &mut *ptr::slice_from_raw_parts_mut(data, len) }
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| trying to reborrow <2091> for Unique permission at alloc1051[0x0], but that tag only grants SharedReadOnly permission for this location
| this error occurs as part of a reborrow at alloc1051[0x0..0x6]
```
This tries to clarify exactly why an access is not valid by printing
what memory range the access was over, which in combination with
tag-tracking may help a user figure out the source of the problem.
Allow varargs for libc::open when it is allowed by the second argument
This PR allows `libc::open` to be called using two or three arguments as defined in https://man7.org/linux/man-pages/man2/open.2.html
The presence of the third argument depends on the value of the second argument. If the second argument dictates that the third argument is *required* miri will emit an error if the argument is missing. If the second argument does *not* require a third argument, then the argument is ignored and passed as 0 internally (it would be ignored by libc anyway)
add flag to forward specific env vars (while isolation remains enabled)
The flag is called `-Zmiri-env-forward=<var>`, but I am open to bikeshedding. ;)
exclude mutable references to !Unpin types from uniqueness guarantees
This basically works around https://github.com/rust-lang/unsafe-code-guidelines/issues/148 by not requiring uniqueness any more for mutable references to self-referential generators. That corresponds to [the same work-around that was applied in rustc itself](b815532674/compiler/rustc_middle/src/ty/layout.rs (L2482)).
I am not entirely sure if this is a good idea since it might hide too many errors in case types are "accidentally" `!Unpin`. OTOH, our test suite still passes, and to my knowledge the vast majority of types is `Unpin`. (`place.layout.ty` is monomorphic, we should always exactly know which type this is.)
intptrcast: Never allocate two objects directly adjecent
When two objects directly follow each other in memory, what is the
provenance of an integer cast to a pointer that points directly between
them? For a zero-size region, it could point into the end of the first
object, or the start of the second.
We can avoid answering this difficult question by simply never
allocating two objects directly beside each other. This fixes some of
the false positives from #1866.
When two objects directly follow each other in memory, what is the
provenance of an integer cast to a pointer that points directly between
them? For a zero-size region, it could point into the end of the first
object, or the start of the second.
We can avoid answering this difficult question by simply never
allocating two objects directly beside each other. This fixes some of
the false positives from #1866.
also ignore 'thread leaks' with -Zmiri-ignore-leaks
This is a step towards https://github.com/rust-lang/miri/issues/1371. The remaining hard part would be supporting checking for memory leaks when there are threads still running. For now we elegantly avoid this problem by using the same flag to control both of these checks. :)
Change the code to either `EACCES` (if the op is performed on the
path), or `EBADF` (if the op is performed the fd)
Updated ops: `stat`, `opendir`, `ftruncate64`, and `readlink`
Add a new test for fs ops in isolation.
Report an error if a `#[no_mangle]`/`#[export_name = ...]` function has the same symbol name as a built-in shim
Implements https://github.com/rust-lang/miri/pull/1776#issuecomment-821322605.
The error looks like this:
```
error: found `malloc` symbol definition that clashes with a built-in shim
--> tests/compile-fail/function_calls/exported_symbol_shim_clashing.rs:12:9
|
12 | malloc(0);
| ^^^^^^^^^ found `malloc` symbol definition that clashes with a built-in shim
|
help: the `malloc` symbol is defined here
--> tests/compile-fail/function_calls/exported_symbol_shim_clashing.rs:2:1
|
2 | / extern "C" fn malloc(_: usize) -> *mut std::ffi::c_void {
3 | | //~^ HELP the `malloc` symbol is defined here
4 | | unreachable!()
5 | | }
| |_^
= note: inside `main` at tests/compile-fail/function_calls/exported_symbol_shim_clashing.rs:12:9
```
This does not implement "better error messages than we do currently for arg/ABI mismatches" in https://github.com/rust-lang/miri/pull/1776#issuecomment-821343175 -- I failed to remove all `check_arg_count()` and `check_abi()` (they are still used in `src/shims/intrinsics.rs` and `call_dlsym()`) and they don't receive the name of the shim.
Add support for panicking in the emulated application when unsupported functionality is encountered
This PR fixes#1807 and allows an optional flag to be specified to panic when an unsupported syscall is encountered. In essence, instead of bubbling up an error in the context of the Miri application Miri will panic within the context of the *emulated* application. This feature is desired to allow CI pipelines to determine if a Miri failure is unsupported functionality or actual UB. Please read [this comment](https://github.com/rust-lang/miri/issues/1807#issuecomment-845425076) for the rationale behind this change.
Note: this change does not cover all cases where unsupported functionality errors may be raised. If you search the repo for `throw_unsup_format!` there are many cases that I think are less likely to occur and may still be problematic for some folks.
TODO:
- [x] README documentation on this new flag
- [x] Add tests
In user interface, added a new flag `-Zmiri-isolation-error` which
takes one of the four values -- hide, warn, warn-nobacktrace, and
abort. This option can be used to configure Miri to either abort or
return an error code upon executing isolated op. If not aborted, Miri
prints a warning, whose verbosity can be configured using this flag.
In implementation, added a new enum `IsolatedOp` to capture all the
settings related to ops requiring communication with the
host. Old `communicate` flag in both miri configs and machine
stats is replaced with a new helper function `communicate()` which
checks `isolated_op` internally.
Added a new helper function `reject_in_isolation` which can be called
by shims to reject ops according to the reject_with settings. Use miri
specific diagnostics function `report_msg` to print backtrace in the
warning. Update it to take an enum value instead of a bool, indicating
the level of diagnostics.
Updated shims related to current dir to use the new APIs. Added a new
test for current dir ops in isolation without halting machine.
