This commit adds a `-Zmiri-check-number-initialization` flag to check
that integers and floats are initialized.
This commit also changes some shims to write at type `MaybeUninit<...>`
in order to prevent spurious errors from the uninit check.
add support for `#[global_allocator]`
This PR adds support for custom global allocators. Unfortunately, the code given in #1207 still causes errors when used with box. I believe this is because Box is special-cased in miri and stacked borrows.
Previously the command line string would have been incorrectly constructed if argv[0] contained a doublequote (`"`) or ended in a trailing backslash (`\`). This is a very rare edge case because, by convention, argv[0] is the path to the application and Windows file names cannot contain doublequotes.
Fixes#1881
add support for `#[start]`
This PR adds support for the `#[start]` attribute and fixes#1825.
It also renames `eval_main` to `eval_entry` to reflect that it can evaluate any entry function.
Add support for the `volatile_set_memory` intrinsic
Add support for the unstable `volatile_set_memory` intrinsic
(Runtime behaviour and soundness requirements for `volatile_set_memory` are identical to those of `write_bytes`, so supporting this intrinsic is trivial.)
also ignore 'thread leaks' with -Zmiri-ignore-leaks
This is a step towards https://github.com/rust-lang/miri/issues/1371. The remaining hard part would be supporting checking for memory leaks when there are threads still running. For now we elegantly avoid this problem by using the same flag to control both of these checks. :)
Change the code to either `EACCES` (if the op is performed on the
path), or `EBADF` (if the op is performed the fd)
Updated ops: `stat`, `opendir`, `ftruncate64`, and `readlink`
Add a new test for fs ops in isolation.
Update posix fs shims to use new API `reject_in_isolation`, which
allows rejection with error code instead of always forcing abort.
Error code chosen for each op is the most appropriate one from the
list in corresponding syscall's manual.
Updated helper APIs to not use quotes (`) around input name while
preparing the message. This allows callers to pass multi-word string
like -- "`read` from stdin".
Filter out items other than non-generic functions and statics in our version of `exported_symbols`
[`#[no_mangle]` on a `use` item](https://docs.rs/brotli-decompressor/2.3.1/src/brotli_decompressor/ffi/mod.rs.html#3-5) can make Miri ICE when compiling a dependency (rust-lang/rust#86261):
```rs
#[no_mangle]
use std::{thread,panic, io, boxed, any, string};
```
<details>
```
error: internal compiler error: compiler/rustc_middle/src/ty/mod.rs:1650:13: item_name: no name for DefPath { data: [DisambiguatedDefPathData { data: Misc, disambiguator: 14 }], krate: crate0 }
thread 'rustc' panicked at 'Box<dyn Any>', compiler/rustc_errors/src/lib.rs:1007:9
stack backtrace:
0: std::panicking::begin_panic
1: std::panic::panic_any
2: rustc_errors::HandlerInner::bug
3: rustc_errors::Handler::bug
4: rustc_middle::ty::context::tls::with_opt
5: rustc_middle::util:🐛:opt_span_bug_fmt
6: rustc_middle::util:🐛:bug_fmt
7: rustc_middle::ty::<impl rustc_middle::ty::context::TyCtxt>::item_name
8: rustc_symbol_mangling::symbol_name_provider
9: rustc_query_impl::<impl rustc_query_system::query::config::QueryAccessors<rustc_query_impl::plumbing::QueryCtxt> for rustc_query_impl::queries::symbol_name>::compute
10: rustc_query_system::query::plumbing::get_query_impl
11: <rustc_query_impl::Queries as rustc_middle::ty::query::QueryEngine>::symbol_name
12: rustc_middle::middle::exported_symbols::ExportedSymbol::symbol_name_for_local_instance
13: rustc_codegen_ssa:🔙:symbol_export::symbol_name_for_instance_in_crate
14: rustc_codegen_ssa:🔙:linker::exported_symbols
15: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::fold
16: rustc_codegen_ssa:🔙:linker::LinkerInfo::new
17: rustc_codegen_ssa:🔙:write::start_async_codegen
18: <rustc_codegen_llvm::LlvmCodegenBackend as rustc_codegen_ssa::traits::backend::CodegenBackend>::codegen_crate
19: rustc_interface::passes::QueryContext::enter
20: rustc_interface::queries::Queries::ongoing_codegen
21: rustc_interface::queries::<impl rustc_interface::interface::Compiler>::enter
22: rustc_span::with_source_map
23: rustc_interface::interface::create_compiler_and_run
24: rustc_span::with_session_globals
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
note: the compiler unexpectedly panicked. this is a bug.
note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md
note: rustc 1.54.0-nightly (a50d72158 2021-06-08) running on x86_64-unknown-linux-gnu
note: compiler flags: -C embed-bitcode=no -C debuginfo=1 --crate-type lib
note: some of the compiler flags provided by cargo are hidden
query stack during panic:
#0 [symbol_name] computing the symbol for `{misc#14}`
end of query stack
```
</details>
This might be because in #1776, we override the `exported_symbols` query, and our version of `exported_symbols` can return a `use` item which don't have a name if the `use` item is tagged with `#[no_mangle]`, and then:
- `rustc_codegen_ssa:🔙:symbol_export::symbol_name_for_instance_in_crate` is called for for every `exported_symbols`: fb3ea63d9b/compiler/rustc_codegen_ssa/src/back/linker.rs (L1300-L1304)
- it calls `rustc_middle::middle::exported_symbols::ExportedSymbol::symbol_name_for_local_instance`: fb3ea63d9b/compiler/rustc_codegen_ssa/src/back/symbol_export.rs (L412)
- which calls `rustc_symbol_mangling::symbol_name_provider`: fb3ea63d9b/compiler/rustc_middle/src/middle/exported_symbols.rs (L37-L44)
- which calls `item_name`: fb3ea63d9b/compiler/rustc_symbol_mangling/src/lib.rs (L216), which triggers the ICE
It might also be problematic for d39f0c64b8/src/shims/foreign_items.rs (L165) which also uses `item_name`, but Miri cannot compile the dependency, so that code can't be reached.
Therefore, this PR makes `exported_symbols` filter out all items that are not functions or statics, so all items returned will have a name, which avoids the ICE (I have tested it in the https://github.com/jorgecarleitao/arrow2 repository).
(This PR also includes a commit that fixes a small (unrelated) bug for `#[no_mangle]` on associated functions -- I found that because I notice `#[no_mangle]` is supported on associated functions and they should not be filtered out in `exported_symbols`.)
Fixes (when the submodule is bumped) rust-lang/rust#86261.
Report an error if a `#[no_mangle]`/`#[export_name = ...]` function has the same symbol name as a built-in shim
Implements https://github.com/rust-lang/miri/pull/1776#issuecomment-821322605.
The error looks like this:
```
error: found `malloc` symbol definition that clashes with a built-in shim
--> tests/compile-fail/function_calls/exported_symbol_shim_clashing.rs:12:9
|
12 | malloc(0);
| ^^^^^^^^^ found `malloc` symbol definition that clashes with a built-in shim
|
help: the `malloc` symbol is defined here
--> tests/compile-fail/function_calls/exported_symbol_shim_clashing.rs:2:1
|
2 | / extern "C" fn malloc(_: usize) -> *mut std::ffi::c_void {
3 | | //~^ HELP the `malloc` symbol is defined here
4 | | unreachable!()
5 | | }
| |_^
= note: inside `main` at tests/compile-fail/function_calls/exported_symbol_shim_clashing.rs:12:9
```
This does not implement "better error messages than we do currently for arg/ABI mismatches" in https://github.com/rust-lang/miri/pull/1776#issuecomment-821343175 -- I failed to remove all `check_arg_count()` and `check_abi()` (they are still used in `src/shims/intrinsics.rs` and `call_dlsym()`) and they don't receive the name of the shim.
Add support for panicking in the emulated application when unsupported functionality is encountered
This PR fixes#1807 and allows an optional flag to be specified to panic when an unsupported syscall is encountered. In essence, instead of bubbling up an error in the context of the Miri application Miri will panic within the context of the *emulated* application. This feature is desired to allow CI pipelines to determine if a Miri failure is unsupported functionality or actual UB. Please read [this comment](https://github.com/rust-lang/miri/issues/1807#issuecomment-845425076) for the rationale behind this change.
Note: this change does not cover all cases where unsupported functionality errors may be raised. If you search the repo for `throw_unsup_format!` there are many cases that I think are less likely to occur and may still be problematic for some folks.
TODO:
- [x] README documentation on this new flag
- [x] Add tests
In user interface, added a new flag `-Zmiri-isolation-error` which
takes one of the four values -- hide, warn, warn-nobacktrace, and
abort. This option can be used to configure Miri to either abort or
return an error code upon executing isolated op. If not aborted, Miri
prints a warning, whose verbosity can be configured using this flag.
In implementation, added a new enum `IsolatedOp` to capture all the
settings related to ops requiring communication with the
host. Old `communicate` flag in both miri configs and machine
stats is replaced with a new helper function `communicate()` which
checks `isolated_op` internally.
Added a new helper function `reject_in_isolation` which can be called
by shims to reject ops according to the reject_with settings. Use miri
specific diagnostics function `report_msg` to print backtrace in the
warning. Update it to take an enum value instead of a bool, indicating
the level of diagnostics.
Updated shims related to current dir to use the new APIs. Added a new
test for current dir ops in isolation without halting machine.
This PR uses the `measureme` crate to profile the call stack of the
program being interpreted by Miri. This is accomplished by starting a
measureme 'event' when we enter a function call, and ending the event
when we exit the call. The `measureme` tooling can be used to produce a
call stack from the generated profile data.
Limitations:
* We currently record every single entry/exit. This might generate very
large profile outputs for programs with a large number of function
calls. In follow-up work, we might want to explore sampling (e.g. only
recording every N function calls).
* This does not integrate very well with Miri's concurrency support.
Each event we record starts when we push a frame, and ends when we pop
a frame. As a result, switching between virtual threads will cause
events from different threads to be interleaved. Additionally, the
recorded for a particular frame will include all of the work Miri does
before that frame completes, including executing another thread.
The `measureme` integration is off by default, and must be enabled via
`-Zmiri-measureme=<output_name>`
Throw UB if f*_fast intrinsic called with non-finite value
Calling these intrinsics with non-finite values is undefined behaviour, since they result in `f*` intrinsics in LLVM with the `fast` flag, and `fast` math on non-finite values results in `poison` values. (technically LLVM only considers it UB upon _using_ the value, but that shouldn't make much of a difference)
only check timeouts when a thread yields
Currently, we check for expired timeouts after each step of execution. That seems excessive. This changes the scheduler to only check for timeouts when the active thread cannot continue running any more.
`@vakaras` does this sound right? `pthread_cond_timedwait` anyway already yields, of course, since it blocks on getting the signal (or the timeout).
Add atomic min and max
Closes#1718
Previous attempt: #1653
TODO:
- [x] Merge `atomic_op` and `atomic_min_max` functions
- [x] Fix CI
**Note:** this PR also removes arbitrary trailing whitespace and generally formats the affected files
Add simple data-race detector
Partially fixes data-race detection, see #1372, based on Dynamic Race Detection for C++11
- This does not explore weak memory behaviour, only exploring one sequentially consistent ordering.
- Data-race detection is only enabled after the first thread is created, so should have minimal overhead for non-concurrent execution.
- ~~Does not attempt to re-use thread id's so creating and joining threads lots of time in an execution will result in the vector clocks growing in size and slowing down program execution~~ It does now
std now looks up `getrandom` and `statx` with `dlsym` before attempting
to use `syscall(SYS_.., ..)`.
It also now passes all arguments as a machine-sized word, instead of
their original types.
Rustc removed the target wrapper and exposed the target directly.
Result of running:
find . -type f -exec sed -i -e 's/target\.target\([)\.,;]\)/target\1/g' {} \;
Plus one manual edit of the rust-version file
Add an `fn_ptr` field to `MiriFrame`
The `backtrace-rs` crate can use this to implement
`Frame::symbol_address`, which is used to skip frames
above the call to `Backtrace::capture` on the stack.
The function pointer will not be useable for comparison purposes if the
function is generic, as CTFE creates a new function pointer for each
cast of a (monomorphized) generic function. However, this already
affects code running under Miri, and isn't a problem for `backtrace-rs`
(which only casts a non-generic function).
I've added logic to allow `MiriFrame` to have either 4 or 5 fields - if
a 5th field is present, we write the function pointer to it.
The `backtrace-rs` crate can use this to implement
`Frame::symbol_address`, which is used to skip frames
above the call to `Backtrace::capture` on the stack.
The function pointer will not be useable for comparison purposes if the
function is generic, as CTFE creates a new function pointer for each
cast of a (monomorphized) generic function. However, this already
affects code running under Miri, and isn't a problem for `backtrace-rs`
(which only casts a non-generic function).
I've added logic to allow `MiriFrame` to have either 4 or 5 fields - if
a 5th field is present, we write the function pointer to it.
This PR adds two new Miri-defined extern functions:
`miri_get_backtrace` and `miri_resolve_frame`, which are documented in
the README. Together, they allow obtaining a backtrace for the currently
executing program.
I've added a test showing how these APIs are used. I've also prepared a
companion PR `backtrace-rs`, which will allow
`backtrace::Backtrace::new()` to work automatically under Miri.
Once these two PRs are merged, we will be able to print backtraces from
the normal Rust panic hook (since libstd is now using backtrace-rs).
A few notes:
* Resolving the backtrace frames is *very* slow - you can actually see
each line being printed out one at a time. Some local testing showed
that this is not (primrary) caused by resolving a `Span` - it seems
to be just Miri being slow.
* For the first time, we now interact directly with a user-defined
struct (instead of just executing the user-provided MIR that
manipulates the struct). To allow for future changes, I've added
a 'version' parameter (currently required to be 0). This should allow
us to change the `MiriFrame` struct should the need ever arise.
* I used the approach suggested by @oli-obk - a returned backtrace
pointer consists of a base function allocation, with the 'offset'
used to encode the `Span.lo`. This allows losslessly reconstructing
the location information in `miri_resolve_frame`.
* There are a few quirks on the `backtrace-rs` side:
* `backtrace-rs` calls `getcwd()` by default to try to simplify
the filename. This results in an isolation error by default,
which could be annoying when printing a backtrace from libstd.
* `backtrace-rs` tries to remove 'internal' frames (everything between
the call to `Backtrace::new()` and the internal API call made by
backtrace-rs) by comparing the returned frame pointer value to
a Rust function pointer. This doesn't work due to the way we
construct the frame pointers passed to the caller. We could
attempt to support this kind of comparison, or just add a
`#[cfg(miri)]` and ignore the frames ourselves.
Redo cargo-miri logic
This rewrite the cargo-miri logic for running the requested crate(s) following what we outlined in https://github.com/rust-lang/miri/issues/739: `cargo miri run/test $FLAGS` (almost) literally invokes `cargo run/test $FLAGS` but with some environment variables set so that we can control what happens:
* `RUSTC_WRAPPER` is set so that we get invoked instead of `rustc`. We use that power to mess with the flags being used for the build (things to be interpreted by Miri use a different sysroot), and when we are detecting a binary crate meant to be run by Miri, we grab the info we care about and put it into a JSON file for later use.
* `CARGO_TARGET_$TARGET_RUNNER` is set so what we get invoked when cargo wants to run a binary. At that point we take that JSON info from before and use it to invoke Miri.
Overall this works great! We get all sorts of cargo magic for free, and do not even need `cargo-metadata` any more. There's one annoying point though, which I have not managed to entirely work around yet: this means we are doing a full build, not just a check-build. Given that our sysroot is MIR-only, I was surprised that this even worked, but still -- this means we are doing more work than we should. So I also added some patching of arguments to make sure `--emit` does not contain `link`, and then more patching was required of the `--extern` flags for the binary because those referenced the `.rlib` files but now only `.rmeta` exists, and that is still not fully working because cargo seems to expect those `.rmeta` files and now triggers a rebuild each time as those files are still missing. My current plan is to make our wrapper create some empty dummy files with the right names, but the amount of hacks we are stacking on top of each other here is getting worrysome.^^ `@ehuss` your input would be welcome on this issue.
Due to re-using cargo more literally, this also changes flag parsing to match `cargo`. So `-Zmiri` flags now have to be passed via an environment variable (Cc https://github.com/rust-lang/miri/issues/1416).
This PR is not ready yet, but the parts that are there I think can be reviewed already. TODO:
* [x] [Fix Windows](https://github.com/rust-lang/miri/pull/1540#issuecomment-688733741).
* [x] Figure out how we can do check-only builds without the rebuild problem above. ~~I am also worried about cases where `build.rs` script might detect check-only builds and then do less work; I think some crates in rustc are doing that and if this is a thing in the wider ecosystem we need to find a way to support this as well.~~ (postponed that until we have a concrete example)
* [x] Currently cargo runs doctests as well, and they are not run in Miri. We should at least figure out a way to not run them at all (resolving https://github.com/rust-lang/miri/issues/584 is left for a future PR).
* [x] For some time, detect the old way of passing `-Zmiri` flags and warn that this will need updating. For some simple situations we can probably make it still support the old way, but I plan to remove those hacks after a bit. This is just to give people and me time to go around and send PRs to all projects that use Miri on CI, and update their use of the flags.
* [x] Add a test for stdin handling (https://github.com/rust-lang/miri/issues/1505). This should work now but we should be sure.
* [x] Add a test for cargo env vars (https://github.com/rust-lang/miri/issues/1515).
* [x] Check if https://github.com/rust-lang/miri/issues/1516 is resolved.
* [x] Check if https://github.com/rust-lang/miri/issues/1001 and https://github.com/rust-lang/miri/issues/1514 are resolved.
* [x] Check if https://github.com/rust-lang/miri/issues/1312 is resolved (not sure if it is wort adding a test).
* [x] Check if https://github.com/rust-lang/miri/issues/1512 is resolved (not sure if it is wort adding a test).
Fixes https://github.com/rust-lang/miri/issues/700.
Fixes https://github.com/rust-lang/miri/issues/739.
Fixes https://github.com/rust-lang/miri/issues/1001.
Fixes https://github.com/rust-lang/miri/issues/1312 (without a test, as we run without cargo's stdin/stdout wrapping now, as the test for stdin confirms).
Fixes https://github.com/rust-lang/miri/issues/1416.
Fixes https://github.com/rust-lang/miri/issues/1505.
Fixes https://github.com/rust-lang/miri/issues/1512 (without a test, as cargo now does all that handling anyway, which various other tests confirm).
Fixes https://github.com/rust-lang/miri/issues/1514.
Fixes https://github.com/rust-lang/miri/issues/1516.
Cc `@alecmocatta` who reported many of the bugs above; would be great if you could help with the tests e.g. by providing some small examples I could try.
r? `@oli-obk`
Support F_DUPFD on stdin/stdout/stderr
Enable `close`-ing stdin/stdout/stderr
For `dup`, check if FD is `File` first
If not, clone the appropriate standard IO stream
Merge POSIX `close` and `dup` tests into same module
Also, add assertion that `write` on a closed FD returns an error.
Add `dup` as FileDescriptor trait fn
Also:
- Fix `close` so it drops `self` instead of reference to it
- Remove FD clamping in insert_fd_with_min_fd, since FDs 0-2 can be
closed
Fix fs_libc tests
Make error message when closing stdin/out/err more specific
Return io::Result from `FileDescriptor::dup`
Change error message when closing stdin/out/err
Refactor `FileDescriptor::dup` impl for `FileHandle`
Remove empty line
