Commit Graph

6136 Commits

Author SHA1 Message Date
Matthias Krüger
89da571b5d shims: fs: silence stderr instead of stdout.
Fixes #2143
2022-05-22 21:54:00 +02:00
Matthias Krüger
d7d7a9a162 [NFC] shims: fs: fmt 2022-05-22 21:47:44 +02:00
bors
e3227cfc7e Auto merge of #2141 - saethlin:early-diagnostics-ice, r=RalfJung
Adjust diagnostics assertion so we don't ICE in setup

Fixes https://github.com/rust-lang/miri/issues/2076 just by handling diagnostics produced during setup. The tracking notes don't have any spans but it's better than an ICE.

It looks like we leak allocations 1..20, and allocations 13..19 don't have any creation notes, and 14 only has a `FreedAlloc` alloc tracking diagnostic.
2022-05-22 16:17:17 +00:00
Ralf Jung
73534a678d
tweak comment 2022-05-22 18:16:59 +02:00
Ben Kimock
486a769935 Handle diagnotics emitted in runtime initialization 2022-05-22 11:28:40 -04:00
bors
d58abab2e7 Auto merge of #2142 - saethlin:cleanup-data-race-ice, r=oli-obk
Make allow_data_races_* public and use it during EnvVars::cleanup

Fixes https://github.com/rust-lang/miri/issues/2020

I've tried for hours now to come up with a test case for this ICE with no luck. I suspect there's something about the way the data race detection works under these conditions that I just don't understand 😩.

But I tried this change out on a handful of crates and I don't see any more ICEs of this form. For whatever reason it seems like `bastion==0.4.5` is a good way to run into this, with the flags
```
MIRIFLAGS="-Zmiri-tag-raw-pointers -Zmiri-panic-on-unsupported -Zmiri-disable-isolation" cargo +miri miri test --no-fail-fast --doc
```
I think all the cases I've run into with this involve both `-Zmiri-panic-on-unsupported` and `-Zmiri-tag-raw-pointers`, so it could be that the combination of an unexpected panic and a machine halt is required.
2022-05-22 06:18:29 +00:00
bors
b5fc544ae8 Auto merge of #2140 - V0ldek:page_size, r=RalfJung
Update GetSystemInfo to work with `page_size` (#2136)

- Change logic in GetSystemInfo shim to take into account the two possible layouts of `SYSTEM_INFO`, the first-field-is-union used by [winapi::um::sysinfoapi::SYSTEM_INFO](https://docs.rs/winapi/latest/winapi/um/sysinfoapi/struct.SYSTEM_INFO.html), and first-two-fields-are-inlined-union used by [num_cpus](5f1b033320/src/lib.rs (L206)).
- Fill out the `dwPageSize` field with the `PAGE_SIZE` constant of `4096`.

Closes #2136
2022-05-22 05:59:39 +00:00
Ralf Jung
9a5c9a5481
comment on test 2022-05-22 07:59:18 +02:00
Mateusz Gienieczko
bd731508d4
Use precomputed layouts. 2022-05-22 01:01:12 +02:00
Mateusz Gienieczko
a40ff562a0
Add i16 and u16 primitive layout. 2022-05-22 01:00:59 +02:00
Mateusz Gienieczko
b7d032c219
Fix comment formatting. 2022-05-22 00:59:49 +02:00
Mateusz Gienieczko
2fa53c0385
Dynamic offset calculation in GetSystemInfo. 2022-05-21 21:42:25 +02:00
Ben Kimock
3cfce6ffb2 Make allow_data_races_* public to silence data races during cleanup 2022-05-21 15:16:15 -04:00
Mateusz Gienieczko
63e98aee0c
Change GetSystemInfo to explicit offset. 2022-05-21 19:44:12 +02:00
Mateusz Gienieczko
c4ee368acb
Set page size in GetSystemInfo. 2022-05-21 18:17:25 +02:00
bors
38111b3916 Auto merge of #2137 - RalfJung:targets, r=oli-obk
explain which targets we support to what extent

This is basically documenting my policy so far: Linux is the target I know best and can spend most time on. Apple is so close to Linux that it can basically ride along without much extra work. I don't have a lot of time to spend on our Windows support.

Of course, if people commit to contributing and maintaining support for a target, we can promise more than what is documented here. :)  But this is what I am willing to promise.

Also reduce the amount of work we do on the Windows test runner, since that one currently takes 10min longer than the other two.

r? `@oli-obk`
2022-05-21 16:15:49 +00:00
Mateusz Gienieczko
e932ea50ba
Add failing page_size test. 2022-05-21 16:10:08 +02:00
Ralf Jung
b4089a77ab refresh our GHA caches 2022-05-21 09:34:29 +02:00
Ralf Jung
aadbe8fd45 explain which targets we support to what extent 2022-05-20 23:41:55 +02:00
Ralf Jung
58fdd55a80 the Windows CI runner takes a lot longer than the others; let it do less work
x86_64-apple-darwin is also used as the host OS for the macOS runner, so no need
to test it twice.
2022-05-20 23:41:55 +02:00
Ralf Jung
3b73eb4456 explain what we mean by 'unsound' 2022-05-20 23:41:55 +02:00
bors
b96610b3de Auto merge of #2135 - RalfJung:rustup, r=RalfJung
rustup
2022-05-20 16:38:24 +00:00
Ralf Jung
8b4d613cc8 rustup 2022-05-20 18:37:57 +02:00
bors
d419efdba8 Auto merge of #2130 - RalfJung:rustup, r=RalfJung
rustup

`Display` of `Ref`/`RefMut` was broken by https://github.com/rust-lang/rust/pull/97027, let's deref them to use the underlying reference `Display`.

Cc https://github.com/rust-lang/rust/issues/97204
2022-05-20 06:09:13 +00:00
Ralf Jung
a941af8161 rustup 2022-05-20 08:08:11 +02:00
bors
9230b92cf2 Auto merge of #2128 - saethlin:fix-creation-range, r=RalfJung
Pass the correct size to the AllocRange for log_creation

Fixes https://github.com/rust-lang/miri/issues/2127

I guess all I needed was a bit of sleep and reassurance that this diagnostic is the wrong part of that situation.
2022-05-19 15:08:22 +00:00
Ben Kimock
ada864f387 Pass the correct size to the AllocRange for log_creation 2022-05-19 09:29:08 -04:00
bors
c5f1cdb323 Auto merge of #2125 - RalfJung:ref-validity, r=RalfJung
test for validity of references pointing to uninhabited types

The new tests for https://github.com/rust-lang/rust/pull/97116
2022-05-18 06:34:48 +00:00
Ralf Jung
439f861101 rustup 2022-05-18 08:33:10 +02:00
Ralf Jung
092c2b9d92 change one of the ref-to-uninhbaited tests to Box 2022-05-18 08:32:38 +02:00
Ralf Jung
30548bb57e test for validity of references pointing to uninhabited types 2022-05-18 08:32:38 +02:00
bors
a5cf247469 Auto merge of #2126 - RalfJung:typo, r=RalfJung
I cannot type
2022-05-17 16:59:56 +00:00
Ralf Jung
6b6f92d4bc I cannot type 2022-05-17 18:59:27 +02:00
bors
3971d3cbb3 Auto merge of #2124 - RalfJung:rustup-toolchain, r=RalfJung
rustup-toolchain: also prepare toolchain for vscode

This is a work-around for https://github.com/rust-lang/cargo/issues/10096.
2022-05-17 13:40:16 +00:00
Ralf Jung
3406829bb3 rustup-toolchain: also prepare toolchain for vscode 2022-05-17 15:36:59 +02:00
bors
19ef76477c Auto merge of #2122 - RalfJung:rustup, r=RalfJung
rustup

I am on mobile and so I couldn't test this... let's see how it goes. ;)
2022-05-15 10:58:41 +00:00
Ralf Jung
ea63a695c8 rustup 2022-05-15 12:58:28 +02:00
bors
8f7c8f7808 Auto merge of #2121 - RalfJung:less-ice, r=RalfJung
don't ICE when libcore is missing

Fixes https://github.com/rust-lang/miri/issues/2120
2022-05-15 08:27:43 +00:00
Ralf Jung
90a190e03b don't ICE when libcore is missing 2022-05-15 10:26:47 +02:00
bors
90d28eada1 Auto merge of #2116 - carbotaniuman:minimal-miri-changes, r=RalfJung
Minimal miri changes for rustc permissive provenance

Simple no-op changes to adapt to new API surface.
2022-05-15 07:11:51 +00:00
bors
98c8c8f9b5 Auto merge of #2030 - saethlin:track-alloc-history, r=oli-obk
Print spans where tags are created and invalidated

5225225 called this "automatic tag tracking" and I think that may be a reasonable description, but I would like to kill tag tracking as a primary use of Miri if possible. Tag tracking isn't always possible; for example if the UB is only detected with isolation off and the failing tag is made unstable by removing isolation. (also it's bad UX to run the tool twice)

This is just one of the things we can do with https://github.com/rust-lang/miri/pull/2024

The memory usage of this is _shockingly_ low, I think because the memory usage of Miri is driven by allocations where each byte ends up with its own very large stack. The memory usage in this change is linear with the number of tags, not tags * bytes. If memory usage gets out of control we can cap the number of events we save per allocation, from experience we tend to only use the most recent few in diagnostics but of course there's no guarantee of that so if we can manage to keep everything that would be best.

In many cases now I can tell exactly what these codebases are doing wrong just from the new outputs here, which I think is extremely cool.

New helps generated with plain old `cargo miri test` on `rust-argon2` v1.0.0:
```
test argon2::tests::single_thread_verification_multi_lane_hash ... error: Undefined Behavior: trying to reborrow <1485898> for Unique permission at alloc110523[0x0], but that tag does not exist in the borrow stack for this location
   --> /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/mem/manually_drop.rs:89:9
    |
89  |         slot.value
    |         ^^^^^^^^^^
    |         |
    |         trying to reborrow <1485898> for Unique permission at alloc110523[0x0], but that tag does not exist in the borrow stack for this location
    |         this error occurs as part of a reborrow at alloc110523[0x0..0x20]
    |
    = help: this indicates a potential bug in the program: it performed an invalid operation, but the rules it violated are still experimental
    = help: see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md for further information
help: <1485898> was created by a retag at offsets [0x0..0x20]
   --> src/memory.rs:42:13
    |
42  |             vec.push(unsafe { &mut (*ptr) });
    |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
help: <1485898> was later invalidated at offsets [0x0..0x20]
   --> src/memory.rs:42:31
    |
42  |             vec.push(unsafe { &mut (*ptr) });
    |                               ^^^^^^^^^^^
```

And with `-Zmiri-tag-raw-pointers` on `slab` v0.4.5
```
error: Undefined Behavior: trying to reborrow <2915> for Unique permission at alloc1418[0x0], but that tag does not exist in the borrow stack for this location
   --> /tmp/slab-0.4.5/src/lib.rs:835:16
    |
835 |         match (&mut *ptr1, &mut *ptr2) {
    |                ^^^^^^^^^^
    |                |
    |                trying to reborrow <2915> for Unique permission at alloc1418[0x0], but that tag does not exist in the borrow stack for this location
    |                this error occurs as part of a reborrow at alloc1418[0x0..0x10]
    |
    = help: this indicates a potential bug in the program: it performed an invalid operation, but the rules it violated are still experimental
    = help: see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md for further information
help: <2915> was created by a retag at offsets [0x0..0x10]
   --> /tmp/slab-0.4.5/src/lib.rs:833:20
    |
833 |         let ptr1 = self.entries.get_unchecked_mut(key1) as *mut Entry<T>;
    |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
help: <2915> was later invalidated at offsets [0x0..0x20]
   --> /tmp/slab-0.4.5/src/lib.rs:834:20
    |
834 |         let ptr2 = self.entries.get_unchecked_mut(key2) as *mut Entry<T>;
    |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
```

And without raw pointer tagging, `cargo miri test` on `half` v1.8.2
```
error: Undefined Behavior: trying to reborrow <untagged> for Unique permission at alloc1340[0x0], but that tag only grants SharedReadOnly permission for this location
   --> /home/ben/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/slice/raw.rs:141:9
    |
141 |         &mut *ptr::slice_from_raw_parts_mut(data, len)
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |         |
    |         trying to reborrow <untagged> for Unique permission at alloc1340[0x0], but that tag only grants SharedReadOnly permission for this location
    |         this error occurs as part of a reborrow at alloc1340[0x0..0x6]
    |
    = help: this indicates a potential bug in the program: it performed an invalid operation, but the rules it violated are still experimental
    = help: see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md for further information
help: tag was most recently created at offsets [0x0..0x6]
   --> /tmp/half-1.8.2/src/slice.rs:309:22
    |
309 |         let length = self.len();
    |                      ^^^^^^^^^^
help: this tag was also created here at offsets [0x0..0x6]
   --> /tmp/half-1.8.2/src/slice.rs:308:23
    |
308 |         let pointer = self.as_ptr() as *mut u16;
    |                       ^^^^^^^^^^^^^
```
The second suggestion is close to guesswork, but from experience it tends to be correct (as in, it tends to locate the pointer the user wanted) more often that it doesn't.
2022-05-14 19:27:04 +00:00
carbotaniuman
f8478df6dc Bump rustc for permissive provenance 2022-05-14 13:03:47 -05:00
Ben Kimock
8ff0aac06c More review feedback
* Store the local crates in an Rc<[CrateNum]>
* Move all the allocation history into Stacks
* Clean up the implementation of get_logs_relevant_to a bit
2022-05-13 19:04:51 -04:00
bors
d76c2c5e1c Auto merge of #2119 - RalfJung:glob, r=RalfJung
data_race: use glob import like most files
2022-05-13 21:19:59 +00:00
Ralf Jung
fde022007b data_race: use glob import like most files 2022-05-13 23:19:17 +02:00
bors
3f111c166a Auto merge of #2114 - cbeuw:shim-rmw, r=RalfJung
Use atomic RMW for `{mutex, rwlock, cond, srwlock}_get_or_create_id` functions

This is required for #1963

`{mutex, rwlock, cond, srwlock}_get_or_create_id()` currently checks whether an ID field is 0 using an atomic read, allocate one and get a new ID if it is, then write it in a separate atomic write. This is fine without weak memory. For instance, in `pthread_mutex_lock` which may be called by two threads concurrently, only one thread can read 0, create and then write a new ID, the later-run thread will always see the newly created ID and never 0.
```rust
    fn pthread_mutex_lock(&mut self, mutex_op: &OpTy<'tcx, Tag>) -> InterpResult<'tcx, i32> {
        let this = self.eval_context_mut();

        let kind = mutex_get_kind(this, mutex_op)?.check_init()?;
        let id = mutex_get_or_create_id(this, mutex_op)?;
        let active_thread = this.get_active_thread();
```

However, with weak memory behaviour, both threads may read 0: the first thread has to see 0 because nothing else was written to it, and the second thread is not guaranteed to observe the latest value, causing a duplicate mutex to be created and both threads "successfully" acquiring the lock at the same time.

This is a pretty typical pattern requiring the use of atomic RMWs. RMW *always* reads the latest value in a location, so only one thread can create the new mutex and ID, all others scheduled later will see the new ID.
2022-05-13 19:04:59 +00:00
Andy Wang
9e38dc4d49
Move and rename offset_and_layout_to_place to deref_operand_and_offset 2022-05-13 18:42:53 +01:00
Andy Wang
10d978c180
Inline _create() calls and add assertions 2022-05-12 21:06:17 +01:00
Andy Wang
a5db2c32e5
Refactor to hide *_next_id functions 2022-05-12 20:31:40 +01:00
bors
d33e7fc31c Auto merge of #2117 - RalfJung:rustup, r=RalfJung
rustup

Cc https://github.com/rust-lang/rust/issues/96975
2022-05-12 17:02:04 +00:00