Commit Graph

6975 Commits

Author SHA1 Message Date
Christian Poveda
6a37643265
make sleep work with isolation enabled 2022-09-13 15:16:40 -05:00
bors
7e66a9ff16 Auto merge of #2543 - saethlin:be-slow-on-linux, r=oli-obk
Run the GC more often on Linux, not MacOS

Linux has more testing and is also faster in CI, we should do the extra slow checks there.
2022-09-13 16:14:27 +00:00
bors
c7f7bc0ca2 Auto merge of #2541 - lengyijun:patch-1, r=RalfJung
fix typo
2022-09-13 15:43:32 +00:00
bors
aebfbf2a91 fix typo 2022-09-13 15:43:32 +00:00
Ben Kimock
93e41f625a Linux has more testing, we should do the extra checks there 2022-09-13 10:12:57 -04:00
bors
a00fa96336 Auto merge of #2479 - saethlin:tag-gc, r=oli-obk
Implement a garbage collector for tags

The general approach here is to scan TLS, all locals, and the main memory map for all provenance, accumulating a `HashSet` of all pointer tags which are stored anywhere (we also have a special case for panic payloads). Then we iterate over every borrow stack and remove tags which are not in said `HashSet`, or which could be terminating a SRW block.

Runtime of benchmarks decreases by between 17% and 81%.

GC off:
```
Benchmark 1: cargo +miri miri run --manifest-path /home/ben/miri/bench-cargo-miri/backtraces/Cargo.toml
  Time (mean ± σ):      7.080 s ±  0.249 s    [User: 6.870 s, System: 0.202 s]
  Range (min … max):    6.933 s …  7.521 s    5 runs

  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet PC without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.

Benchmark 1: cargo +miri miri run --manifest-path /home/ben/miri/bench-cargo-miri/mse/Cargo.toml
  Time (mean ± σ):      1.875 s ±  0.031 s    [User: 1.630 s, System: 0.245 s]
  Range (min … max):    1.825 s …  1.910 s    5 runs

Benchmark 1: cargo +miri miri run --manifest-path /home/ben/miri/bench-cargo-miri/serde1/Cargo.toml
  Time (mean ± σ):      2.785 s ±  0.075 s    [User: 2.536 s, System: 0.168 s]
  Range (min … max):    2.698 s …  2.851 s    5 runs

Benchmark 1: cargo +miri miri run --manifest-path /home/ben/miri/bench-cargo-miri/serde2/Cargo.toml
  Time (mean ± σ):      6.267 s ±  0.066 s    [User: 6.072 s, System: 0.190 s]
  Range (min … max):    6.152 s …  6.314 s    5 runs

Benchmark 1: cargo +miri miri run --manifest-path /home/ben/miri/bench-cargo-miri/slice-get-unchecked/Cargo.toml
  Time (mean ± σ):      4.733 s ±  0.080 s    [User: 4.177 s, System: 0.513 s]
  Range (min … max):    4.681 s …  4.874 s    5 runs

Benchmark 1: cargo +miri miri run --manifest-path /home/ben/miri/bench-cargo-miri/unicode/Cargo.toml
  Time (mean ± σ):      3.770 s ±  0.034 s    [User: 3.549 s, System: 0.211 s]
  Range (min … max):    3.724 s …  3.819 s    5 runs
```
GC on:
```
Benchmark 1: cargo +miri miri run --manifest-path /home/ben/miri/bench-cargo-miri/backtraces/Cargo.toml
  Time (mean ± σ):      5.886 s ±  0.054 s    [User: 5.696 s, System: 0.182 s]
  Range (min … max):    5.799 s …  5.937 s    5 runs

Benchmark 1: cargo +miri miri run --manifest-path /home/ben/miri/bench-cargo-miri/mse/Cargo.toml
  Time (mean ± σ):     936.4 ms ±   7.0 ms    [User: 815.4 ms, System: 119.6 ms]
  Range (min … max):   925.7 ms … 945.0 ms    5 runs

Benchmark 1: cargo +miri miri run --manifest-path /home/ben/miri/bench-cargo-miri/serde1/Cargo.toml
  Time (mean ± σ):      2.126 s ±  0.022 s    [User: 1.979 s, System: 0.146 s]
  Range (min … max):    2.089 s …  2.143 s    5 runs

Benchmark 1: cargo +miri miri run --manifest-path /home/ben/miri/bench-cargo-miri/serde2/Cargo.toml
  Time (mean ± σ):      4.242 s ±  0.066 s    [User: 4.051 s, System: 0.160 s]
  Range (min … max):    4.196 s …  4.357 s    5 runs

Benchmark 1: cargo +miri miri run --manifest-path /home/ben/miri/bench-cargo-miri/slice-get-unchecked/Cargo.toml
  Time (mean ± σ):     907.4 ms ±   2.4 ms    [User: 788.6 ms, System: 118.2 ms]
  Range (min … max):   903.5 ms … 909.4 ms    5 runs

Benchmark 1: cargo +miri miri run --manifest-path /home/ben/miri/bench-cargo-miri/unicode/Cargo.toml
  Time (mean ± σ):      1.821 s ±  0.011 s    [User: 1.687 s, System: 0.133 s]
  Range (min … max):    1.802 s …  1.831 s    5 runs
```

But much more importantly for me this drops the peak memory usage of the first 1 minute of running `regex`'s tests from 103  GB to 1.7 GB.

Thanks to `@oli-obk` for suggesting a while ago that this was possible and `@darksonn` for reminding me that we can just search through memory to find Provenance to locate pointers.

Fixes https://github.com/rust-lang/miri/issues/1367
2022-09-13 11:05:55 +00:00
bors
3886a63ea4 Auto merge of #2542 - lengyijun:patch-2, r=oli-obk
fix typo
2022-09-13 09:53:04 +00:00
lyj
f6cbba12ce
fix typo 2022-09-13 17:49:41 +08:00
Ben Kimock
f59605ce52 In CI set the GC interval to 1 for Linux only 2022-09-10 23:05:56 -04:00
Ben Kimock
d61d4c6af7 Implement -Zmiri-tag-gc a garbage collector for tags 2022-09-10 23:05:41 -04:00
bors
beed5eddb0 Auto merge of #2539 - oli-obk:rustup, r=oli-obk
Rustup

for https://github.com/rust-lang/rust/issues/101649
2022-09-10 13:56:56 +00:00
Oli Scherer
e91db9f03c Rustup 2022-09-10 13:56:05 +00:00
bors
ef3f649e49 Auto merge of #2535 - oli-obk:rustup, r=oli-obk
Rustup

fix breakage in https://github.com/rust-lang/rust/pull/101522
2022-09-07 13:27:46 +00:00
Oli Scherer
f9ff70cef9 Rustup 2022-09-07 13:26:35 +00:00
bors
b8a97c4b97 Auto merge of #2534 - saethlin:rustup, r=saethlin
rustup: bring in Miri backtrace-rs pruning fix

The pruning logic in `backtrace` has always been broken in Miri. But now there's a fix!
2022-09-03 22:16:50 +00:00
Ben Kimock
4f30af5273 rustup: bring in Miri backtrace-rs pruning fix 2022-09-03 17:40:02 -04:00
bors
ec43f1dd9b Auto merge of #2533 - saethlin:windows-rng, r=ChrisDenton
Support BCRYPT_RNG_ALG_HANDLE rust-lang/rust#101325

I haven't tested this on a Windows host, brace for CI...
2022-09-03 19:32:09 +00:00
Ben Kimock
ee1c1e6d78 Add support for BCRYPT_RNG_ALG_HANDLE 2022-09-03 12:28:04 -04:00
bors
90731796c5 Auto merge of #2532 - Enselic:fix-build-with-unix_sigpipe, r=RalfJung
Fix build with `#[unix_sigpipe = "..."]` support in rustc

Closes https://github.com/rust-lang/rust/issues/101352

CC `@RalfJung` `@oli-obk`
2022-09-03 11:18:54 +00:00
Martin Nordholts
c011126f1a Fix build with #[unix_sigpipe = "..."] support in rustc 2022-09-03 12:23:05 +02:00
bors
dba35d2be7 Auto merge of #2530 - oli-obk:rustup, r=RalfJung
Rustup

Breakage issue: https://github.com/rust-lang/rust/issues/101324
2022-09-02 14:10:46 +00:00
Ralf Jung
169569cccb
tweak variable name 2022-09-02 16:10:24 +02:00
Oli Scherer
6a29a6842a Clippy after rustup 2022-09-02 13:55:26 +00:00
Oli Scherer
7155a2190e Rustup 2022-09-02 13:55:18 +00:00
bors
92c24c4c37 Auto merge of #2529 - RalfJung:yesffi, r=RalfJung
re-enable FFI support

https://github.com/tov/libffi-rs/pull/58 landed so the license should no longer be an issue. :)

Fixes https://github.com/rust-lang/miri/issues/2526
2022-09-02 12:41:03 +00:00
Ralf Jung
2f348abafc Revert "disable extern-so ffi support for now due to licensing situation"
This reverts commit 5f3545e773.
With https://github.com/tov/libffi-rs/pull/58 landed, we no longer depend on abort_on_panic.
2022-09-02 14:39:25 +02:00
bors
8c8b479be7 Auto merge of #2527 - RalfJung:noffi, r=RalfJung
disable extern-so ffi support for now due to licensing situation

libffi depends on abort_on_panic which has a [very unfortunate license situation going on](https://rust-lang.zulipchat.com/#narrow/stream/231349-t-core.2Flicensing). For now, in order to let us update Miri in rustc, I see no way but to disable our FFI support again. Sorry `@emarteca` :(

Cc https://github.com/rust-lang/miri/issues/2526
2022-09-01 20:28:06 +00:00
Ralf Jung
5f3545e773 disable extern-so ffi support for now due to licensing situation 2022-09-01 22:25:14 +02:00
bors
2d50443401 Auto merge of #2523 - saethlin:protector-test, r=RalfJung
Add a protector test that demonstrates the base tag diagnostic

Per https://github.com/rust-lang/miri/pull/2519#issuecomment-1232736295, this demonstrates this case for protector diagnostics:
```
help: <3131> was created here, as a base tag for alloc1623
  --> tests/fail/stacked_borrows/invalidate_against_protector3.rs:10:19
   |
10 |         let ptr = std::alloc::alloc(std::alloc::Layout::for_value(&0i32)) as *mut i32;
   |                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
```
This diagnostic is inspired by what Miri used to do with https://github.com/rust-lang/rust/issues/60076#issuecomment-1214169179
2022-09-01 13:49:27 +00:00
bors
8218248fd1 Auto merge of #2525 - RalfJung:zulip, r=RalfJung
Zulip notifications: ping the Miri team
2022-09-01 13:22:04 +00:00
Ralf Jung
e12962b4aa Zulip notifications: ping the Miri team 2022-09-01 15:20:05 +02:00
Ben Kimock
3cfb9915fc Add a protector test that demonstrates the base tag diagnostic 2022-08-31 21:37:44 -04:00
bors
0da5f76a92 Auto merge of #2516 - RalfJung:read-pointer-as-bytes, r=RalfJung
Adjust for supporting more implicit ptr-to-int transmutation

This is the Miri side of https://github.com/rust-lang/rust/pull/101101.
Fixes https://github.com/rust-lang/miri/issues/2456.
2022-08-31 16:25:06 +00:00
Ralf Jung
0113f9e727 rustup 2022-08-31 18:19:14 +02:00
Ralf Jung
d21b601b6e make Miri build again with rustc provenance changes 2022-08-31 18:19:09 +02:00
bors
95b315d7f6 Auto merge of #2524 - RalfJung:progress-report, r=RalfJung
fix progress report being deduplicated

Fixes https://github.com/rust-lang/miri/issues/2522
2022-08-31 15:33:17 +00:00
Ralf Jung
1135ad31ab make shim_arg_size ptr-width-independent 2022-08-31 17:32:39 +02:00
Ralf Jung
671a4b8b0f make backtrace header a bit more visible 2022-08-31 17:32:39 +02:00
Ralf Jung
4cb26afc0c fix progress report being deduplicated 2022-08-31 15:59:53 +02:00
bors
feeeba2925 Auto merge of #2521 - oli-obk:bump_ui_test, r=RalfJung
Bump UI test dependency

This gives us the new diff renderer as well as the ability to run tests without parallelism if we'd want to.
2022-08-31 11:37:27 +00:00
Ralf Jung
240f92aae8
add comment 2022-08-31 13:36:25 +02:00
Oli Scherer
eafc100b50 Bump ui_test to 0.3.1 2022-08-31 11:06:35 +00:00
Oli Scherer
b74654f25c Bump UI test dependency 2022-08-31 10:27:50 +00:00
bors
da45adc622 Auto merge of #2519 - saethlin:rustup, r=RalfJung
Use the better FnEntry spans in protector errors

Example error, from `tests/fail/stacked_borrows/invalidate_against_protector1.rs`:
```
error: Undefined Behavior: not granting access to tag <3095> because that would remove [Unique for <3099>] which is protected because it is an argument of call 943
  --> tests/fail/stacked_borrows/invalidate_against_protector1.rs:5:25
   |
5  |     let _val = unsafe { *x }; //~ ERROR: protect
   |                         ^^ not granting access to tag <3095> because that would remove [Unique for <3099>] which is protected because it is an argument of call 943
   |
   = help: this indicates a potential bug in the program: it performed an invalid operation, but the Stacked Borrows rules it violated are still experimental
   = help: see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md for further information
help: <3095> was created by a SharedReadWrite retag at offsets [0x0..0x4]
  --> tests/fail/stacked_borrows/invalidate_against_protector1.rs:10:16
   |
10 |     let xraw = &mut x as *mut _;
   |                ^^^^^^
help: <3099> is this argument
  --> tests/fail/stacked_borrows/invalidate_against_protector1.rs:1:23
   |
1  | fn inner(x: *mut i32, _y: &mut i32) {
   |                       ^^
   = note: backtrace:
   = note: inside `inner` at tests/fail/stacked_borrows/invalidate_against_protector1.rs:5:25
note: inside `main` at tests/fail/stacked_borrows/invalidate_against_protector1.rs:12:5
  --> tests/fail/stacked_borrows/invalidate_against_protector1.rs:12:5
   |
12 |     inner(xraw, xref);
   |     ^^^^^^^^^^^^^^^^^
```

Benchmarks report no change, within noise.
2022-08-31 10:06:05 +00:00
Ben Kimock
da0d4829bf Use the better FnEntry spans in protector errors 2022-08-29 18:06:30 -04:00
bors
284b59c4dc Auto merge of #2517 - saethlin:zst-field-retagging, r=RalfJung
Skip field retagging on ZSTs, it can take forever

I just tried running the `alloc`'s tests with `miri-test-libstd` with field retagging enabled. The test suite eventually hangs on a few tests which pass around ZSTs that have a lot of fields.

I don't really know how to test this effectively. The test passes, but if you remove this fast-path it effectively just hangs the interpreter. And since it hangs _inside_ a step, there's no hope for doing some kind of timeout within the test.
2022-08-29 11:46:26 +00:00
Ralf Jung
c9b36b4ded
clarify test purpose 2022-08-29 07:46:03 -04:00
Ben Kimock
70b960b879 Skip field retagging on ZSTs, it can take forever 2022-08-29 00:31:46 -04:00
bors
fec1c7aa32 Auto merge of #2513 - RalfJung:protected, r=saethlin
slightly improve protector-related error messages

I find the current retag messages confusing, since they sound like the item *was* protected, when it still actively *is* protected (and that is, in fact, the issue).

Example error message:
```
error: Undefined Behavior: not granting access to tag <3095> because incompatible item [Unique for <3099>] is protected by call 943
  --> tests/fail/stacked_borrows/invalidate_against_barrier1.rs:5:25
   |
5  |     let _val = unsafe { *x }; //~ ERROR: protect
   |                         ^^ not granting access to tag <3095> because incompatible item [Unique for <3099>] is protected by call 943
   |
   = help: this indicates a potential bug in the program: it performed an invalid operation, but the Stacked Borrows rules it violated are still experimental
   = help: see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md for further information
help: <3095> was created by a SharedReadWrite retag at offsets [0x0..0x4]
  --> tests/fail/stacked_borrows/invalidate_against_barrier1.rs:10:16
   |
10 |     let xraw = &mut x as *mut _;
   |                ^^^^^^
help: <3095> cannot be used for memory access because that would remove protected tag <3099>, protected by this function call
  --> tests/fail/stacked_borrows/invalidate_against_barrier1.rs:1:1
   |
1  | / fn inner(x: *mut i32, _y: &mut i32) {
2  | |     // If `x` and `y` alias, retagging is fine with this... but we really
3  | |     // shouldn't be allowed to use `x` at all because `y` was assumed to be
4  | |     // unique for the duration of this call.
5  | |     let _val = unsafe { *x }; //~ ERROR: protect
6  | | }
   | |_^
help: <3099> was derived from <3098>, which in turn was created here
  --> tests/fail/stacked_borrows/invalidate_against_barrier1.rs:12:17
   |
12 |     inner(xraw, xref);
   |                 ^^^^
   = note: backtrace:
   = note: inside `inner` at tests/fail/stacked_borrows/invalidate_against_barrier1.rs:5:25
note: inside `main` at tests/fail/stacked_borrows/invalidate_against_barrier1.rs:12:5
  --> tests/fail/stacked_borrows/invalidate_against_barrier1.rs:12:5
   |
12 |     inner(xraw, xref);
   |     ^^^^^^^^^^^^^^^^^
```

r? `@saethlin`
2022-08-28 16:01:49 +00:00
Ralf Jung
abe890d2ce slightly improve protector-related error messages
also rename some tests that still used outdated "barrier" terminology
2022-08-28 09:27:10 -04:00