Commit Graph

6823 Commits

Author SHA1 Message Date
bors
8ec3425a8a Auto merge of #2349 - saethlin:isatty, r=RalfJung
Improve isatty support

Per https://github.com/rust-lang/miri/issues/2292#issuecomment-1171858283, this is an attempt at

> do something more clever with Miri's `isatty` shim

Since Unix -> Unix is very simple, I'm starting with a patch that just does that. Happy to augment/rewrite this based on feedback.

The linked file in libtest specifically only supports stdout. If we're doing this to support terminal applications, I think it would be strange to support one but not all 3 of the standard streams.

The `atty` crate contains a bunch of extra logic that libtest does not contain, in order to support MSYS terminals: db8d55f88e so I think if we're going to do Windows support, we should probably access all that logic somehow. I think it's pretty clear that the implementation is not going to change, so I think if we want to, pasting the contents of the `atty` crate into Miri is on the table, instead of taking a dependency.
2022-07-18 01:37:38 +00:00
Ben Kimock
eefdeacdb1 Test that isatty doesn't crash 2022-07-17 21:13:59 -04:00
Ben Kimock
2f84cb34c1 Pass through isatty if the host is also unix 2022-07-17 16:53:14 -04:00
bors
36a7a654b0 Auto merge of #2378 - RalfJung:sb, r=RalfJung
use PlaceTy visitor and dedup sime retagging code

I benchmarked this and as far as I can see the difference to the old code is totally within noise. And this makes the code a lot simpler and removes duplication so yay. :)
2022-07-17 14:40:37 +00:00
Ralf Jung
39866f817a remove a fastpath that does not seem to actually help 2022-07-17 10:35:19 -04:00
Ralf Jung
68510600a3 use PlaceTy visitor 2022-07-17 10:19:29 -04:00
bors
416cddb251 Auto merge of #2376 - RalfJung:rustup, r=RalfJung
rustup
2022-07-17 12:19:31 +00:00
Ralf Jung
e8ab64e424 make unused flags work like they used to 2022-07-17 08:18:55 -04:00
bors
006bb3cd26 Auto merge of #2377 - rust-lang:only-on-host, r=oli-obk
Add test flag for running a test only on the host
2022-07-17 11:00:55 +00:00
Oli Scherer
8527bfc63b clippy 2022-07-17 08:07:26 +00:00
Oli Scherer
444841bba9 Add test flag for running a test only on the host 2022-07-17 07:56:57 +00:00
Oli Scherer
43100f5f4b Remove --offline from ./miri install as otherwise we can't add more dependencies to miri as CI will fail.
if you want this locally, pass it to your invocation.
2022-07-17 07:46:02 +00:00
Ralf Jung
9782b7b039 rustup 2022-07-16 23:40:36 -04:00
bors
86911fd8f6 Auto merge of #2368 - RalfJung:debug, r=oli-obk
Make "./miri {build,run,test}" use debug assertions but "./miri install" not

This makes `./miri run`/`./miri test` use the full set of debug assertions (including the rather expensive ones that check consistency of the Stacked Borrows cache), but `./miri install` installs a Miri *without* those debug assertions.

That's the same behavior as cargo, and helps catch Miri bugs with the test suite while making installed Miri usable for larger runs.
2022-07-15 15:54:47 +00:00
bors
7b79801d23 Auto merge of #2375 - RalfJung:rustup, r=RalfJung
Rustup
2022-07-15 13:39:11 +00:00
Ralf Jung
98c401977b rustup 2022-07-15 08:09:43 -04:00
Ralf Jung
8c13a9ccd3 clarify how to run the manually installed Miri 2022-07-15 08:02:13 -04:00
Ralf Jung
d6cbe5d014 ensure that RangeMap panics on OOB 2022-07-14 15:09:20 -04:00
Ralf Jung
421f92bee6 make some debug assertions in RangeObjectMap be full assertions 2022-07-14 13:23:35 -04:00
Ralf Jung
fa7811bbe1 use the cargo default for debug/release builds 2022-07-14 13:19:44 -04:00
Ralf Jung
5d5999ab13 make cache consistency checks into regular debug assertions 2022-07-14 13:00:35 -04:00
Ralf Jung
efc76af134 don't forcefully enable debug assertions, but make -debug mode usable
still set those flags on CI though, we want to catch overflow there
2022-07-14 13:00:35 -04:00
bors
3b1eeabc49 Auto merge of #2372 - RalfJung:rustup, r=RalfJung
rustup

Cc https://github.com/rust-lang/rust/issues/99224
2022-07-14 13:55:40 +00:00
Ralf Jung
eaa7f10cb1 rustup 2022-07-14 09:54:20 -04:00
bors
e0672589d3 Auto merge of #2370 - RalfJung:weak-mem-test, r=RalfJung
add another weak mem consistency test

`@cbeuw` your code seems to do this correctly? :D
2022-07-14 12:23:47 +00:00
Ralf Jung
b8a0c49e51 test that we can see this weak behavior 2022-07-14 08:23:31 -04:00
Ralf Jung
07c3e42bd7 replace a macro by a function 2022-07-14 08:17:18 -04:00
Ralf Jung
820f322b86 add work-around for #2164 2022-07-14 08:07:59 -04:00
bors
6f6e01ef3b Auto merge of #2371 - RalfJung:rustup, r=RalfJung
rustup

Cc https://github.com/rust-lang/rust/issues/99224
2022-07-14 11:59:46 +00:00
Ralf Jung
11f2b83464 fix a missing thread join 2022-07-14 07:59:22 -04:00
Ralf Jung
bd69a92b4b rustup 2022-07-13 22:35:45 -04:00
Ralf Jung
a272ca2cf7 add another weak mem consistency test 2022-07-13 21:50:19 -04:00
bors
af2c50fb89 Auto merge of #2328 - RalfJung:perf, r=RalfJung
move checking ptr tracking on item pop into cold helper function

Before:
```
Benchmark 1: cargo miri run --manifest-path bench-cargo-miri/serde1/Cargo.toml
  Time (mean ± σ):      6.729 s ±  0.050 s    [User: 6.608 s, System: 0.124 s]
  Range (min … max):    6.665 s …  6.799 s    5 runs

Benchmark 2: cargo miri run --manifest-path bench-cargo-miri/unicode/Cargo.toml
  Time (mean ± σ):     20.923 s ±  0.271 s    [User: 20.386 s, System: 0.537 s]
  Range (min … max):   20.580 s … 21.165 s    5 runs
```
After:
```
Benchmark 1: cargo miri run --manifest-path bench-cargo-miri/serde1/Cargo.toml
  Time (mean ± σ):      6.562 s ±  0.023 s    [User: 6.430 s, System: 0.135 s]
  Range (min … max):    6.544 s …  6.594 s    5 runs

Benchmark 2: cargo miri run --manifest-path bench-cargo-miri/unicode/Cargo.toml
  Time (mean ± σ):     20.375 s ±  0.228 s    [User: 19.964 s, System: 0.413 s]
  Range (min … max):   20.201 s … 20.736 s    5 runs
```
Nothing major, but we'll take it I guess. 🤷

Fixes https://github.com/rust-lang/miri/issues/2132
2022-07-14 00:34:00 +00:00
bors
e16705de25 Auto merge of #2369 - RalfJung:reborrow, r=RalfJung
reborrow error: clarify that we are reborrowing *from* that tag

`@saethlin` I found the current message not entirely clear, so what do you think about this?
2022-07-14 00:04:40 +00:00
Ralf Jung
cc42cb1b21 reborrow error: clarify that we are reborrowing *from* that tag 2022-07-13 19:40:53 -04:00
Ralf Jung
83b9172774 move stacked_borrows.rs together with the other files of its module 2022-07-13 19:37:41 -04:00
bors
8884ea48af Auto merge of #2367 - RalfJung:ui-test-regex, r=RalfJung
use ui_test regex capabilities

We can have more precise patterns now, so use them for some of the tests where that makes sense. :)
2022-07-13 23:00:23 +00:00
Ralf Jung
757e88c901 use ui_test regex capabilities 2022-07-13 18:59:33 -04:00
Ralf Jung
3bd0e8a2ca move checking ptr tracking on item pop into cold helper function 2022-07-13 18:07:33 -04:00
bors
4e9de31174 Auto merge of #2364 - rust-lang:regex_error_annotations, r=RalfJung
Error patterns can be regexes

fixes #2362
2022-07-13 16:53:38 +00:00
Oli Scherer
837bf84271 Error patterns can be regexes 2022-07-13 14:36:03 +00:00
bors
a45d6ef779 Auto merge of #2352 - saethlin:new-benchmark, r=RalfJung
Add a benchmark of the hang-on-test-failure code path

This is the code pattern that produces the performance problem in https://github.com/rust-lang/miri/issues/2273

I figured out what I was stuck on in https://github.com/rust-lang/miri/pull/2315#discussion_r916387919. For a while I was just doing `let x: &[u8] = &[0u8; 4096];` but that doesn't produce the runtime inside `Stack::item_popped` that I was looking for, I think because this allocation is never deallocated. But with `Vec`, I get the profile I'm looking for.
2022-07-13 14:24:14 +00:00
Ben Kimock
7d9f04f7a8 Add a benchmark of the hang-on-test-failure code path 2022-07-13 09:24:19 -04:00
bors
db5a2b9747 Auto merge of #2315 - saethlin:shrink-item, r=saethlin
Optimizing Stacked Borrows (part 2): Shrink Item

This moves protectors out of `Item`, storing them both in a global `HashSet` which contains all currently-protected tags as well as a `Vec<SbTag>` on each `Frame` so that when we return from a function we know which tags to remove from the protected set.

This also bit-packs the 64-bit tag and the 2-bit permission together when they are stored in memory. This means we theoretically run out of tags sooner, but I doubt that limit will ever be hit.

Together these optimizations reduce the memory footprint of Miri when executing programs which stress Stacked Borrows by ~66%. For example, running a test with isolation off which only panics currently peaks at ~19 GB, with this PR it peaks at ~6.2 GB.

To-do
- [x] Enforce the 62-bit limit
- [x] Decide if there is a better order to pack the tag and permission in
- [x] Wait for `UnsafeCell` to become infectious, or express offsets + tags in the global protector set

Benchmarks before:
```
Benchmark 1: cargo +miri miri run --manifest-path bench-cargo-miri/backtraces/Cargo.toml
  Time (mean ± σ):      8.948 s ±  0.253 s    [User: 8.752 s, System: 0.158 s]
  Range (min … max):    8.619 s …  9.279 s    5 runs

Benchmark 1: cargo +miri miri run --manifest-path bench-cargo-miri/mse/Cargo.toml
  Time (mean ± σ):      2.129 s ±  0.037 s    [User: 1.849 s, System: 0.248 s]
  Range (min … max):    2.086 s …  2.176 s    5 runs

Benchmark 1: cargo +miri miri run --manifest-path bench-cargo-miri/serde1/Cargo.toml
  Time (mean ± σ):      3.334 s ±  0.017 s    [User: 3.211 s, System: 0.103 s]
  Range (min … max):    3.315 s …  3.352 s    5 runs

Benchmark 1: cargo +miri miri run --manifest-path bench-cargo-miri/serde2/Cargo.toml
  Time (mean ± σ):      3.316 s ±  0.038 s    [User: 3.207 s, System: 0.095 s]
  Range (min … max):    3.282 s …  3.375 s    5 runs

Benchmark 1: cargo +miri miri run --manifest-path bench-cargo-miri/unicode/Cargo.toml
  Time (mean ± σ):      6.391 s ±  0.323 s    [User: 5.928 s, System: 0.412 s]
  Range (min … max):    6.090 s …  6.917 s    5 runs
 ```
 After:
 ```
Benchmark 1: cargo +miri miri run --manifest-path bench-cargo-miri/backtraces/Cargo.toml
  Time (mean ± σ):      6.955 s ±  0.051 s    [User: 6.807 s, System: 0.132 s]
  Range (min … max):    6.900 s …  7.038 s    5 runs

Benchmark 1: cargo +miri miri run --manifest-path bench-cargo-miri/mse/Cargo.toml
  Time (mean ± σ):      1.784 s ±  0.012 s    [User: 1.627 s, System: 0.156 s]
  Range (min … max):    1.772 s …  1.797 s    5 runs

Benchmark 1: cargo +miri miri run --manifest-path bench-cargo-miri/serde1/Cargo.toml
  Time (mean ± σ):      2.505 s ±  0.095 s    [User: 2.311 s, System: 0.096 s]
  Range (min … max):    2.405 s …  2.603 s    5 runs

Benchmark 1: cargo +miri miri run --manifest-path bench-cargo-miri/serde2/Cargo.toml
  Time (mean ± σ):      2.449 s ±  0.031 s    [User: 2.306 s, System: 0.100 s]
  Range (min … max):    2.395 s …  2.467 s    5 runs

Benchmark 1: cargo +miri miri run --manifest-path bench-cargo-miri/unicode/Cargo.toml
  Time (mean ± σ):      3.667 s ±  0.110 s    [User: 3.498 s, System: 0.140 s]
  Range (min … max):    3.564 s …  3.814 s    5 runs
 ```
 The decrease in system time is probably due to spending less time in the page fault handler.
2022-07-13 01:44:01 +00:00
Ben Kimock
4eff60ad6e Rearrange and document the new implementation
stacked_borrow now has an item module, and its own FrameExtra. These
serve to protect the implementation of Item (which is a bunch of
bit-packing tricks) from the primary logic of Stacked Borrows, and the
FrameExtra we have separates Stacked Borrows more cleanly from the
interpreter itself.

The new strategy for checking protectors also makes some subtle
performance tradeoffs, so they are now documented in Stack::item_popped
because that function primarily benefits from them, and it also touches
every aspect of them.

Also separating the actual CallId that is protecting a Tag from the Tag
makes it inconvienent to reproduce exactly the same protector errors, so
this also takes the opportunity to use some slightly cleaner English in
those errors. We need to make some change, might as well make it good.
2022-07-12 21:03:54 -04:00
Ben Kimock
afa1dddcf9 Store protectors outside Item, pack Tag and Perm
Previously, Item was a struct of a NonZeroU64, an Option which was
usually unset or irrelevant, and a 4-variant enum. So collectively, the
size of an Item was 24 bytes, but only 8 bytes were used for the most
part.

So this takes advantage of the fact that it is probably impossible to
exhaust the total space of SbTags, and steals 3 bits from it to pack the
whole struct into a single u64. This bit-packing means that we reduce
peak memory usage when Miri goes memory-bound by ~3x. We also get CPU
performance improvements of varying size, because not only are we simply
accessing less memory, we can now compare a Vec<Item> using a memcmp
because it does not have any padding.
2022-07-12 21:01:33 -04:00
bors
f223a5fe70 Auto merge of #2361 - RalfJung:uninit, r=RalfJung
test that we also find bad uses of mem::uninitialized

Also we really don't need to separately test signed and unsigned integers... our test suite is big enough as it is. ;)
2022-07-12 13:56:49 +00:00
Ralf Jung
cd3535d256 test that we also find bad uses of mem::uninitialized 2022-07-12 09:55:28 -04:00
bors
2643e41674 Auto merge of #2359 - RalfJung:triagebot, r=RalfJung
set up triagebot

`@Mark-Simulacrum` does this look like a sensible configuration?
2022-07-12 02:56:24 +00:00
Ralf Jung
9f1cab53a7 allow some relabeling as well 2022-07-11 22:55:59 -04:00