Commit Graph

6803 Commits

Author SHA1 Message Date
Ralf Jung
5d5999ab13 make cache consistency checks into regular debug assertions 2022-07-14 13:00:35 -04:00
Ralf Jung
efc76af134 don't forcefully enable debug assertions, but make -debug mode usable
still set those flags on CI though, we want to catch overflow there
2022-07-14 13:00:35 -04:00
bors
3b1eeabc49 Auto merge of #2372 - RalfJung:rustup, r=RalfJung
rustup

Cc https://github.com/rust-lang/rust/issues/99224
2022-07-14 13:55:40 +00:00
Ralf Jung
eaa7f10cb1 rustup 2022-07-14 09:54:20 -04:00
bors
e0672589d3 Auto merge of #2370 - RalfJung:weak-mem-test, r=RalfJung
add another weak mem consistency test

`@cbeuw` your code seems to do this correctly? :D
2022-07-14 12:23:47 +00:00
Ralf Jung
b8a0c49e51 test that we can see this weak behavior 2022-07-14 08:23:31 -04:00
Ralf Jung
07c3e42bd7 replace a macro by a function 2022-07-14 08:17:18 -04:00
Ralf Jung
820f322b86 add work-around for #2164 2022-07-14 08:07:59 -04:00
bors
6f6e01ef3b Auto merge of #2371 - RalfJung:rustup, r=RalfJung
rustup

Cc https://github.com/rust-lang/rust/issues/99224
2022-07-14 11:59:46 +00:00
Ralf Jung
11f2b83464 fix a missing thread join 2022-07-14 07:59:22 -04:00
Ralf Jung
bd69a92b4b rustup 2022-07-13 22:35:45 -04:00
Ralf Jung
a272ca2cf7 add another weak mem consistency test 2022-07-13 21:50:19 -04:00
bors
af2c50fb89 Auto merge of #2328 - RalfJung:perf, r=RalfJung
move checking ptr tracking on item pop into cold helper function

Before:
```
Benchmark 1: cargo miri run --manifest-path bench-cargo-miri/serde1/Cargo.toml
  Time (mean ± σ):      6.729 s ±  0.050 s    [User: 6.608 s, System: 0.124 s]
  Range (min … max):    6.665 s …  6.799 s    5 runs

Benchmark 2: cargo miri run --manifest-path bench-cargo-miri/unicode/Cargo.toml
  Time (mean ± σ):     20.923 s ±  0.271 s    [User: 20.386 s, System: 0.537 s]
  Range (min … max):   20.580 s … 21.165 s    5 runs
```
After:
```
Benchmark 1: cargo miri run --manifest-path bench-cargo-miri/serde1/Cargo.toml
  Time (mean ± σ):      6.562 s ±  0.023 s    [User: 6.430 s, System: 0.135 s]
  Range (min … max):    6.544 s …  6.594 s    5 runs

Benchmark 2: cargo miri run --manifest-path bench-cargo-miri/unicode/Cargo.toml
  Time (mean ± σ):     20.375 s ±  0.228 s    [User: 19.964 s, System: 0.413 s]
  Range (min … max):   20.201 s … 20.736 s    5 runs
```
Nothing major, but we'll take it I guess. 🤷

Fixes https://github.com/rust-lang/miri/issues/2132
2022-07-14 00:34:00 +00:00
bors
e16705de25 Auto merge of #2369 - RalfJung:reborrow, r=RalfJung
reborrow error: clarify that we are reborrowing *from* that tag

`@saethlin` I found the current message not entirely clear, so what do you think about this?
2022-07-14 00:04:40 +00:00
Ralf Jung
cc42cb1b21 reborrow error: clarify that we are reborrowing *from* that tag 2022-07-13 19:40:53 -04:00
Ralf Jung
83b9172774 move stacked_borrows.rs together with the other files of its module 2022-07-13 19:37:41 -04:00
bors
8884ea48af Auto merge of #2367 - RalfJung:ui-test-regex, r=RalfJung
use ui_test regex capabilities

We can have more precise patterns now, so use them for some of the tests where that makes sense. :)
2022-07-13 23:00:23 +00:00
Ralf Jung
757e88c901 use ui_test regex capabilities 2022-07-13 18:59:33 -04:00
Ralf Jung
3bd0e8a2ca move checking ptr tracking on item pop into cold helper function 2022-07-13 18:07:33 -04:00
bors
4e9de31174 Auto merge of #2364 - rust-lang:regex_error_annotations, r=RalfJung
Error patterns can be regexes

fixes #2362
2022-07-13 16:53:38 +00:00
Oli Scherer
837bf84271 Error patterns can be regexes 2022-07-13 14:36:03 +00:00
bors
a45d6ef779 Auto merge of #2352 - saethlin:new-benchmark, r=RalfJung
Add a benchmark of the hang-on-test-failure code path

This is the code pattern that produces the performance problem in https://github.com/rust-lang/miri/issues/2273

I figured out what I was stuck on in https://github.com/rust-lang/miri/pull/2315#discussion_r916387919. For a while I was just doing `let x: &[u8] = &[0u8; 4096];` but that doesn't produce the runtime inside `Stack::item_popped` that I was looking for, I think because this allocation is never deallocated. But with `Vec`, I get the profile I'm looking for.
2022-07-13 14:24:14 +00:00
Ben Kimock
7d9f04f7a8 Add a benchmark of the hang-on-test-failure code path 2022-07-13 09:24:19 -04:00
bors
db5a2b9747 Auto merge of #2315 - saethlin:shrink-item, r=saethlin
Optimizing Stacked Borrows (part 2): Shrink Item

This moves protectors out of `Item`, storing them both in a global `HashSet` which contains all currently-protected tags as well as a `Vec<SbTag>` on each `Frame` so that when we return from a function we know which tags to remove from the protected set.

This also bit-packs the 64-bit tag and the 2-bit permission together when they are stored in memory. This means we theoretically run out of tags sooner, but I doubt that limit will ever be hit.

Together these optimizations reduce the memory footprint of Miri when executing programs which stress Stacked Borrows by ~66%. For example, running a test with isolation off which only panics currently peaks at ~19 GB, with this PR it peaks at ~6.2 GB.

To-do
- [x] Enforce the 62-bit limit
- [x] Decide if there is a better order to pack the tag and permission in
- [x] Wait for `UnsafeCell` to become infectious, or express offsets + tags in the global protector set

Benchmarks before:
```
Benchmark 1: cargo +miri miri run --manifest-path bench-cargo-miri/backtraces/Cargo.toml
  Time (mean ± σ):      8.948 s ±  0.253 s    [User: 8.752 s, System: 0.158 s]
  Range (min … max):    8.619 s …  9.279 s    5 runs

Benchmark 1: cargo +miri miri run --manifest-path bench-cargo-miri/mse/Cargo.toml
  Time (mean ± σ):      2.129 s ±  0.037 s    [User: 1.849 s, System: 0.248 s]
  Range (min … max):    2.086 s …  2.176 s    5 runs

Benchmark 1: cargo +miri miri run --manifest-path bench-cargo-miri/serde1/Cargo.toml
  Time (mean ± σ):      3.334 s ±  0.017 s    [User: 3.211 s, System: 0.103 s]
  Range (min … max):    3.315 s …  3.352 s    5 runs

Benchmark 1: cargo +miri miri run --manifest-path bench-cargo-miri/serde2/Cargo.toml
  Time (mean ± σ):      3.316 s ±  0.038 s    [User: 3.207 s, System: 0.095 s]
  Range (min … max):    3.282 s …  3.375 s    5 runs

Benchmark 1: cargo +miri miri run --manifest-path bench-cargo-miri/unicode/Cargo.toml
  Time (mean ± σ):      6.391 s ±  0.323 s    [User: 5.928 s, System: 0.412 s]
  Range (min … max):    6.090 s …  6.917 s    5 runs
 ```
 After:
 ```
Benchmark 1: cargo +miri miri run --manifest-path bench-cargo-miri/backtraces/Cargo.toml
  Time (mean ± σ):      6.955 s ±  0.051 s    [User: 6.807 s, System: 0.132 s]
  Range (min … max):    6.900 s …  7.038 s    5 runs

Benchmark 1: cargo +miri miri run --manifest-path bench-cargo-miri/mse/Cargo.toml
  Time (mean ± σ):      1.784 s ±  0.012 s    [User: 1.627 s, System: 0.156 s]
  Range (min … max):    1.772 s …  1.797 s    5 runs

Benchmark 1: cargo +miri miri run --manifest-path bench-cargo-miri/serde1/Cargo.toml
  Time (mean ± σ):      2.505 s ±  0.095 s    [User: 2.311 s, System: 0.096 s]
  Range (min … max):    2.405 s …  2.603 s    5 runs

Benchmark 1: cargo +miri miri run --manifest-path bench-cargo-miri/serde2/Cargo.toml
  Time (mean ± σ):      2.449 s ±  0.031 s    [User: 2.306 s, System: 0.100 s]
  Range (min … max):    2.395 s …  2.467 s    5 runs

Benchmark 1: cargo +miri miri run --manifest-path bench-cargo-miri/unicode/Cargo.toml
  Time (mean ± σ):      3.667 s ±  0.110 s    [User: 3.498 s, System: 0.140 s]
  Range (min … max):    3.564 s …  3.814 s    5 runs
 ```
 The decrease in system time is probably due to spending less time in the page fault handler.
2022-07-13 01:44:01 +00:00
Ben Kimock
4eff60ad6e Rearrange and document the new implementation
stacked_borrow now has an item module, and its own FrameExtra. These
serve to protect the implementation of Item (which is a bunch of
bit-packing tricks) from the primary logic of Stacked Borrows, and the
FrameExtra we have separates Stacked Borrows more cleanly from the
interpreter itself.

The new strategy for checking protectors also makes some subtle
performance tradeoffs, so they are now documented in Stack::item_popped
because that function primarily benefits from them, and it also touches
every aspect of them.

Also separating the actual CallId that is protecting a Tag from the Tag
makes it inconvienent to reproduce exactly the same protector errors, so
this also takes the opportunity to use some slightly cleaner English in
those errors. We need to make some change, might as well make it good.
2022-07-12 21:03:54 -04:00
Ben Kimock
afa1dddcf9 Store protectors outside Item, pack Tag and Perm
Previously, Item was a struct of a NonZeroU64, an Option which was
usually unset or irrelevant, and a 4-variant enum. So collectively, the
size of an Item was 24 bytes, but only 8 bytes were used for the most
part.

So this takes advantage of the fact that it is probably impossible to
exhaust the total space of SbTags, and steals 3 bits from it to pack the
whole struct into a single u64. This bit-packing means that we reduce
peak memory usage when Miri goes memory-bound by ~3x. We also get CPU
performance improvements of varying size, because not only are we simply
accessing less memory, we can now compare a Vec<Item> using a memcmp
because it does not have any padding.
2022-07-12 21:01:33 -04:00
bors
f223a5fe70 Auto merge of #2361 - RalfJung:uninit, r=RalfJung
test that we also find bad uses of mem::uninitialized

Also we really don't need to separately test signed and unsigned integers... our test suite is big enough as it is. ;)
2022-07-12 13:56:49 +00:00
Ralf Jung
cd3535d256 test that we also find bad uses of mem::uninitialized 2022-07-12 09:55:28 -04:00
bors
2643e41674 Auto merge of #2359 - RalfJung:triagebot, r=RalfJung
set up triagebot

`@Mark-Simulacrum` does this look like a sensible configuration?
2022-07-12 02:56:24 +00:00
Ralf Jung
9f1cab53a7 allow some relabeling as well 2022-07-11 22:55:59 -04:00
Ralf Jung
49a6c230cb remove deprecated options 2022-07-11 21:28:54 -04:00
Ralf Jung
c6061116e9 set up triagebot 2022-07-11 21:27:09 -04:00
bors
58e7b8f62f Auto merge of #2356 - rust-lang:slash_slash_at, r=RalfJung
require level and colon in `//~` style comments

implements https://github.com/rust-lang/miri/pull/2348#issuecomment-1179551953
2022-07-11 20:13:00 +00:00
Oli Scherer
bfc23e2cdd Document require-annotations-for-level 2022-07-11 17:25:56 +00:00
Oli Scherer
1ca8d60734 Fix annotation filtering with revisions 2022-07-11 17:19:10 +00:00
Oli Scherer
01253437ea Pacify clippy 2022-07-11 12:21:02 +00:00
Oli Scherer
a64f9e7250 Remove error annotations stderr
they still showed up in multiline messages
2022-07-11 11:50:33 +00:00
Oli Scherer
f3496cbe4b require level and colon in //~ style comments 2022-07-11 11:48:56 +00:00
bors
97d115cd75 Auto merge of #2354 - Techie-Pi:patch-1, r=RalfJung
Fix README typo

"behaivours" -> "behaviours".
Just something I noticed while reading
2022-07-11 00:47:16 +00:00
bors
b994fea1f8 Auto merge of #2355 - RalfJung:rustup, r=RalfJung
make a test deterministic
2022-07-11 00:21:38 +00:00
Ralf Jung
444ba75ac5 make more tests deterministic 2022-07-10 20:21:19 -04:00
Ralf Jung
45abee46be make a test deterministic 2022-07-10 20:15:35 -04:00
TechiePi
bf5af84640
Fix README typo
"behaivours" -> "behaviours".
Just something I noticed while reading
2022-07-10 23:41:35 +02:00
bors
aada09f79a Auto merge of #2351 - RalfJung:readlink, r=RalfJung
extend a comment in readlink
2022-07-09 16:50:00 +00:00
Ralf Jung
c9b207eba6 extend a comment in readlink 2022-07-09 12:49:37 -04:00
bors
6bdc5d6f4c Auto merge of #2348 - RalfJung:ui-test, r=RalfJung
ui_test: require colon after command

Turns out we already use colons everywhere properly.

Also rename ui_test::comments to parser. I think that better describes its contents. :)
2022-07-09 11:51:32 +00:00
Ralf Jung
2850db9f6b
typo
Co-authored-by: Oli Scherer <github35764891676564198441@oli-obk.de>
2022-07-09 07:51:06 -04:00
Ralf Jung
d260dffa19 rename ui_test::comments → parser 2022-07-08 17:18:17 -04:00
Ralf Jung
fdffaf740b ui_test: require colon after command 2022-07-08 17:17:44 -04:00
bors
7a6a812794 Auto merge of #2332 - rust-lang:slash_slash_at, r=oli-obk
More robust comment parsing

fixes #2170

I haven't ported the entire test suite yet. Once we've done that, I will remove the old parsing system (or in fact, turn them into errors so that accidental usage of old-style comments will be detected)
2022-07-08 16:09:06 +00:00