Add retag in MIR transform: `Adt` for `Unique` may contain a reference
Following #112662 , `may_contain_reference` in `rustc_mir_transform::add_retag` underapproximates too much the types that require retagging.
r? ``@RalfJung``
Add a fully fledged `Clause` type, rename old `Clause` to `ClauseKind`
Does two basic things before I put up a more delicate set of PRs (along the lines of #112714, but hopefully much cleaner) that migrate existing usages of `ty::Predicate` to `ty::Clause` (`predicates_of`/`item_bounds`/`ParamEnv::caller_bounds`).
1. Rename `Clause` to `ClauseKind`, so it's parallel with `PredicateKind`.
2. Add a new `Clause` type which is parallel to `Predicate`.
* This type exposes `Clause::kind(self) -> Binder<'tcx, ClauseKind<'tcx>>` which is parallel to `Predicate::kind` 😸
The new `Clause` type essentially acts as a newtype wrapper around `Predicate` that asserts that it is specifically a `PredicateKind::Clause`. Turns out from experimentation[^1] that this is not negative performance-wise, which is wonderful, since this a much simpler design than something that requires encoding the discriminant into the alignment bits of a predicate kind, or something else like that...
r? ``@lcnr`` or ``@oli-obk``
[^1]: https://github.com/rust-lang/rust/pull/112714#issuecomment-1595653910
Switch the BB CFG cache from postorder to RPO
The `BasicBlocks` CFG cache is interesting:
- it stores a postorder, but `traversal::postorder` doesn't use it
- `traversal::reverse_postorder` does traverse the postorder cache backwards
- we do more RPO traversals than postorder traversals (around 20x on the perf.rlo benchmarks IIRC) but it's not cached
- a couple places here and there were manually reversing the non-cached postorder traversal
This PR switches the order of the cache, and makes a bit more use of it. This is a tiny win locally, but it's also for consistency and aesthetics.
r? `@ghost`
Launch a non-unwinding panic for misaligned pointer deref
This panic already never unwinds, but that's only because it always hits the unwind guard that's created by our `UnwindAction::Terminate`. Hitting the unwind guard generates a huge double-panic backtrace. Now we generate a normal-looking panic message when this check is hit.
r? `@thomcc`
Prevent `.eh_frame` from being emitted for `-C panic=abort`
Since `CheckAlignment` pass is after the `AbortUnwindingCalls` pass, the `UnwindAction::Terminate` inserted in it has no chance to be converted to `UnwindAction::Unreachable` anymore, causing us to emit landing pads that are not necessary. Although these landing pads can themselves be eliminated by LLVM, `.eh_frame` sections are still generated. This causes trouble for Rust-for-Linux project recently.
This PR changes it to generate `UnwindAction::Terminate` when we opt for `-Cpanic=unwind`, and `UnwindAction::Unreachable` for `-Cpanic=abort`.
`@ojeda`
Add MVP suggestion for `unsafe_op_in_unsafe_fn`
Rebase of https://github.com/rust-lang/rust/pull/99827
cc tracking issue https://github.com/rust-lang/rust/issues/71668
No real changes since the original PR, just migrated the new suggestion to use fluent messages and added a couple more testcases, AFAICT from the discussion there were no outstanding changes requested.
Write to stdout if `-` is given as output file
With this PR, if `-o -` or `--emit KIND=-` is provided, output will be written to stdout instead. Binary output (those of type `obj`, `llvm-bc`, `link` and `metadata`) being written this way will result in an error unless stdout is not a tty. Multiple output types going to stdout will trigger an error too, as they will all be mixded together.
This implements https://github.com/rust-lang/compiler-team/issues/431
The idea behind the changes is to introduce an `OutFileName` enum that represents the output - be it a real path or stdout - and to use this enum along the code paths that handle different output types.
Take MIR dataflow analyses by mutable reference
The main motivation here is any analysis requiring dynamically sized scratch memory to work. One concrete example would be pointer target tracking, where tracking the results of a dereference can result in multiple possible targets. This leads to processing multi-level dereferences requiring the ability to handle a changing number of potential targets per step. A (simplified) function for this would be `fn apply_deref(potential_targets: &mut Vec<Target>)` which would use the scratch space contained in the analysis to send arguments and receive the results.
The alternative to this would be to wrap everything in a `RefCell`, which is what `MaybeRequiresStorage` currently does. This comes with a small perf cost and loses the compiler's guarantee that we don't try to take multiple borrows at the same time.
For the implementation:
* `AnalysisResults` is an unfortunate requirement to avoid an unconstrained type parameter error.
* `CloneAnalysis` could just be `Clone` instead, but that would result in more work than is required to have multiple cursors over the same result set.
* `ResultsVisitor` now takes the results type on in each function as there's no other way to have access to the analysis without cloning it. This could use an associated type rather than a type parameter, but the current approach makes it easier to not care about the type when it's not necessary.
* `MaybeRequiresStorage` now no longer uses a `RefCell`, but the graphviz formatter now does. It could be removed, but that would require even more changes and doesn't really seem necessary.
If `-o -` or `--emit KIND=-` is provided, output will be written
to stdout instead. Binary output (`obj`, `llvm-bc`, `link` and
`metadata`) being written this way will result in an error unless
stdout is not a tty. Multiple output types going to stdout will
trigger an error too, as they will all be mixded together.
Only check inlining counter after recursing.
This PR aims to reduce the strength of https://github.com/rust-lang/rust/pull/105119 even more.
In the current implementation, we check the inline count before recursing. This means that we never actually reach inlining depth 3.
This PR checks the counter after recursion, to give a chance to inline at depth >= 3.
r? `@scottmcm`
cc `@JakobDegen`
Use translatable diagnostics in `rustc_const_eval`
This PR:
* adds a `no_span` parameter to `note` / `help` attributes when using `Subdiagnostic` to allow adding notes/helps without using a span
* has minor tweaks and changes to error messages
Enable ConstGoto and SeparateConstSwitch passes by default
These 2 passes implement a limited form of jump-threading.
Filing this PR to see if enabling them would be lighter than https://github.com/rust-lang/rust/pull/107009.
Enable ScalarReplacementOfAggregates in optimized builds
Like MatchBranchSimplification, this pass is known to produce significant runtime improvements in Cranelift artifacts, and I believe based on the perf runs here that the primary effect of this pass is to empower MatchBranchSimplification. ScalarReplacementOfAggregates on its own has little effect on anything, but when this was rebased up to include https://github.com/rust-lang/rust/pull/112001 we started seeing significant and majority-positive results.
Based on the fact that we see most of the regressions in debug builds (https://github.com/rust-lang/rust/pull/112002#issuecomment-1566270144) and some rather significant ones in cycles and wall time, I'm only enabling this in optimized builds at the moment.
Only rewrite valtree-constants to patterns and keep other constants opaque
Now that we can reliably fall back to comparing constants with `PartialEq::eq` to the match scrutinee, we can
1. eagerly try to convert constants to valtrees
2. then deeply convert the valtree to a pattern
3. if the to-valtree conversion failed, create an "opaque constant" pattern.
This PR specifically avoids any behavioral changes or major cleanups. What we can now do as follow ups is
* move the two remaining call sites to `destructure_mir_constant` off that query
* make valtree to pattern conversion infallible
* this needs to be done after careful analysis of the effects. There may be user visible changes from that.
based on https://github.com/rust-lang/rust/pull/111768
change `BorrowKind::Unique` to be a mutating `PlaceContext`
fixes#112056
I believe that `BorrowKind::Unique` is a footgun in general, so I added a FIXME and opened https://github.com/rust-lang/rust/issues/112072. This is a bit too involved for this PR though.
MIR: opt-in normalization of `BasicBlock` and `Local` numbering
This doesn't matter at all for actual codegen, but after spending some time reading pre-codegen MIR, I was wishing I didn't have to jump around so much in reading post-inlining code.
So this add two passes that are off by default for every mir level, but can be enabled (`-Zmir-enable-passes=+ReorderBasicBlocks,+ReorderLocals`) for humans.
Don't check for misaligned raw pointer derefs inside Rvalue::AddressOf
From https://github.com/rust-lang/rust/pull/112026#issuecomment-1565686697:
rustc 1.70 (stable next week) added a Mir pass to add pointer alignment checks in debug mode. Adding these checks caused some crates to break, but that was expected, since they contain broken code (https://github.com/rust-lang/rust/issues/111487) for tracking that.
However, the checks added are slightly more aggressive than they should have been. Specifically, they also check the place in an `addr_of!` expression. Whether lack of alignment there is or isn't UB is unclear. This PR modifies the pass to not affect those cases.
I spot checked the crater regressions and the ones I saw were not the case that this PR is modifying. It still seems good to not land anything overaggressive though
Enable MatchBranchSimplification
This pass is one of the small number of benefits from `-Zmir-opt-level=3` that has motivated rustc_codegen_cranelift to use it:
19ed0aade6/compiler/rustc_codegen_cranelift/build_system/build_sysroot.rs (L244-L246)
Cranelift's motivation for this is _runtime_ performance improvements in debug builds. Lifting this pass all the way to `-Zmir-opt-level=1` seems to come without significant perf overhead, so that's what I'm suggesting here.
Ensure Fluent messages are in alphabetical order
Fixes#111847
This adds a tidy check to ensure Fluent messages are in alphabetical order, as well as sorting all existing messages. I think the error could be worded better, would appreciate suggestions.
<details>
<summary>Script used to sort files</summary>
```py
import sys
import re
fn = sys.argv[1]
with open(fn, 'r') as f:
data = f.read().split("\n")
chunks = []
cur = ""
for line in data:
if re.match(r"^([a-zA-Z0-9_]+)\s*=\s*", line):
chunks.append(cur)
cur = ""
cur += line + "\n"
chunks.append(cur)
chunks.sort()
with open(fn, 'w') as f:
f.write(''.join(chunks).strip("\n\n") + "\n")
```
</details>
Rollup of 5 pull requests
Successful merges:
- #111741 (Use `ObligationCtxt` in custom type ops)
- #111840 (Expose more information in `get_body_with_borrowck_facts`)
- #111876 (Roll compiler_builtins to 0.1.92)
- #111912 (Use `Option::is_some_and` and `Result::is_ok_and` in the compiler )
- #111915 (libtest: Improve error when missing `-Zunstable-options`)
r? `@ghost`
`@rustbot` modify labels: rollup
Use `Option::is_some_and` and `Result::is_ok_and` in the compiler
`.is_some_and(..)`/`.is_ok_and(..)` replace `.map_or(false, ..)` and `.map(..).unwrap_or(false)`, making the code more readable.
This PR is a sibling of https://github.com/rust-lang/rust/pull/111873#issuecomment-1561316515
Work around `rust-analyzer` false-positive type errors
rust-analyzer incorrectly reports two type errors in `debug.rs`:
> expected &dyn Display, found &i32
> expected &dyn Display, found &i32
This is due to a known bug in r-a: (https://github.com/rust-lang/rust-analyzer/issues/11847).
In these particular cases, changing `&0` to `&0i32` seems to be enough to avoid the bug.
Preprocess and cache dominator tree
Preprocessing dominators has a very strong effect for https://github.com/rust-lang/rust/pull/111344.
That pass checks that assignments dominate their uses repeatedly. Using the unprocessed dominator tree caused a quadratic runtime (number of bbs x depth of the dominator tree).
This PR also caches the dominator tree and the pre-processed dominators in the MIR cfg cache.
Rebase of https://github.com/rust-lang/rust/pull/107157
cc `@tmiasko`
Stop turning transmutes into discriminant reads in mir-opt
Partially reverts #109612, as after #109993 these aren't actually equivalent any more, and I'm no longer confident this was ever an improvement in the first place.
Having this "simplification" meant that similar-looking code actually did somewhat different things. For example,
```rust
pub unsafe fn demo1(x: std::cmp::Ordering) -> u8 {
std::mem::transmute(x)
}
pub unsafe fn demo2(x: std::cmp::Ordering) -> i8 {
std::mem::transmute(x)
}
```
in nightly today is generating <https://rust.godbolt.org/z/dPK58zW18>
```llvm
define noundef i8 `@_ZN7example5demo117h341ef313673d2ee6E(i8` noundef %x) unnamed_addr #0 {
%0 = icmp uge i8 %x, -1
%1 = icmp ule i8 %x, 1
%2 = or i1 %0, %1
call void `@llvm.assume(i1` %2)
ret i8 %x
}
define noundef i8 `@_ZN7example5demo217h5ad29f361a3f5700E(i8` noundef %0) unnamed_addr #0 {
%x = alloca i8, align 1
store i8 %0, ptr %x, align 1
%1 = load i8, ptr %x, align 1, !range !2, !noundef !3
ret i8 %1
}
```
Which feels too different when the original code is essentially identical.
---
Aside: that example is different *after* optimizations too:
```llvm
define noundef i8 `@_ZN7example5demo117h341ef313673d2ee6E(i8` noundef returned %x) unnamed_addr #0 {
%0 = add i8 %x, 1
%1 = icmp ult i8 %0, 3
tail call void `@llvm.assume(i1` %1)
ret i8 %x
}
define noundef i8 `@_ZN7example5demo217h5ad29f361a3f5700E(i8` noundef returned %0) unnamed_addr #1 {
ret i8 %0
}
```
so turning the `Transmute` into a `Discriminant` was arguably just making things worse, so leaving it alone instead -- and thus having less code in rustc -- seems clearly better.
Handle error body in generator layout
Fixes#111468
I feel like making this query return `Option<GeneratorLayout>` might be better but had some issues with that approach
Partially reverts 109612, as after 109993 these aren't actually equivalent any more, and I'm no longer confident this was ever an improvement in the first place.
allow mutating function args through `&raw const`
Fixes https://github.com/rust-lang/rust/issues/111502 by "turning off the sketchy optimization while we figure out if this is ok", like `@JakobDegen` said.
The first commit in this PR removes some suspicious looking logic from the same method, but should have no functional changes, since it doesn't modify the `context` outside of the method. Best reviewed commit by commit.
r? opsem
Verify copies of mutable pointers in 2 stages in ReferencePropagation
Fixes#111422
In the first stage, we mark the copies as reborrows, to be checked later.
In the second stage, we walk the reborrow chains to verify that all stages are fully replacable.
The replacement itself mirrors the check, and iterates through the reborrow chain.
r? ``````@RalfJung``````
cc ``````@JakobDegen``````
Optimize dataflow-const-prop place-tracking infra
Optimization opportunities found while investigating https://github.com/rust-lang/rust/pull/110719
Computing places breadth-first ensures that we create short projections before deep projections, since the former are more likely to be propagated.
The most relevant is the pre-computation of flooded places. Callgrind showed `flood_*` methods and especially `preorder_preinvoke` were especially hot. This PR attempts to pre-compute the set of `ValueIndex` that `preorder_invoke` would visit.
Using this information, we make some `PlaceIndex` inaccessible when they contain no `ValueIndex`, allowing to skip computations for those places.
cc `@jachris` as original author
Rollup of 6 pull requests
Successful merges:
- #104070 (Prevent aborting guard from aborting the process in a forced unwind)
- #109410 (Introduce `AliasKind::Inherent` for inherent associated types)
- #111004 (Migrate `mir_transform` to translatable diagnostics)
- #111118 (Suggest struct when we get colon in fileds in enum)
- #111170 (Diagnostic args are still args if they're documented)
- #111354 (Fix miscompilation when calling default methods on `Future`)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
Disable nrvo mir opt
See #111005 and #110902 . The ICE can definitely be hit on stable, the miscompilation I'm not sure about. The pass makes some pretty sketchy assumptions though, and we should not have it on while that's the case.
I'm not going to work on actually fixing this, it's probably not excessively difficult though.
r? rust-lang/mir-opt
ConstProp into PlaceElem::Index.
Noticed this while looking at keccak output MIR.
This pass aims to replace `ProjectionElem::Index` with `ProjectionElem::ConstantIndex` during ConstProp.
r? `@ghost`
Make `(try_)subst_and_normalize_erasing_regions` take `EarlyBinder`
Changes `subst_and_normalize_erasing_regions` and `try_subst_and_normalize_erasing_regions` to take `EarlyBinder<T>` instead of `T`.
(related to #105779)
This was suggested by `@BoxyUwU` in https://github.com/rust-lang/rust/pull/107753#discussion_r1105828139. After changing `type_of` to return `EarlyBinder`, there were several places where the binder was immediately skipped to call `tcx.subst_and_normalize_erasing_regions`, only for the binder to be reconstructed inside of that method.
r? `@BoxyUwU`
Currently a `{D,Subd}iagnosticMessage` can be created from any type that
impls `Into<String>`. That includes `&str`, `String`, and `Cow<'static,
str>`, which are reasonable. It also includes `&String`, which is pretty
weird, and results in many places making unnecessary allocations for
patterns like this:
```
self.fatal(&format!(...))
```
This creates a string with `format!`, takes a reference, passes the
reference to `fatal`, which does an `into()`, which clones the
reference, doing a second allocation. Two allocations for a single
string, bleh.
This commit changes the `From` impls so that you can only create a
`{D,Subd}iagnosticMessage` from `&str`, `String`, or `Cow<'static,
str>`. This requires changing all the places that currently create one
from a `&String`. Most of these are of the `&format!(...)` form
described above; each one removes an unnecessary static `&`, plus an
allocation when executed. There are also a few places where the existing
use of `&String` was more reasonable; these now just use `clone()` at
the call site.
As well as making the code nicer and more efficient, this is a step
towards possibly using `Cow<'static, str>` in
`{D,Subd}iagnosticMessage::{Str,Eager}`. That would require changing
the `From<&'a str>` impls to `From<&'static str>`, which is doable, but
I'm not yet sure if it's worthwhile.
Box AssertKind
r? `@nnethercote` this feels like your kind of thing
I want to add a new variant to `AssertKind` that needs 3 operands, and that ends up breaking a bunch of size assertions. So... what if we go the opposite direction first; shrinking `AssertKind` by boxing it?
Don't validate constants in const propagation
Validation is neither necessary nor desirable.
The constant validation is already omitted at mir-opt-level >= 3, so there there are not changes in MIR test output (the propagation of invalid constants is covered by an existing test in tests/mir-opt/const_prop/invalid_constant.rs).
Previously, when borrowck failed it would taint all promoteds within the MIR
body. An attempt to evaluated the promoteds would subsequently fail with
spurious "note: erroneous constant used". For example:
```console
...
note: erroneous constant used
--> tests/ui/borrowck/tainted-promoteds.rs:7:9
|
7 | a = &0 * &1 * &2 * &3;
| ^^
note: erroneous constant used
--> tests/ui/borrowck/tainted-promoteds.rs:7:14
|
7 | a = &0 * &1 * &2 * &3;
| ^^
note: erroneous constant used
--> tests/ui/borrowck/tainted-promoteds.rs:7:19
|
7 | a = &0 * &1 * &2 * &3;
| ^^
note: erroneous constant used
--> tests/ui/borrowck/tainted-promoteds.rs:7:24
|
7 | a = &0 * &1 * &2 * &3;
| ^^
```
Borrowck failure doesn't indicate that there is anything wrong with
promoteds. Leave them untainted.
Validation is neither necessary nor desirable.
The validation is already omitted at mir-opt-level >= 3, so there there
are not changes in MIR test output (the propagation of invalid constants
is covered by an existing test in tests/mir-opt/const_prop/invalid_constant.rs).
