Implement a generic Destination Propagation optimization on MIR
This takes the work that was originally started by `@eddyb` in https://github.com/rust-lang/rust/pull/47954, and then explored by me in https://github.com/rust-lang/rust/pull/71003, and implements it in a general (ie. not limited to acyclic CFGs) and dataflow-driven way (so that no additional infrastructure in rustc is needed).
The pass is configured to run at `mir-opt-level=2` and higher only. To enable it by default, some followup work on it is still needed:
* Performance needs to be evaluated. I did some light optimization work and tested against `tuple-stress`, which caused trouble in my last attempt, but didn't go much in depth here.
* We can also enable the pass only at `opt-level=2` and higher, if it is too slow to run in debug mode, but fine when optimizations run anyways.
* Debuginfo needs to be fixed after locals are merged. I did not look into what is required for this.
* Live ranges of locals (aka `StorageLive` and `StorageDead`) are currently deleted. We either need to decide that this is fine, or if not, merge the variable's live ranges (or remove these statements entirely – https://github.com/rust-lang/rust/issues/68622).
Some benchmarks of the pass were done in https://github.com/rust-lang/rust/pull/72635.
As a side effect, we now represent most promoteds as `ConstValue::Scalar` again. This is useful because all implict promoteds are just references anyway and most explicit promoteds are numeric arguments to `asm!` or SIMD instructions.
Some MIR statements and terminators have an (undocumented...) invariant
that some of their input and outputs must not overlap. This records
conflicts between locals used in these positions.
compare generic constants using `AbstractConst`s
This is a MVP of rust-lang/compiler-team#340. The changes in this PR should only be relevant if `feature(const_evaluatable_checked)` is enabled.
~~currently based on top of #76559, so blocked on that.~~
r? `@oli-obk` cc `@varkor` `@eddyb`
Issue 72408 nested closures exponential
This fixes#72408.
Nested closures were resulting in exponential compilation time.
This PR is enhancing asymptotic complexity, but also increasing the constant, so I would love to see perf run results.
[mir-opt] Disable the `ConsideredEqual` logic in SimplifyBranchSame opt
The logic is currently broken and we need to disable it to fix a beta
regression (see #76803)
r? `@oli-obk`
Mostly to fix ui/issues/issue-37311-type-length-limit/issue-37311.rs.
Most parts of the compiler can handle deeply nested types with a lot
of duplicates just fine, but some parts still attempt to naively
traverse type tree.
Before such problems were caught by type length limit check,
but now these places will have to be changed to handle
duplicated types gracefully.
move guaranteed{ne,eq} implementation to compile-time machine
Currently, Miri needs a special hack to avoid using the core engine implementation of these intrinsics. That seems silly, so let's move them to the CTFE machine, which is the only machine that wants to use them.
I also added a reference to https://github.com/rust-lang/rust/issues/73722 as a warning to anyone who wants to adjust `guaranteed_eq`.
Make graphviz font configurable
Alternative to PR #76776.
To change the graphviz output to use an alternative `fontname` value,
add a command line option like: `rustc --graphviz-font=monospace`.
r? @ecstatic-morse
Strip a single leading tab when rendering dataflow diffs
The `fmt_diff_with` formatter uses a tab to separate additions from subtractions. Strip it when rendering those diffs on separate lines.
r? @Mark-Simulacrum (since you're speedy)
Alternative to PR ##76776.
To change the graphviz output to use an alternative `fontname` value,
add a command line option like: `rustc --graphviz-font=monospace`.
Introduce a PartitioningCx struct
This contains all the data used by the partitioning algorithm and allows that data to be used at each stage of the partitioning. This is useful for other approaches to partitioning which may want different pieces of the data available at each step.
cc @rust-lang/wg-incr-comp
When removing unwinds to no-op blocks and folding jumps to no-op blocks,
remove the unwind target first. Otherwise we cannot determine if target
has been already folded or not.
Previous implementation incorrectly assumed that all resume targets had
been folded already, occasionally resulting in an underflow:
remove_noop_landing_pads: removed 18446744073709551613 jumps and 3 landing pads
Note when a a move/borrow error is caused by a deref coercion
Fixes#73268
When a deref coercion occurs, we may end up with a move error if the
base value has been partially moved out of. However, we do not indicate
anywhere that a deref coercion is occuring, resulting in an error
message with a confusing span.
This PR adds an explicit note to move errors when a deref coercion is
involved. We mention the name of the type that the deref-coercion
resolved to, as well as the `Deref::Target` associated type being used.
use sort_unstable to sort primitive types
It's not important to retain original order if we have &[1, 1, 2, 3] for example.
clippy::stable_sort_primitive
inliner: Emit storage markers for introduced arg temporaries
When introducing argument temporaries during inlining, emit storage
marker statements just before the assignment and in the beginning of
the return block.
This ensures that such temporaries will not be considered live across
yield points after inlining inside a generator.
Fixes#71793.
NRVO: Allow occurrences of the return place in var debug info
The non-use occurrence of the return place in var debug info does not
currently inhibit NRVO optimization, but it will fail assertion in
`visit_place` when optimization is performed.
Relax assertion check to allow the return place in var debug info.
This case might be impossible to hit in optimization pipelines as of
now, but can be encountered in customized mir-opt-level=2 pipeline with
copy propagation disabled. For example in:
```rust
pub fn b(s: String) -> String {
a(s)
}
#[inline]
pub fn a(s: String) -> String {
let x = s;
let y = x;
y
}
```
Fixes#73268
When a deref coercion occurs, we may end up with a move error if the
base value has been partially moved out of. However, we do not indicate
anywhere that a deref coercion is occuring, resulting in an error
message with a confusing span.
This PR adds an explicit note to move errors when a deref coercion is
involved. We mention the name of the type that the deref-coercion
resolved to, as well as the `Deref::Target` associated type being used.
Validate removal of AscribeUserType, FakeRead, and Shallow borrow
Those statements are removed by CleanupNonCodegenStatements pass
in drop lowering phase, and should not occur afterwards.
Add CONST_ITEM_MUTATION lint
Fixes#74053Fixes#55721
This PR adds a new lint `CONST_ITEM_MUTATION`.
Given an item `const FOO: SomeType = ..`, this lint fires on:
* Attempting to write directly to a field (`FOO.field = some_val`) or
array entry (`FOO.array_field[0] = val`)
* Taking a mutable reference to the `const` item (`&mut FOO`), including
through an autoderef `FOO.some_mut_self_method()`
The lint message explains that since each use of a constant creates a
new temporary, the original `const` item will not be modified.
VS code graphviz extensions use d3-graphviz, which supports `Courier`
fontname but does not support `monospace`. This caused graphs to render
poorly because the text sizes were wrong.
Many developers use a dark theme with editors and IDEs, but this
typically doesn't extend to graphviz output.
When I bring up a MIR graphviz document, the white background is
strikingly bright. This new option changes the colors used for graphviz
output to work better in dark-themed UIs.
Do not promote &mut of a non-ZST ever
Since ~pre-1.0~ 1.36, we have accepted code like this:
```rust
static mut TEST: &'static mut [i32] = {
let x = &mut [1,2,3];
x
};
```
I tracked it back to https://github.com/rust-lang/rust/pull/21744, but unfortunately could not find any discussion or RFC that would explain why we thought this was a good idea. And it's not, it breaks all sorts of things -- see https://github.com/rust-lang/rust/issues/75556.
To fix https://github.com/rust-lang/rust/issues/75556, we have to stop promoting non-ZST mutable references no matter the context, which is what this PR does. It's a breaking change.
Notice that this still works, since it does not rely on promotion:
```rust
static mut TEST: &'static mut [i32] = &mut [0,1,2];
```
Cc `@rust-lang/wg-const-eval`
Support dataflow problems on arbitrary lattices
This PR implements last of the proposed extensions I mentioned in the design meeting for the original dataflow refactor. It extends the current dataflow framework to work with arbitrary lattices, not just `BitSet`s. This is a prerequisite for dataflow-enabled MIR const-propagation. Personally, I am skeptical of the usefulness of doing const-propagation pre-monomorphization, since many useful constants only become known after monomorphization (e.g. `size_of::<T>()`) and users have a natural tendency to hand-optimize the rest. It's probably worth exprimenting with, however, and others have shown interest cc `@rust-lang/wg-mir-opt.`
The `Idx` associated type is moved from `AnalysisDomain` to `GenKillAnalysis` and replaced with an associated `Domain` type that must implement `JoinSemiLattice`. Like before, each `Analysis` defines the "bottom value" for its domain, but can no longer override the dataflow join operator. Analyses that want to use set intersection must now use the `lattice::Dual` newtype. `GenKillAnalysis` impls have an additional requirement that `Self::Domain: BorrowMut<BitSet<Self::Idx>>`, which effectively means that they must use `BitSet<Self::Idx>` or `lattice::Dual<BitSet<Self::Idx>>` as their domain.
Most of these changes were mechanical. However, because a `Domain` is no longer always a powerset of some index type, we can no longer use an `IndexVec<BasicBlock, GenKillSet<A::Idx>>>` to store cached block transfer functions. Instead, we use a boxed `dyn Fn` trait object. I discuss a few alternatives to the current approach in a commit message.
The majority of new lines of code are to preserve existing Graphviz diagrams for those unlucky enough to have to debug dataflow analyses. I find these diagrams incredibly useful when things are going wrong and considered regressing them unacceptable, especially the pretty-printing of `MovePathIndex`s, which are used in many dataflow analyses. This required a parallel `fmt` trait used only for printing dataflow domains, as well as a refactoring of the `graphviz` module now that we cannot expect the domain to be a `BitSet`. Some features did have to be removed, such as the gen/kill display mode (which I didn't use but existed to mirror the output of the old dataflow framework) and line wrapping. Since I had to rewrite much of it anyway, I took the opportunity to switch to a `Visitor` for printing dataflow state diffs instead of using cursors, which are error prone for code that must be generic over both forward and backward analyses. As a side-effect of this change, we no longer have quadratic behavior when writing graphviz diagrams for backward dataflow analyses.
r? `@pnkfelix`
Fixes#74053Fixes#55721
This PR adds a new lint `CONST_ITEM_MUTATION`.
Given an item `const FOO: SomeType = ..`, this lint fires on:
* Attempting to write directly to a field (`FOO.field = some_val`) or
array entry (`FOO.array_field[0] = val`)
* Taking a mutable reference to the `const` item (`&mut FOO`), including
through an autoderef `FOO.some_mut_self_method()`
The lint message explains that since each use of a constant creates a
new temporary, the original `const` item will not be modified.
* Adds missing "tail" spans (spans that continue beyond the end of
overlapping spans)
* Adds a caret to highlight empty spans associated with MIR elements
that have a position, but otherwise would not be visible.
* Adds visual pointing brackets at the beginning and end of each span
inliner: Check for codegen fn attributes compatibility
* Check for target features compatibility
* Check for no_sanitize attribute compatibility
Fixes#76259.
This commit adjusts MIR shim construction so that substitutions are
applied to function pointer shims during construction, rather than
during codegen (as determined by `substs_for_mir_body`) - as
substitutions will no longer occur during codegen, function pointer
shims can now be polymorphic without incurring double substitutions.
Signed-off-by: David Wood <david@davidtw.co>
Tools, tests, and experimenting with MIR-derived coverage counters
Leverages the new mir_dump output file in HTML+CSS (from #76074) to visualize coverage code regions
and the MIR features that they came from (including overlapping spans).
See example below.
The `run-make-fulldeps/instrument-coverage` test has been refactored to maximize test coverage and reduce code duplication. The new tests support testing with and without `-Clink-dead-code`, so Rust coverage can be tested on MSVC (which, currently, only works with `link-dead-code` _disabled_).
New tests validate coverage region generation and coverage reports with multiple counters per function. Starting with a simple `if-else` branch tests, coverage tests for each additional syntax type can be added by simply dropping in a new Rust sample program.
Includes a basic, MIR-block-based implementation of coverage injection,
available via `-Zexperimental-coverage`. This implementation has known
flaws and omissions, but is simple enough to validate the new tools and
tests.
The existing `-Zinstrument-coverage` option currently enables
function-level coverage only, which at least appears to generate
accurate coverage reports at that level.
Experimental coverage is not accurate at this time. When branch coverage
works as intended, the `-Zexperimental-coverage` option should be
removed.
This PR replaces the bulk of PR #75828, with the remaining parts of
that PR distributed among other separate and indentpent PRs.
This PR depends on two of those other PRs: #76002, #76003 and #76074
Rust compiler MCP rust-lang/compiler-team#278
Relevant issue: #34701 - Implement support for LLVMs code coverage
instrumentation
r? @tmandry
FYI: @wesleywiser
The non-use occurrence of the return place in var debug info does not
currently inhibit NRVO optimization, but it will fail assertion in
`visit_place` when optimization is performed.
Relax assertion check to allow the return place in var debug info.
This case might be impossible to hit in optimization pipelines as of
now, but can be encountered in customized mir-opt-level=2 pipeline with
copy propagation disabled. For example in:
```
pub fn b(s: String) -> String {
a(s)
}
#[inline]
pub fn a(s: String) -> String {
let x = s;
let y = x;
y
}
```
diagnostics: shorten paths of unique symbols
This is a step towards implementing a fix for #50310, and continuation of the discussion in [Pre-RFC: Nicer Types In Diagnostics - compiler - Rust Internals](https://internals.rust-lang.org/t/pre-rfc-nicer-types-in-diagnostics/11139). Impressed upon me from previous discussion in #21934 that an RFC for this is not needed, and I should just come up with code.
The recent improvements to `use` suggestions that I've contributed have given rise to this implementation. Contrary to previous suggestions, it's rather simple logic, and I believe it only reduces the amount of cognitive load that a developer would need when reading type errors.
-----
If a symbol name can only be imported from one place, and as long as it was not glob-imported anywhere in the current crate, we can trim its printed path to the last component.
This has wide implications on error messages with types, for example, shortening `std::vec::Vec` to just `Vec`, as long as there is no other `Vec` importable from anywhere.
When introducing argument temporaries during inlining, emit storage
marker statements just before the assignment and in the beginning of
the return block.
This ensures that such temporaries will not be considered live across
yield points after inlining inside a generator.
Adds a new mir_dump output file in HTML/CSS to visualize code regions
and the MIR features that they came from (including overlapping spans).
See example below:
Includes a basic, MIR-block-based implementation of coverage injection,
available via `-Zexperimental-coverage`. This implementation has known
flaws and omissions, but is simple enough to validate the new tools and
tests.
The existing `-Zinstrument-coverage` option currently enables
function-level coverage only, which at least appears to generate
accurate coverage reports at that level.
Experimental coverage is not accurate at this time. When branch coverage
works as intended, the `-Zexperimental-coverage` option should be
removed.
This PR replaces the bulk of PR #75828, with the remaining parts of
that PR distributed among other separate and indentpent PRs.
This PR depends on three of those other PRs: #76000, #76002, and
Rust compiler MCP rust-lang/compiler-team#278
Relevant issue: #34701 - Implement support for LLVMs code coverage
instrumentation
If a symbol name can only be imported from one place for a type, and
as long as it was not glob-imported anywhere in the current crate, we
can trim its printed path and print only the name.
This has wide implications on error messages with types, for example,
shortening `std::vec::Vec` to just `Vec`, as long as there is no other
`Vec` importable anywhere.
This adds a new '-Z trim-diagnostic-paths=false' option to control this
feature.
On the good path, with no diagnosis printed, we should try to avoid
issuing this query, so we need to prevent trimmed_def_paths query on
several cases.
This change also relies on a previous commit that differentiates
between `Debug` and `Display` on various rustc types, where the latter
is trimmed and presented to the user and the former is not.
The HIR Id trick is insufficient to prevent query cycles when optimizing
generators, since merely requesting a layout of a generator also
computes its `optimized_mir`.
Make no attempts to inline functions into generators within the same
crate to avoid query cycles.
Similar to `-Z dump-mir-graphviz`, this adds the option to write
HTML+CSS files that allow users to analyze the spans associated with MIR
elements (by individual statement, just terminator, or overall basic
block).
This PR was split out from PR #76004, and exposes an API for spanview
HTML+CSS files that is also used to analyze code regions chosen for
coverage instrumentation (in a follow-on PR).
Rust compiler MCP rust-lang/compiler-team#278
Relevant issue: #34701 - Implement support for LLVMs code coverage
instrumentation
Fix `-Z instrument-coverage` on MSVC
Found that `-C link-dead-code` (which was enabled automatically
under `-Z instrument-coverage`) was causing the linking error that
resulted in segmentation faults in coverage instrumented binaries. Link
dead code is now disabled under MSVC, allowing `-Z instrument-coverage`
to be enabled under MSVC for the first time.
More details are included in Issue #76038 .
Note this PR makes it possible to support `Z instrument-coverage` but
does not enable instrument coverage for MSVC in existing tests. It will be
enabled in another PR to follow this one (both PRs coming from original
PR #75828).
r? @tmandry
FYI: @wesleywiser
Found that -C link-dead-code (which was enabled automatically
under -Z instrument-coverage) was causing the linking error that
resulted in segmentation faults in coverage instrumented binaries. Link
dead code is now disabled under MSVC, allowing `-Z instrument-coverage`
to be enabled under MSVC for the first time.
More details are included in Issue #76038.
(This PR was broken out from PR #75828)
I've tried a few ways of implementing this, but each fell short.
Adding an auxiliary `_Idx` associated type to `Analysis` that defaults
to `!` but is overridden in the blanket impl of `Analysis` for `A:
GenKillAnalysis` to `A::Idx` seems promising, but the trait solver is
unable to prove equivalence between `A::Idx` and `A::_Idx` within the
overridden version of `into_engine`. Without full-featured
specialization, removing `into_engine` or splitting it into a different
trait would have a significant ergonomic penalty.
Alternatively, we could erase the index type and store a
`GenKillSet<u32>` as well as a function pointer for transmuting between
`&mut A::Domain` and `&mut BitSet<u32>` in the hopes that LLVM can
devirtualize a simple function pointer better than the boxed closure.
However, this is brittle, requires `unsafe` code, and doesn't work for
index types that aren't the same size as a `u32` (e.g. `usize`) since
`GenKillSet` stores a `HybridBitSet`, which may be a `Vec<I>`. Perhaps
safe transmute could help here?
This commit removes the obsolete printer and replaces all uses of it
with `FmtPrinter`. Of the replaced uses, all but one use was in `debug!`
logging, two cases were notable:
- `MonoItem::to_string` is used in `-Z print-mono-items` and therefore
affects the output of all codegen-units tests.
- `DefPathBasedNames` was used in `librustc_codegen_llvm/type_of.rs`
with `LLVMStructCreateNamed` and that'll now get different values, but
this should result in no functional change.
Signed-off-by: David Wood <david@davidtw.co>
