Introduce `ChunkedBitSet` and use it for some dataflow analyses.
This reduces peak memory usage significantly for some programs with very
large functions.
r? `@ghost`
This reduces peak memory usage significantly for some programs with very
large functions, such as:
- `keccak`, `unicode_normalization`, and `match-stress-enum`, from
the `rustc-perf` benchmark suite;
- `http-0.2.6` from crates.io.
The new type is used in the analyses where the bitsets can get huge
(e.g. 10s of thousands of bits): `MaybeInitializedPlaces`,
`MaybeUninitializedPlaces`, and `EverInitializedPlaces`.
Some refactoring was required in `rustc_mir_dataflow`. All existing
analysis domains are either `BitSet` or a trivial wrapper around
`BitSet`, and access in a few places is done via `Borrow<BitSet>` or
`BorrowMut<BitSet>`. Now that some of these domains are `ClusterBitSet`,
that no longer works. So this commit replaces the `Borrow`/`BorrowMut`
usage with a new trait `BitSetExt` containing the needed bitset
operations. The impls just forward these to the underlying bitset type.
This required fiddling with trait bounds in a few places.
The commit also:
- Moves `static_assert_size` from `rustc_data_structures` to
`rustc_index` so it can be used in the latter; the former now
re-exports it so existing users are unaffected.
- Factors out some common "clear excess bits in the final word"
functionality in `bit_set.rs`.
- Uses `fill` in a few places instead of loops.
Adopt let else in more places
Continuation of #89933, #91018, #91481, #93046, #93590, #94011.
I have extended my clippy lint to also recognize tuple passing and match statements. The diff caused by fixing it is way above 1 thousand lines. Thus, I split it up into multiple pull requests to make reviewing easier. This is the biggest of these PRs and handles the changes outside of rustdoc, rustc_typeck, rustc_const_eval, rustc_trait_selection, which were handled in PRs #94139, #94142, #94143, #94144.
Specifically, rename the `Const` struct as `ConstS` and re-introduce `Const` as
this:
```
pub struct Const<'tcx>(&'tcx Interned<ConstS>);
```
This now matches `Ty` and `Predicate` more closely, including using
pointer-based `eq` and `hash`.
Notable changes:
- `mk_const` now takes a `ConstS`.
- `Const` was copy, despite being 48 bytes. Now `ConstS` is not, so need a
we need separate arena for it, because we can't use the `Dropless` one any
more.
- Many `&'tcx Const<'tcx>`/`&Const<'tcx>` to `Const<'tcx>` changes
- Many `ct.ty` to `ct.ty()` and `ct.val` to `ct.val()` changes.
- Lots of tedious sigil fiddling.
The variant names are exported, so we can use them directly (possibly
with a `ty::` qualifier). Lots of places already do this, this commit
just increases consistency.
Specifically, change `Region` from this:
```
pub type Region<'tcx> = &'tcx RegionKind;
```
to this:
```
pub struct Region<'tcx>(&'tcx Interned<RegionKind>);
```
This now matches `Ty` and `Predicate` more closely.
Things to note
- Regions have always been interned, but we haven't been using pointer-based
`Eq` and `Hash`. This is now happening.
- I chose to impl `Deref` for `Region` because it makes pattern matching a lot
nicer, and `Region` can be viewed as just a smart wrapper for `RegionKind`.
- Various methods are moved from `RegionKind` to `Region`.
- There is a lot of tedious sigil changes.
- A couple of types like `HighlightBuilder`, `RegionHighlightMode` now have a
`'tcx` lifetime because they hold a `Ty<'tcx>`, so they can call `mk_region`.
- A couple of test outputs change slightly, I'm not sure why, but the new
outputs are a little better.
Specifically, change `Ty` from this:
```
pub type Ty<'tcx> = &'tcx TyS<'tcx>;
```
to this
```
pub struct Ty<'tcx>(Interned<'tcx, TyS<'tcx>>);
```
There are two benefits to this.
- It's now a first class type, so we can define methods on it. This
means we can move a lot of methods away from `TyS`, leaving `TyS` as a
barely-used type, which is appropriate given that it's not meant to
be used directly.
- The uniqueness requirement is now explicit, via the `Interned` type.
E.g. the pointer-based `Eq` and `Hash` comes from `Interned`, rather
than via `TyS`, which wasn't obvious at all.
Much of this commit is boring churn. The interesting changes are in
these files:
- compiler/rustc_middle/src/arena.rs
- compiler/rustc_middle/src/mir/visit.rs
- compiler/rustc_middle/src/ty/context.rs
- compiler/rustc_middle/src/ty/mod.rs
Specifically:
- Most mentions of `TyS` are removed. It's very much a dumb struct now;
`Ty` has all the smarts.
- `TyS` now has `crate` visibility instead of `pub`.
- `TyS::make_for_test` is removed in favour of the static `BOOL_TY`,
which just works better with the new structure.
- The `Eq`/`Ord`/`Hash` impls are removed from `TyS`. `Interned`s impls
of `Eq`/`Hash` now suffice. `Ord` is now partly on `Interned`
(pointer-based, for the `Equal` case) and partly on `TyS`
(contents-based, for the other cases).
- There are many tedious sigil adjustments, i.e. adding or removing `*`
or `&`. They seem to be unavoidable.
Inherit lifetimes for async fn instead of duplicating them.
The current desugaring of `async fn foo<'a>(&usize) -> &u8` is equivalent to
```rust
fn foo<'a, '0>(&'0 usize) -> foo<'static, 'static>::Opaque<'a, '0, '_>;
type foo<'_a, '_0>::Opaque<'a, '0, '1> = impl Future<Output = &'1 u8>;
```
following the RPIT model.
Duplicating all the inherited lifetime parameters and setting the inherited version to `'static` makes lowering more complex and causes issues like #61949. This PR removes the duplication of inherited lifetimes to directly use
```rust
fn foo<'a, '0>(&'0 usize) -> foo<'a, '0>::Opaque<'_>;
type foo<'a, '0>::Opaque<'1> = impl Future<Output = &'1 u8>;
```
following the TAIT model.
Fixes https://github.com/rust-lang/rust/issues/61949
Currently, any higher-ranked region errors involving opaque types
fall back to a generic "higher-ranked subtype error" message when
run under NLL. This PR adds better error message handling for this
case, giving us the same kinds of error messages that we currently
get without NLL:
```
error: implementation of `MyTrait` is not general enough
--> $DIR/opaque-hrtb.rs:12:13
|
LL | fn foo() -> impl for<'a> MyTrait<&'a str> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ implementation of `MyTrait` is not general enough
|
= note: `impl MyTrait<&'2 str>` must implement `MyTrait<&'1 str>`, for any lifetime `'1`...
= note: ...but it actually implements `MyTrait<&'2 str>`, for some specific lifetime `'2`
error: aborting due to previous error
```
To accomplish this, several different refactoring needed to be made:
* We now have a dedicated `InstantiateOpaqueType` struct which
implements `TypeOp`. This is used to invoke `instantiate_opaque_types`
during MIR type checking.
* `TypeOp` is refactored to pass around a `MirBorrowckCtxt`, which is
needed to report opaque type region errors.
* We no longer assume that all `TypeOp`s correspond to canonicalized
queries. This allows us to properly handle opaque type instantiation
(which does not occur in a query) as a `TypeOp`.
A new `ErrorInfo` associated type is used to determine what
additional information is used during higher-ranked region error
handling.
* The body of `try_extract_error_from_fulfill_cx`
has been moved out to a new function `try_extract_error_from_region_constraints`.
This allows us to re-use the same error reporting code between
canonicalized queries (which can extract region constraints directly
from a fresh `InferCtxt`) and opaque type handling (which needs to take
region constraints from the pre-existing `InferCtxt` that we use
throughout MIR borrow checking).
Lazy type-alias-impl-trait
Previously opaque types were processed by
1. replacing all mentions of them with inference variables
2. memorizing these inference variables in a side-table
3. at the end of typeck, resolve the inference variables in the side table and use the resolved type as the hidden type of the opaque type
This worked okayish for `impl Trait` in return position, but required lots of roundabout type inference hacks and processing.
This PR instead stops this process of replacing opaque types with inference variables, and just keeps the opaque types around.
Whenever an opaque type `O` is compared with another type `T`, we make the comparison succeed and record `T` as the hidden type. If `O` is compared to `U` while there is a recorded hidden type for it, we grab the recorded type (`T`) and compare that against `U`. This makes implementing
* https://github.com/rust-lang/rfcs/pull/2515
much simpler (previous attempts on the inference based scheme were very prone to ICEs and general misbehaviour that was not explainable except by random implementation defined oddities).
r? `@nikomatsakis`
fixes#93411fixes#88236
[borrowck] Fix help on mutating &self in async fns
Previously, when rustc was provided an async function that tried to
mutate through a shared reference to an implicit self (as shown in the
ui test), rustc would suggest modifying the parameter signature
to `&mut` + the fully qualified name of the ty (in the case of the repro
`S`). If a user modified their code to match the suggestion, the
compiler would not accept it.
This commit modifies the suggestion so that when rustc is provided the
ui test that is also attached in this commit, it suggests (correctly)
`&mut self`. We try to be careful about distinguishing between implicit
and explicit self annotations, since the latter seem to be handled
correctly already.
This is my first PR here so I'm pretty sure I probably missed something/could use better terminology. I also didn't try to make the match exhaustive since implicit self is the only real special case that I need to handle (that I'm aware of), and I'm pretty sure there's a cleaner way to do this so any advice would be greatly appreciated! (I'm also not terribly confident about how I wrote the ui tests)
here is your cc as requested `@compiler-errors`
This is an attempt to fix#93093
by using an opaque type obligation to bubble up comparisons between opaque types and other types
Also uses proper obligation causes so that the body id works, because out of some reason nll uses body ids for logic instead of just diagnostics.
Store a `Symbol` instead of an `Ident` in `AssocItem`
This is the same idea as #92533, but for `AssocItem` instead
of `VariantDef`/`FieldDef`.
With this change, we no longer have any uses of
`#[stable_hasher(project(...))]`
Remove ordering traits from `OutlivesConstraint`
In two cases where this ordering was used, I've replaced the sorting to use a key that does not rely on `DefId` being `Ord`. This is part of #90317. If I understand correctly, whether this is correct depends on whether the `RegionVid`s are tracked during incremental compilation. But I might be mistaken in this approach. cc `@cjgillot`
Previously, when rustc was provided an async function that tried to
mutate through a shared reference to an implicit self (as shown in the
ui test), rustc would suggest modifying the parameter signature
to `&mut` + the fully qualified name of the ty (in the case of the repro
`S`). If a user modified their code to match the suggestion, the
compiler would not accept it.
This commit modifies the suggestion so that when rustc is provided the
ui test that is also attached in this commit, it suggests (correctly)
`&mut self`. We try to be careful about distinguishing between implicit
and explicit self annotations, since the latter seem to be handled
correctly already.
Fixesrust-lang/rust#93093
Ensure that early-bound function lifetimes are always 'local'
During borrowchecking, we treat any free (early-bound) regions on
the 'defining type' as `RegionClassification::External`. According
to the doc comments, we should only have 'external' regions when
checking a closure/generator.
However, a plain function can also have some if its regions
be considered 'early bound' - this occurs when the region is
constrained by an argument, appears in a `where` clause, or
in an opaque type. This was causing us to incorrectly mark these
regions as 'external', which caused some diagnostic code
to act as if we were referring to a 'parent' region from inside
a closure.
This PR marks all instantiated region variables as 'local'
when we're borrow-checking something other than a
closure/generator/inline-const.
Update some rustc dependencies to deduplicate them
This PR updates `rand` and `itertools` in rustc (not the whole workspace) in order to deduplicate them (and hopefully slightly improve compile times).
~~Currently, `object` is still duplicated, but https://github.com/rust-lang/thorin/pull/15 and updating `thorin` in the future will remove the use of version 0.27.~~ Update: Thorin 0.2 has now been released, and this PR updates `rustc_codegen_ssa` to use it and deduplicate the `object` crate.
There's a final tiny rustc dependency, `cfg-if`, which will be left: as both versions 0.1.x and 1.0 looked to be heavily depended on, they will require a few cascading updates to be removed.
This is the same idea as #92533, but for `AssocItem` instead
of `VariantDef`/`FieldDef`.
With this change, we no longer have any uses of
`#[stable_hasher(project(...))]`
In two cases where this ordering was used, I've replaced the sorting
to use a key that does not include DefId. I'm not sure this is correct
in terms of our goals from #90317, or otherwise.
ProjectionPredicate should be able to handle both associated types and consts so this adds the
first step of that. It mainly just pipes types all the way down, not entirely sure how to handle
consts, but hopefully that'll come with time.
Remove deprecated LLVM-style inline assembly
The `llvm_asm!` was deprecated back in #87590 1.56.0, with intention to remove
it once `asm!` was stabilized, which already happened in #91728 1.59.0. Now it
is time to remove `llvm_asm!` to avoid continued maintenance cost.
Closes#70173.
Closes#92794.
Closes#87612.
Closes#82065.
cc `@rust-lang/wg-inline-asm`
r? `@Amanieu`
Add `#[track_caller]` to `mirbug`
When a "'no errors encountered even though `delay_span_bug` issued" error results from the `mirbug` function, the file location information points to the `mirbug` function itself, rather than its caller. This doesn't make sense, since the caller is the real source of the bug. Adding `#[track_caller]` will produce diagnostics that are more useful to anyone fixing the ICE.
Closure capture cleanup & refactor
Follow up of #89648
Each commit is self-contained and the rationale/changes are documented in the commit message, so it's advisable to review commit by commit.
The code is significantly cleaner (at least IMO), but that could have some perf implication, so I'd suggest a perf run.
r? `@wesleywiser`
cc `@arora-aman`
The field is also renamed from `ident` to `name. In most cases,
we don't actually need the `Span`. A new `ident` method is added
to `VariantDef` and `FieldDef`, which constructs the full `Ident`
using `tcx.def_ident_span()`. This method is used in the cases
where we actually need an `Ident`.
This makes incremental compilation properly track changes
to the `Span`, without all of the invalidations caused by storing
a `Span` directly via an `Ident`.
Region info is completely unnecessary for upvar capture kind computation
and is only needed to create the final upvar tuple ty. Doing so makes
creation of UpvarCapture very cheap and expose further cleanup opportunity.
Remove `NullOp::Box`
Follow up of #89030 and MCP rust-lang/compiler-team#460.
~1 month later nothing seems to be broken, apart from a small regression that #89332 (1aac85bb716c09304b313d69d30d74fe7e8e1a8e) shows could be regained by remvoing the diverging path, so it shall be safe to continue and remove `NullOp::Box` completely.
r? `@jonas-schievink`
`@rustbot` label T-compiler
During borrowchecking, we treat any free (early-bound) regions on
the 'defining type' as `RegionClassification::External`. According
to the doc comments, we should only have 'external' regions when
checking a closure/generator.
However, a plain function can also have some if its regions
be considered 'early bound' - this occurs when the region is
constrained by an argument, appears in a `where` clause, or
in an opaque type. This was causing us to incorrectly mark these
regions as 'external', which caused some diagnostic code
to act as if we were referring to a 'parent' region from inside
a closure.
This PR marks all instantiated region variables as 'local'
when we're borrow-checking something other than a
closure/generator/inline-const.
Region inference contains several bitsets which are filled with large intervals
representing liveness. These can cause excessive memory usage, and are
relatively slow when growing to large sizes compared to the IntervalSet.
Instead of special-casing mutable pointers/references, we
now support general generic types (currently, we handle
`ty::Ref`, `ty::RawPtr`, and `ty::Adt`)
When a `ty::Adt` is involved, we show an additional note
explaining which of the type's generic parameters is
invariant (e.g. the `T` in `Cell<T>`). Currently, we don't
explain *why* a particular generic parameter ends up becoming
invariant. In the general case, this could require printing
a long 'backtrace' of types, so doing this would be
more suitable for a follow-up PR.
We still only handle the case where our variance switches
to `ty::Invariant`.
The `AggregateKind` enum ends up in the final mir `Body`. Currently,
any changes to `AdtDef` (regardless of how significant they are)
will legitimately cause the overall result of `optimized_mir` to change,
invalidating any codegen re-use involving that mir.
This will get worse once we start hashing the `Span` inside `FieldDef`
(which is itself contained in `AdtDef`).
To try to reduce these kinds of invalidations, this commit changes
`AggregateKind::Adt` to store just the `DefId`, instead of the full
`AdtDef`. This allows the result of `optimized_mir` to be unchanged
if the `AdtDef` changes in a way that doesn't actually affect any
of the MIR we build.
Remove `SymbolStr`
This was originally proposed in https://github.com/rust-lang/rust/pull/74554#discussion_r466203544. As well as removing the icky `SymbolStr` type, it allows the removal of a lot of `&` and `*` occurrences.
Best reviewed one commit at a time.
r? `@oli-obk`
Point at capture points for non-`'static` reference crossing a `yield` point
```
error[E0759]: `self` has an anonymous lifetime `'_` but it needs to satisfy a `'static` lifetime requirement
--> $DIR/issue-72312.rs:10:24
|
LL | pub async fn start(&self) {
| ^^^^^ this data with an anonymous lifetime `'_`...
...
LL | require_static(async move {
| -------------- ...is required to live as long as `'static` here...
LL | &self;
| ----- ...and is captured here
|
note: `'static` lifetime requirement introduced by this trait bound
--> $DIR/issue-72312.rs:2:22
|
LL | fn require_static<T: 'static>(val: T) -> T {
| ^^^^^^^
error: aborting due to previous error
For more information about this error, try `rustc --explain E0759`.
```
Fix#72312.
In Rust, nesting method calls with both require `&mut` access to `self`
produces a borrow-check error:
error[E0499]: cannot borrow `*self` as mutable more than once at a time
--> src/lib.rs:7:14
|
7 | self.foo(self.bar());
| ---------^^^^^^^^^^-
| | | |
| | | second mutable borrow occurs here
| | first borrow later used by call
| first mutable borrow occurs here
That's because Rust has a left-to-right evaluation order, and the method
receiver is passed first. Thus, the argument to the method cannot then
mutate `self`.
There's an easy solution to this error: just extract a local variable
for the inner argument:
let tmp = self.bar();
self.foo(tmp);
However, the error doesn't give any suggestion of how to solve the
problem. As a result, new users may assume that it's impossible to
express their code correctly and get stuck.
This commit adds a (non-structured) suggestion to extract a local
variable for the inner argument to solve the error. The suggestion uses
heuristics that eliminate most false positives, though there are a few
false negatives (cases where the suggestion should be emitted but is
not). Those other cases can be implemented in a future change.
```
error[E0759]: `self` has an anonymous lifetime `'_` but it needs to satisfy a `'static` lifetime requirement
--> $DIR/issue-72312.rs:10:24
|
LL | pub async fn start(&self) {
| ^^^^^ this data with an anonymous lifetime `'_`...
...
LL | require_static(async move {
| -------------- ...is required to live as long as `'static` here...
LL | &self;
| ----- ...and is captured here
|
note: `'static` lifetime requirement introduced by this trait bound
--> $DIR/issue-72312.rs:2:22
|
LL | fn require_static<T: 'static>(val: T) -> T {
| ^^^^^^^
error: aborting due to previous error
For more information about this error, try `rustc --explain E0759`.
```
Fix#72312.
Fix ICE when `yield`ing in function returning `impl Trait`
Change an assert to a `delay_span_bug` and remove an unwrap, that should fix it.
Fixes#91477
Cleanup: Eliminate ConstnessAnd
This is almost a behaviour-free change and purely a refactoring. "almost" because we appear to be using the wrong ParamEnv somewhere already, and this is now exposed by failing a test using the unstable `~const` feature.
We most definitely need to review all `without_const` and at some point should probably get rid of many of them by using `TraitPredicate` instead of `TraitRef`.
This is a continuation of https://github.com/rust-lang/rust/pull/90274.
r? `@oli-obk`
cc `@spastorino` `@ecstatic-morse`
