NLL: improve inference with flow results, represent regions with bitsets, and more
This PR begins with a number of edits to the NLL code and then includes a large number of smaller refactorings (these refactorings ought not to change behavior). There are a lot of commits here, but each is individually simple. The goal is to land everything up to but not including the changes to how we handle closures, which are conceptually more complex.
The NLL specific changes are as follows (in order of appearance):
**Modify the region inferencer's approach to free regions.** Previously, for each free region (lifetime parameter) `'a`, it would compute the set of other free regions that `'a` outlives (e.g., if we have `where 'a: 'b`, then this set would be `{'a, 'b}`). Then it would mark those free regions as "constants" and report an error if inference tried to extend `'a` to include any other region (e.g., `'c`) that is not in that outlives set. In this way, the value of `'a` would never grow beyond the maximum that could type check. The new approach is to allow `'a` to grow larger. Then, after the fact, we check over the value of `'a` and see what other free regions it is required to outlive, and we check that those outlives relationships are justified by the where clauses in scope etc.
**Modify constraint generation to consider maybe-init.** When we have a "drop-live" variable `x` (i.e., a variable that will be dropped but will not be otherwise used), we now consider whether `x` is "maybe initialized" at that point. If not, then we know the drop is a no-op, and we can allow its regions to be dead. Due to limitations in the fragment code, this currently only works at the level of entire variables.
**Change representation of regions to use a `BitMatrix`.** We used to use a `BTreeSet`, which was rather silly. We now use a MxN matrix of bits, where `M` is the number of variables and `N` is the number of possible elements in each set (size of the CFG + number of free regions).
The remaining commits (starting from
extract the `implied_bounds` code into a helper function ") are all "no-op" refactorings, I believe.
~~One concern I have is with the commit "with -Zverbose, print all details of closure substs"; this commit seems to include some "internal" stuff in the mir-dump files, such as internal interner numbers, that I fear may vary by platform. Annoying. I guess we will see.~~ (I removed this commit.)
As for reviewer, @arielb1 has been reviewing the PRs, and they are certainly welcome to review this one too. But I figured it'd maybe be good to have more people taking a look and being familiar with this code, so I'll "nominate" @pnkfelix .
r? @pnkfelix
In particular, if we see a variable is DROP-LIVE, but it is not
MAYBE-INIT, then we can ignore the drop. This leavess attempt to use
more complex refinements of the idea (e.g., for subpaths or subfields)
to future work.
Rather than declaring some region variables to be constant, and
reporting errors when they would have to change, we instead populate
each free region X with a minimal set of points (the CFG plus end(X)),
and then we let inference do its thing. This may add other `end(Y)`
points into X; we can then check after the fact that indeed `X: Y`
holds.
This requires a bit of "blame" detection to find where the bad
constraint came from: we are currently using a pretty dumb
algorithm. Good place for later expansion.
Fix CopyPropagation regression (2)
Remaining part of MIR copyprop regression by (I think) #45380, which I missed in #45753.
```rust
fn foo(mut x: i32) -> i32 {
let y = x;
x = 123; // `x` is assigned only once in MIR, but cannot be propagated to `y`
y
}
```
So any assignment to an argument cannot be propagated.
create a drop ladder for an array if any value is moved out
r? @arielb1
first commit for fix https://github.com/rust-lang/rust/issues/34708 (note: this still handles the subslice case in a very broken manner)
This simplifies analysis and borrow-checking because liveness at the
resume point can always be simply propagated.
Later on, the "dead" Resumes are removed.
incr.comp.: Some preparatory work for caching more query results.
This PR
* adds and updates some encoding/decoding routines for various query result types so they can be cached later, and
* adds missing `[input]` annotations for a few `DepNode` variants.
The situation around having to explicitly mark dep-nodes/queries as inputs is not really satisfactory. I hope we can find a way of making this more fool-proof in the future.
r? @nikomatsakis
avoid type-live-for-region obligations on dummy nodes
Type-live-for-region obligations on DUMMY_NODE_ID cause an ICE, and it
turns out that in the few cases they are needed, these obligations are not
needed anyway because they are verified elsewhere.
Fixes#46069.
Beta-nominating because this is a regression for our new beta.
r? @nikomatsakis
To handle packed structs with destructors (which you'll think are a rare
case, but the `#[repr(packed)] struct Packed<T>(T);` pattern is
ever-popular, which requires handling packed structs with destructors to
avoid monomorphization-time errors), drops of subfields of packed
structs should drop a local move of the field instead of the original
one.
cc #27060 - this should deal with that issue after codegen of drop glue
is updated.
The new errors need to be changed to future-compatibility warnings, but
I'll rather do a crater run first with them as errors to assess the
impact.
Type-live-for-region obligations on DUMMY_NODE_ID cause an ICE, and it
turns out that in the few cases they are needed, these obligations are not
needed anyway because they are verified elsewhere.
Fixes#46069.
Add a MIR pass to lower 128-bit operators to lang item calls
Runs only with `-Z lower_128bit_ops` since it's not hooked into targets yet.
This isn't really useful on its own, but the declarations for the lang items need to be in the compiler before compiler-builtins can be updated to define them, so this is part 1 of at least 3.
cc https://github.com/rust-lang/rust/issues/45676 @est31 @nagisa
move closure kind, signature into `ClosureSubsts`
Instead of using side-tables, store the closure-kind and signature in the substitutions themselves. This has two key effects:
- It means that the closure's type changes as inference finds out more things, which is very nice.
- As a result, it avoids the need for the `freshen_closure_like` code (though we still use it for generators).
- It avoids cyclic closures calls.
- These were never meant to be supported, precisely because they make a lot of the fancy inference that we do much more complicated. However, due to an oversight, it was previously possible -- if challenging -- to create a setup where a closure *directly* called itself (see e.g. #21410).
We have to see what the effect of this change is, though. Needs a crater run. Marking as [WIP] until that has been assessed.
r? @arielb1
* The overflow-checking shift items need to take a full 128-bit type, since they need to be able to detect idiocy like `1i128 << (1u128 << 127)`
* The unchecked ones just take u32, like the `*_sh?` methods in core
* Because shift-by-anything is allowed, cast into a new local for every shift
Refactor type memory layouts and ABIs, to be more general and easier to optimize.
To combat combinatorial explosion, type layouts are now described through 3 orthogonal properties:
* `Variants` describes the plurality of sum types (where applicable)
* `Single` is for one inhabited/active variant, including all C `struct`s and `union`s
* `Tagged` has its variants discriminated by an integer tag, including C `enum`s
* `NicheFilling` uses otherwise-invalid values ("niches") for all but one of its inhabited variants
* `FieldPlacement` describes the number and memory offsets of fields (if any)
* `Union` has all its fields at offset `0`
* `Array` has offsets that are a multiple of its `stride`; guarantees all fields have one type
* `Arbitrary` records all the field offsets, which can be out-of-order
* `Abi` describes how values of the type should be passed around, including for FFI
* `Uninhabited` corresponds to no values, associated with unreachable control-flow
* `Scalar` is ABI-identical to its only integer/floating-point/pointer "scalar component"
* `ScalarPair` has two "scalar components", but only applies to the Rust ABI
* `Vector` is for SIMD vectors, typically `#[repr(simd)]` `struct`s in Rust
* `Aggregate` has arbitrary contents, including all non-transparent C `struct`s and `union`s
Size optimizations implemented so far:
* ignoring uninhabited variants (i.e. containing uninhabited fields), e.g.:
* `Option<!>` is 0 bytes
* `Result<T, !>` has the same size as `T`
* using arbitrary niches, not just `0`, to represent a data-less variant, e.g.:
* `Option<bool>`, `Option<Option<bool>>`, `Option<Ordering>` are all 1 byte
* `Option<char>` is 4 bytes
* using a range of niches to represent *multiple* data-less variants, e.g.:
* `enum E { A(bool), B, C, D }` is 1 byte
Code generation now takes advantage of `Scalar` and `ScalarPair` to, in more cases, pass around scalar components as immediates instead of indirectly, through pointers into temporary memory, while avoiding LLVM's "first-class aggregates", and there's more untapped potential here.
Closes#44426, fixes#5977, fixes#14540, fixes#43278.
integrate MIR type-checker with NLL inference
This branch refactors NLL type inference so that it uses the MIR type-checker to gather constraints. Along the way, it also refactors how region constraints are gathered in the normal inference context mildly. The new setup is like this:
- What used to be `region_inference` is split into two parts:
- `region_constraints`, which just collects up sets of constraints
- `lexical_region_resolve`, which does the iterative, lexical region resolution
- When `resolve_regions_and_report_errors` is invoked, the inference engine converts the constraints into final values.
- In the MIR type checker, however, we do not invoke this method, but instead periodically take the region constraints and package them up for the NLL solver to use later.
- This allows us to track when and where those constraints were incurred.
- We also remove the central fulfillment context from the MIR type checker, instead instantiating new fulfillment contexts at each point. This allows us to capture the set of obligations that occurred at a particular point, and also to ensure that if the same obligation arises at two points, we will enforce the region constraints at both locations.
- The MIR type checker is also enhanced to instantiate late-bound-regions with fresh variables and handle a few other corner cases that arose.
- I also extracted some of the 'outlives' logic from the regionck, which will be needed later (see future work) to handle the type-outlives relationships.
One concern I have with this branch: since the MIR type checker is used even without the `-Znll` switch, I'm not sure if it will impact performance. One simple fix here would be to only enable the MIR type-checker if debug-assertions are enabled, since it just serves to validate the MIR. Longer term I hope to address this by improving the interface to the trait solver to be more query-based (ongoing work).
There is plenty of future work left. Here are two things that leap to mind:
- **Type-region outlives.** Currently, the NLL solver will ICE if it is required to handle a constraint like `T: 'a`. Fixing this will require a small amount of refactoring to extract the implied bounds code. I plan to follow a file-up bug on this (hopefully with mentoring instructions).
- **Testing.** It's a good idea to enumerate some of the tricky scenarios that need testing, but I think it'd be nice to try and parallelize some of the actual test writing (and resulting bug fixing):
- Same obligation occurring at two points.
- Well-formedness and trait obligations of various kinds (which are not all processed by the current MIR type-checker).
- More tests for how subtyping and region inferencing interact.
- More suggestions welcome!
r? @arielb1
We are heading towards deeper integration with the region inference
system in infcx; in particular, prior to the creation of the
`RegionInferenceContext`, it will be the "owner" of the set of region
variables.
check_unsafety: fix unused unsafe block duplication
The duplicate error message is later removed by error message
deduplication, but it still appears on beta and is still a bug.
r? @eddyb
Fix MIR CopyPropagation errneously propagating assignments to function args
Compiling this code with MIR CopyPropagation activated will result in printing `5`,
because CopyProp errneously propagates the assignment of `5` to all `x`:
```rust
fn bar(mut x: u8) {
println!("{}", x);
x = 5;
}
fn main() {
bar(123);
}
```
If a local is propagated, it will result in an ICE at trans due to an use-before-def:
```rust
fn dummy(x: u8) -> u8 { x }
fn foo(mut x: u8) {
x = dummy(x); // this will assign a local to `x`
}
```
Currently CopyProp conservatively gives up if there are multiple assignments to a local,
but it is not took into account that arguments are already assigned from the beginning.
This PR fixes the problem by preventing propagation of assignments to function arguments.
extend NLL with preliminary support for free regions on functions
This PR extends https://github.com/rust-lang/rust/pull/45538 with support for free regions. This is pretty preliminary and will no doubt want to change in various ways, particularly as we add support for closures, but it's enough to get the basic idea in place:
- We now create specific regions to represent each named lifetime declared on the function.
- Region values can contain references to these regions (represented for now as a `BTreeSet<RegionIndex>`).
- If we wind up trying to infer that `'a: 'b` must hold, but no such relationship was declared, we report an error.
It also does a number of drive-by refactorings.
r? @arielb1
cc @spastorino
The macro now takes a format string. It no longer defaults to using the
type name. Didn't seem worth going through contortions to maintain. I
also changed most of the debug formats to be `foo[N]` instead of `fooN`.
Also, factor out `do_mir_borrowck`, which is the code that actually
performs the MIR borrowck from within the scope of an inference context.
This change should be a pure refactoring.
Extend mir dump to dump each region
Building on #44878, implement the feature discussed in #44872.
Through discussions on the WG-nll-gitter, @nikomatsakis and I decided to implement this by extending `dump_mir` and all functions that it calls to take a callback of signature `FnMut(PassWhere, &mut Write) -> io::Result<()>` where `PassWhere` is an enum that represents possible locations that we may want to print out extra data in the process of dumping the MIR.
I'm not particularly wedded to the name `PassWhere`, but I felt that simply calling the enum `Where` wasn't the right thing to name it.
This work depends strongly on #44878, and should be rebased on the final version of that tree, whatever that may be.
MIR-borrowck: gather and signal any move errors
When building up the `MoveData` structure for a given MIR, also accumulate any erroneous actions, and then report all of those errors when the construction is complete.
This PR adds a host of move-related error constructor methods to `trait BorrowckErrors`. I think I got the notes right; but we should plan to audit all of the notes before turning MIR-borrowck on by default.
Fix#44830
Extend `dump_mir` and functions it calls in order to allow callers to
add custom information. We do this by adding an enum `PassWhere` and
an extra argument of type `FnMut(PassWhere, &mut Write) ->
io::Result<()>`. This callback is responsible for printing the extra
information when MIR is dumped at various stages.
For the "nll" pass, use the new mechanism to dump the `Region`
information after the header, but before the control flow graph for
every function.
In the interest of keeping the output somewhat concise, implement
a custom Debug impl for `Region`
Open Questions:
* What should we call what has been called `PassWhere` so far?
incr.comp.: Switch to red/green change tracking, remove legacy system.
This PR finally switches incremental compilation to [red/green tracking](https://github.com/rust-lang/rust/issues/42293) and completely removes the legacy dependency graph implementation -- which includes a few quite costly passes that are simply not needed with the new system anymore.
There's still some documentation to be done and there's certainly still lots of optimizing and tuning ahead -- but the foundation for red/green is in place with this PR. This has been in the making for a long time `:)`
r? @nikomatsakis
cc @alexcrichton, @rust-lang/compiler
Move monomorphize::resolve() to librustc
this moves `monomorphize::resolve(..)` to librustc, and re-enables inlining for some trait methods, fixing #44389
@nikomatsakis I've kept the calls to the new `ty::Instance::resolve(....)` always `.unwrap()`-ing for the moment, how/do you want to add more debugging info via `.unwrap_or()` or something like this?
we still have some related `resolve_*` functions on monomorphize, but I wasn't sure moving them was into the scope for this PR too.
@eddyb mind to take a look too?
bring TyCtxt into scope
got comments both from @eddyb and @nikomatsakis (via https://github.com/rust-lang/rust/pull/44505) that we should always put `TyCtxt` in scope
should I just go and import it at other places in the codebase or we just keep doing small improvements?
Individualize feature gates for const fn invocation
This PR changes the meaning of `#![feature(const_fn)]` so it is only required to declare a const fn but not to call one. Based on discussion at #24111. I was hoping we could have an FCP here in order to move that conversation forward.
This sets the stage for future stabilization of the constness of several functions in the standard library (listed below), so could someone please tag the lang team for review.
- `std::cell`
- `Cell::new`
- `RefCell::new`
- `UnsafeCell::new`
- `std::mem`
- `size_of`
- `align_of`
- `std::ptr`
- `null`
- `null_mut`
- `std::sync`
- `atomic`
- `Atomic{Bool,Ptr,Isize,Usize}::new`
- `once`
- `Once::new`
- primitives
- `{integer}::min_value`
- `{integer}::max_value`
Some other functions are const but they are also unstable or hidden, e.g. `Unique::new` so they don't have to be considered at this time.
After this stabilization, the following `*_INIT` constants in the standard library can be deprecated. I wasn't sure whether to include those deprecations in the current PR.
- `std::sync`
- `atomic`
- `ATOMIC_{BOOL,ISIZE,USIZE}_INIT`
- `once`
- `ONCE_INIT`
