Per the discussion on #34765, we make one `DepNode::Mir` variant and use
it to represent both the MIR tracking map as well as passes that operate
on MIR. We also track loads of cached MIR (which naturally comes from
metadata).
Note that the "HAIR" pass adds a read of TypeckItemBody because it uses
a myriad of tables that are not individually tracked.
[MIR] Deaggregate structs to enable further optimizations
Currently, we generate MIR like:
```
tmp0 = ...;
tmp1 = ...;
tmp3 = Foo { a: ..., b: ... };
```
This PR implements "deaggregation," i.e.:
```
tmp3.0 = ...
tmp3.1 = ...
```
Currently, the code only deaggregates structs, not enums. My understanding is that we do not have MIR to set the discriminant of an enum.
Better attribute and metaitem encapsulation throughout the compiler
This PR refactors most (hopefully all?) of the `MetaItem` interactions outside of `libsyntax` (and a few inside) to interact with MetaItems through the provided traits instead of directly creating / destruct / matching against them. This is a necessary first step to eventually converting `MetaItem`s to internally use `TokenStream` representations (which will make `MetaItem` interactions much nicer for macro writers once the new macro system is in place).
r? @nrc
Convert built-in targets to JSON
Convert the built-in targets to JSON to ensure that the JSON parser is always fully featured. This follows on #32988 and #32847. The PR includes a number of extra commits that are just intermediate changes necessary for bisectibility and the ability to prove correctness of the change.
We used to use `Name`, but the session outlives the tokenizer, which
means that attempts to read this field after trans has complete
otherwise panic. All reads want an `InternedString` anyhow.
Since we can know which targets are instantiable on a particular host,
it does not make sense to list invalid targets in the target print code.
Filter the list of targets to only include the targets that can be
instantiated.
Simplify librustc_errors
This is part 2 of the error crate refactor, starting with #34403.
In this refactor, I focused on slimming down the error crate to fewer moving parts. As such, I've removed quite a few parts and replaced the with simpler, straight-line code. Specifically, this PR:
* Removes BasicEmitter
* Remove emit from emitter, leaving emit_struct
* Renames emit_struct to emit
* Removes CoreEmitter and focuses on a single Emitter
* Implements the latest changes to error format RFC (#1644)
* Removes (now-unused) code in emitter.rs and snippet.rs
* Moves more tests to the UI tester, removing some duplicate tests in the process
There is probably more that could be done with some additional refactoring, but this felt like it was getting to a good state.
r? @alexcrichton cc: @Manishearth (as there may be breaking changes in stuff I removed/changed)
Simplify the macro hygiene algorithm
This PR removes renaming from the hygiene algorithm and treats differently marked identifiers as unequal.
This change makes the scope of identifiers in `macro_rules!` items empty. That is, identifiers in `macro_rules!` definitions do not inherit any semantics from the `macro_rules!`'s scope.
Since `macro_rules!` macros are items, the scope of their identifiers "should" be the same as that of other items; in particular, the scope should contain only items. Since all items are unhygienic today, this would mean the scope should be empty.
However, the scope of an identifier in a `macro_rules!` statement today is the scope that the identifier would have if it replaced the `macro_rules!` (excluding anything unhygienic, i.e. locals only).
To continue to support this, this PR tracks the scope of each `macro_rules!` and uses it in `resolve` to ensure that an identifier expanded from a `macro_rules!` gets a chance to resolve to the locals in the `macro_rules!`'s scope.
This PR is a pure refactoring. After this PR,
- `syntax::ext::expand` is much simpler.
- We can expand macros in any order without causing problems for hygiene (needed for macro modularization).
- We can deprecate or remove today's `macro_rules!` scope easily.
- Expansion performance improves by 25%, post-expansion memory usage decreases by ~5%.
- Expanding a block is no longer quadratic in the number of `let` statements (fixes#10607).
r? @nrc
Add x86 intrinsics for bit manipulation (BMI 1.0, BMI 2.0, and TBM).
This PR adds the LLVM x86 intrinsics for the bit manipulation instruction sets (BMI 1.0, BMI 2.0, and TBM).
The objective of this pull-request is to allow building a library that implements all the algorithms offered by those instruction sets, using compiler intrinsics for the targets that support them (by means of `target_feature`).
The target features added are:
- `bmi`: Bit Manipulation Instruction Set 1.0, available in Intel >= Haswell and AMD's >= Jaguar/Piledriver,
- `bmi2`: Bit Manipulation Instruction Set 2.0, available in Intel >= Haswell and AMD's >= Excavator,
- `tbm`: Trailing Bit Manipulation, available only in AMD's Piledriver (won't be available in newer CPUs).
The intrinsics added are:
- BMI 1.0:
- `bextr`: Bit field extract (with register).
- BMI 2.0:
- `bzhi`: Zero high bits starting with specified bit position.
- `pdep`: Parallel bits deposit.
- `pext`: Parallel bits extract.
- TBM:
- `bextri`: Bit field extract (with immediate).
To allow these braced macro invocation, this PR removes the optional expression from `ast::Block` and instead uses a `StmtKind::Expr` at the end of the statement list.
Currently, braced macro invocations in blocks can expand into statements (and items) except when they are last in a block, in which case they can only expand into expressions.
For example,
```rust
macro_rules! make_stmt {
() => { let x = 0; }
}
fn f() {
make_stmt! {} //< This is OK...
let x = 0; //< ... unless this line is commented out.
}
```
Fixes#34418.
MIR cleanups and predecessor cache
This PR cleans up a few things in MIR and adds a predecessor cache to allow graph algorithms to be run easily.
r? @nikomatsakis
Refactor away the prelude injection fold
Instead, just inject `#[prelude_import] use [core|std]::prelude::v1::*;` at the crate root while injecting `extern crate [core|std];` and process `#[no_implicit_prelude]` attributes in `resolve`.
r? @nrc
rustc: Try to contain prepends to PATH
This commit attempts to bring our prepends to PATH on Windows when loading
plugins because we've been seeing quite a few issues with failing to spawn a
process on Windows, the leading theory of which is that PATH is too large as a
result of this. Currently this is mostly a stab in the dark as it's not
confirmed to actually fix the problem, but it's probably not a bad change to
have anyway!
cc #33844Closes#17360
This commit attempts to bring our prepends to PATH on Windows when loading
plugins because we've been seeing quite a few issues with failing to spawn a
process on Windows, the leading theory of which is that PATH is too large as a
result of this. Currently this is mostly a stab in the dark as it's not
confirmed to actually fix the problem, but it's probably not a bad change to
have anyway!
cc #33844Closes#17360
Add AST validation pass and move some checks to it
The purpose of this pass is to catch constructions that fit into AST data structures, but not permitted by the language. As an example, `impl`s don't have visibilities, but for convenience and uniformity with other items they are represented with a structure `Item` which has `Visibility` field.
This pass is intended to run after expansion of macros and syntax extensions (and before lowering to HIR), so it can catch erroneous constructions that were generated by them. This pass allows to remove ad hoc semantic checks from the parser, which can be overruled by syntax extensions and occasionally macros.
The checks can be put here if they are simple, local, don't require results of any complex analysis like name resolution or type checking and maybe don't logically fall into other passes. I expect most of errors generated by this pass to be non-fatal and allowing the compilation to proceed.
I intend to move some more checks to this pass later and maybe extend it with new checks, like, for example, identifier validity. Given that syntax extensions are going to be stabilized in the measurable future, it's important that they would not be able to subvert usual language rules.
In this patch I've added two new checks - a check for labels named `'static` and a check for lifetimes and labels named `'_`. The first one gives a hard error, the second one - a future compatibility warning.
Fixes https://github.com/rust-lang/rust/issues/33059 ([breaking-change])
cc https://github.com/rust-lang/rfcs/pull/1177
r? @nrc
Perform `cfg` attribute processing during macro expansion and fix bugs
This PR refactors `cfg` attribute processing and fixes bugs. More specifically:
- It merges gated feature checking for stmt/expr attributes, `cfg_attr` processing, and `cfg` processing into a single fold.
- This allows feature gated `cfg` variables to be used in `cfg_attr` on unconfigured items. All other feature gated attributes can already be used on unconfigured items.
- It performs `cfg` attribute processing during macro expansion instead of after expansion so that macro-expanded items are configured the same as ordinary items. In particular, to match their non-expanded counterparts,
- macro-expanded unconfigured macro invocations are no longer expanded,
- macro-expanded unconfigured macro definitions are no longer usable, and
- feature gated `cfg` variables on macro-expanded macro definitions/invocations are now errors.
This is a [breaking-change]. For example, the following would break:
```rust
macro_rules! m {
() => {
#[cfg(attr)]
macro_rules! foo { () => {} }
foo!(); // This will be an error
macro_rules! bar { () => { fn f() {} } }
#[cfg(attr)] bar!(); // This will no longer be expanded ...
fn g() { f(); } // ... so that `f` will be unresolved.
#[cfg(target_thread_local)] // This will be a gated feature error
macro_rules! baz { () => {} }
}
}
m!();
```
r? @nrc
rustc: Add a new crate type, cdylib
This commit is an implementation of [RFC 1510] which adds a new crate type,
`cdylib`, to the compiler. This new crate type differs from the existing `dylib`
crate type in a few key ways:
* No metadata is present in the final artifact
* Symbol visibility rules are the same as executables, that is only reachable
`extern` functions are visible symbols
* LTO is allowed
* All libraries are always linked statically
This commit is relatively simple by just plubming the compiler with another
crate type which takes different branches here and there. The only major change
is an implementation of the `Linker::export_symbols` function on Unix which now
actually does something. This helps restrict the public symbols from a cdylib on
Unix.
With this PR a "hello world" `cdylib` is 7.2K while the same `dylib` is 2.4MB,
which is some nice size savings!
[RFC 1510]: https://github.com/rust-lang/rfcs/pull/1510Closes#33132
This commit is an implementation of [RFC 1510] which adds a new crate type,
`cdylib`, to the compiler. This new crate type differs from the existing `dylib`
crate type in a few key ways:
* No metadata is present in the final artifact
* Symbol visibility rules are the same as executables, that is only reachable
`extern` functions are visible symbols
* LTO is allowed
* All libraries are always linked statically
This commit is relatively simple by just plubming the compiler with another
crate type which takes different branches here and there. The only major change
is an implementation of the `Linker::export_symbols` function on Unix which now
actually does something. This helps restrict the public symbols from a cdylib on
Unix.
With this PR a "hello world" `cdylib` is 7.2K while the same `dylib` is 2.4MB,
which is some nice size savings!
[RFC 1510]: https://github.com/rust-lang/rfcs/pull/1510Closes#33132
Only break critical edges where actually needed
Currently, to prepare for MIR trans, we break _all_ critical edges,
although we only actually need to do this for edges originating from a
call that gets translated to an invoke instruction in LLVM.
This has the unfortunate effect of undoing a bunch of the things that
SimplifyCfg has done. A particularly bad case arises when you have a
C-like enum with N variants and a derived PartialEq implementation.
In that case, the match on the (&lhs, &rhs) tuple gets translated into
nested matches with N arms each and a basic block each, resulting in N²
basic blocks. SimplifyCfg reduces that to roughly 2*N basic blocks, but
breaking the critical edges means that we go back to N².
In nickel.rs, there is such an enum with roughly N=800. So we get about
640K basic blocks or 2.5M lines of LLVM IR. LLVM takes a while to
reduce that to the final "disr_a == disr_b".
So before this patch, we had 2.5M lines of IR with 640K basic blocks,
which took about about 3.6s in LLVM to get optimized and translated.
After this patch, we get about 650K lines with about 1.6K basic blocks
and spent a little less than 0.2s in LLVM.
cc #33111
r? @Aatch
Clean up `hir::lowering`
Clean up `hir::lowering`:
- give lowering functions mutable access to the lowering context
- refactor the `lower_*` functions and other functions that take a lowering context into methods
- simplify the API that `hir::lowering` exposes to `driver`
- other miscellaneous cleanups
r? @nrc
Make --emit dep-info work correctly with -Z no-analysis again.
Previously, it would attempt to resolve some external crates that weren't necessary for dep-info output.
Fixes#33231.
Currently, to prepare for MIR trans, we break _all_ critical edges,
although we only actually need to do this for edges originating from a
call that gets translated to an invoke instruction in LLVM.
This has the unfortunate effect of undoing a bunch of the things that
SimplifyCfg has done. A particularly bad case arises when you have a
C-like enum with N variants and a derived PartialEq implementation.
In that case, the match on the (&lhs, &rhs) tuple gets translated into
nested matches with N arms each and a basic block each, resulting in N²
basic blocks. SimplifyCfg reduces that to roughly 2*N basic blocks, but
breaking the critical edges means that we go back to N².
In nickel.rs, there is such an enum with roughly N=800. So we get about
640K basic blocks or 2.5M lines of LLVM IR. LLVM takes a while to
reduce that to the final "disr_a == disr_b".
So before this patch, we had 2.5M lines of IR with 640K basic blocks,
which took about about 3.6s in LLVM to get optimized and translated.
After this patch, we get about 650K lines with about 1.6K basic blocks
and spent a little less than 0.2s in LLVM.
cc #33111
Implement constant support in MIR.
All of the intended features in `trans::consts` are now supported by `mir::constant`.
The implementation is considered a temporary measure until `miri` replaces it.
A `-Z orbit` bootstrap build will only translate LLVM IR from AST for `#[rustc_no_mir]` functions.
Furthermore, almost all checks of constant expressions have been moved to MIR.
In non-`const` functions, trees of temporaries are promoted, as per RFC 1414 (rvalue promotion).
Promotion before MIR borrowck would allow reasoning about promoted values' lifetimes.
The improved checking comes at the cost of four `[breaking-change]`s:
* repeat counts must contain a constant expression, e.g.:
`let arr = [0; { println!("foo"); 5 }];` used to be allowed (it behaved like `let arr = [0; 5];`)
* dereference of a reference to a `static` cannot be used in another `static`, e.g.:
`static X: [u8; 1] = [1]; static Y: u8 = (&X)[0];` was unintentionally allowed before
* the type of a `static` *must* be `Sync`, irrespective of the initializer, e.g.
`static FOO: *const T = &BAR;` worked as `&T` is `Sync`, but it shouldn't because `*const T` isn't
* a `static` cannot wrap `UnsafeCell` around a type that *may* need drop, e.g.
`static X: MakeSync<UnsafeCell<Option<String>>> = MakeSync(UnsafeCell::new(None));`
was previously allowed based on the fact `None` alone doesn't need drop, but in `UnsafeCell`
it can be later changed to `Some(String)` which *does* need dropping
The drop restrictions are relaxed by RFC 1440 (#33156), which is implemented, but feature-gated.
However, creating `UnsafeCell` from constants is unstable, so users can just enable the feature gate.
There is now a CoreEmitter that everything desugars to, but without
losing any information. Also remove RenderSpan::FileLine. This lets the
rustc_driver tests build.
Major changes:
- Remove old snippet rendering code and use the new stuff.
- Introduce `span_label` method to add a label
- Remove EndSpan mode and replace with a fn to get the last
character of a span.
- Stop using `Option<MultiSpan>` and just use an empty `MultiSpan`
- and probably a bunch of other stuff :)
Remove the requirement that ast->hir lowering be reproducible
This PR changes the ast->hir lowerer to be non-reproducible, and it removes the lowering context's id cache.
If the `hir` of an `ast` node needs to be reproduced, we can use the hir map instead of the lowerer -- for example, `tcx.map.expect_expr(expr.id)` instead of `lower_expr(lcx, expr)`.
r? @nrc
rustc_driver: Allow running the compiler with a FileLoader
cc @nrc. I chose to implement this in such a way that it doesn't break anything. Please let me know if you want me to change anything.
Feature gate clean
This PR does a bit of cleaning in the feature-gate-handling code of libsyntax. It also fixes two bugs (#32782 and #32648). Changes include:
* Change the way the existing features are declared in `feature_gate.rs`. The array of features and the `Features` struct are now defined together by a single macro. `featureck.py` has been updated accordingly. Note: there are now three different arrays for active, removed and accepted features instead of a single one with a `Status` item to tell wether a feature is active, removed, or accepted. This is mainly due to the way I implemented my macro in the first time and I can switch back to a single array if needed. But an advantage of the way it is now is that when an active feature is used, the parser only searches through the list of active features. It goes through the other arrays only if the feature is not found. I like to think that error checking (in this case, checking that an used feature is active) does not slow down compilation of valid code. :) But this is not very important...
* Feature-gate checking pass now use the `Features` structure instead of looking through a string vector. This should speed them up a bit. The construction of the `Features` struct should be faster too since it is build directly when parsing features instead of calling `has_feature` dozens of times.
* The MacroVisitor pass has been removed, it was mostly useless since the `#[cfg]-stripping` phase happens before (fixes#32648). The features that must actually be checked before expansion are now checked at the time they are used. This also allows us to check attributes that are generated by macro expansion and not visible to MacroVisitor, but are also removed by macro expansion and thus not visible to PostExpansionVisitor either. This fixes#32782. Note that in order for `#[derive_*]` to be feature-gated but still accepted when generated by `#[derive(Trait)]`, I had to do a little bit of trickery with spans that I'm not totally confident into. Please review that part carefully. (It's in `libsyntax_ext/deriving/mod.rs`.)::
Note: this is a [breaking change], since programs with feature-gated attributes on macro-generated macro invocations were not rejected before. For example:
```rust
macro_rules! bar (
() => ()
);
macro_rules! foo (
() => (
#[allow_internal_unstable] //~ ERROR allow_internal_unstable side-steps
bar!();
);
);
```
foo!();
In fact, we make JSOn the default and add an option for save-analysis-csv for the legacy behaviour.
We also rename some bits and pieces `dxr` -> `save-analysis`
This pass was supposed to check use of gated features before
`#[cfg]`-stripping but this was not the case since it in fact happens
after. Checks that are actually important and must be done before macro
expansion are now made where the features are actually used. Close#32648.
Also ensure that attributes on macro-generated macro invocations are
checked as well. Close#32782 and #32655.
Compute `target_feature` from LLVM
This is a work-in-progress fix for #31662.
The logic that computes the target features from the command line has been replaced with queries to the `TargetMachine`.
Assert that the feature strings are NUL terminated, so that they will
be well-formed as C strings.
This is a safety check to ease the maintaninace and update of the
feature lists.
