macros: stackless expansion
After this PR, macro expansion cannot overflow the stack unless the expanded crate is too deep to fold.
Everything but the stackless placeholder expansion commit is also groundwork for macro modularization.
r? @nrc or @eddyb
incr. comp.: Take spans into account for ICH
This PR makes the ICH (incr. comp. hash) take spans into account when debuginfo is enabled.
A side-effect of this is that the SVH (which is based on the ICHs of all items in the crate) becomes sensitive to the tiniest change in a code base if debuginfo is enabled. Since we are not trying to model ABI compatibility via the SVH anymore (this is done via the crate disambiguator now), this should be not be a problem.
Fixes#33888.
Fixes#32753.
Allow CompilerControllers to access rustc_plugin::registry::Registry
fixes#36064
I chose to put ructc_plugin::registry::Registry structure
into CompilerState structure, instead of Session structure.
This will preserve dependencies among librustc, libructc_driver, and libructc_plugin.
@jseyfried @sanxiyn
Replace `_, _` with `..` in patterns
This is how https://github.com/rust-lang/rust/issues/33627 looks in action.
Looks especially nice in leftmost/rightmost positions `(first, ..)`/`(.., last)`.
I haven't touched libsyntax intentionally because the feature is still unstable.
Add --Zsave-analysis-api
This is a save-analysis variation which can be used with libraries distributed without their source (e.g., libstd). It will allow IDEs and other tools to get info about types and create URLs to docs and source, without the unnecessary clutter of internal-only save-analysis info. I'm sure we'll iterate somewhat on the design, but this is a first draft.
This commit is an implementation of [RFC 1681] which adds support to the
compiler for first-class user-define custom `#[derive]` modes with a far more
stable API than plugins have today.
[RFC 1681]: https://github.com/rust-lang/rfcs/blob/master/text/1681-macros-1.1.md
The main features added by this commit are:
* A new `rustc-macro` crate-type. This crate type represents one which will
provide custom `derive` implementations and perhaps eventually flower into the
implementation of macros 2.0 as well.
* A new `rustc_macro` crate in the standard distribution. This crate will
provide the runtime interface between macro crates and the compiler. The API
here is particularly conservative right now but has quite a bit of room to
expand into any manner of APIs required by macro authors.
* The ability to load new derive modes through the `#[macro_use]` annotations on
other crates.
All support added here is gated behind the `rustc_macro` feature gate, both for
the library support (the `rustc_macro` crate) as well as the language features.
There are a few minor differences from the implementation outlined in the RFC,
such as the `rustc_macro` crate being available as a dylib and all symbols are
`dlsym`'d directly instead of having a shim compiled. These should only affect
the implementation, however, not the public interface.
This commit also ended up touching a lot of code related to `#[derive]`, making
a few notable changes:
* Recognized derive attributes are no longer desugared to `derive_Foo`. Wasn't
sure how to keep this behavior and *not* expose it to custom derive.
* Derive attributes no longer have access to unstable features by default, they
have to opt in on a granular level.
* The `derive(Copy,Clone)` optimization is now done through another "obscure
attribute" which is just intended to ferry along in the compiler that such an
optimization is possible. The `derive(PartialEq,Eq)` optimization was also
updated to do something similar.
---
One part of this PR which needs to be improved before stabilizing are the errors
and exact interfaces here. The error messages are relatively poor quality and
there are surprising spects of this such as `#[derive(PartialEq, Eq, MyTrait)]`
not working by default. The custom attributes added by the compiler end up
becoming unstable again when going through a custom impl.
Hopefully though this is enough to start allowing experimentation on crates.io!
syntax-[breaking-change]
Implement synchronization scheme for incr. comp. directory
This PR implements a copy-on-write-based synchronization scheme for the incremental compilation cache directory. For technical details, see the documentation at the beginning of `rustc_incremental/persist/fs.rs`.
The PR contains unit tests for some functions but for testing whether the scheme properly handles races, a more elaborate test setup would be needed. It would probably involve a small tool that allows to manipulate the incremental compilation directory in a controlled way and then letting a compiler instance run against directories in different states. I don't know if it's worth the trouble of adding another test category to `compiletest`, but I'd be happy to do so.
Fixes#32754Fixes#34957
This avoids the compile-time overhead of computing them twice. It also fixes
an issue where the hash computed after typeck is differen than the hash before,
because typeck mutates the def-map in place.
Fixes#35549.
Fixes#35593.
Kicking off libproc_macro
This PR introduces `libproc_macro`, which is currently quite bare-bones (just a few macro construction tools and an initial `quote!` macro).
This PR also introduces a few test cases for it, and an additional `shim` file (at `src/libsyntax/ext/proc_macro_shim.rs` to allow a facsimile usage of Macros 2.0 *today*!
Implement the `!` type
This implements the never type (`!`) and hides it behind the feature gate `#[feature(never_type)]`. With the feature gate off, things should build as normal (although some error messages may be different). With the gate on, `!` is usable as a type and diverging type variables (ie. types that are unconstrained by anything in the code) will default to `!` instead of `()`.
Take commandline arguments into account for incr. comp.
Implements the conservative strategy described in https://github.com/rust-lang/rust/issues/33727.
From now one, every time a new commandline option is added, one has to specify if it influences the incremental compilation cache. I've tried to implement this as automatic as possible: One just has to added either the `[TRACKED]` or the `[UNTRACKED]` marker next to the field. The `Options`, `CodegenOptions`, and `DebuggingOptions` definitions in `session::config` show plenty of examples.
The PR removes some cruft from `session::config::Options`, mostly unnecessary copies of flags also present in `DebuggingOptions` or `CodeGenOptions` in the same struct.
One notable removal is the `cfg` field that contained the values passed via `--cfg` commandline arguments. I chose to remove it because (1) its content is only a subset of what later is stored in `hir::Crate::config` and it's pretty likely that reading the cfgs from `Options` would not be what you wanted, and (2) we could not incorporate it into the dep-tracking hash of the `Options` struct because of how the test framework works, leaving us with a piece of untracked but vital data.
It is now recommended (just as before) to access the crate config via the `krate()` method in the HIR map.
Because the `cfg` field is not present in the `Options` struct any more, some methods in the `CompilerCalls` trait now take the crate config as an explicit parameter -- which might constitute a breaking change for plugin authors.
Implement `impl Trait` in return type position by anonymization.
This is the first step towards implementing `impl Trait` (cc #34511).
`impl Trait` types are only allowed in function and inherent method return types, and capture all named lifetime and type parameters, being invariant over them.
No lifetimes that are not explicitly named lifetime parameters are allowed to escape from the function body.
The exposed traits are only those listed explicitly, i.e. `Foo` and `Clone` in `impl Foo + Clone`, with the exception of "auto traits" (like `Send` or `Sync`) which "leak" the actual contents.
The implementation strategy is anonymization, i.e.:
```rust
fn foo<T>(xs: Vec<T>) -> impl Iterator<Item=impl FnOnce() -> T> {
xs.into_iter().map(|x| || x)
}
// is represented as:
type A</*invariant over*/ T> where A<T>: Iterator<Item=B<T>>;
type B</*invariant over*/ T> where B<T>: FnOnce() -> T;
fn foo<T>(xs: Vec<T>) -> A<T> {
xs.into_iter().map(|x| || x): $0 where $0: Iterator<Item=$1>, $1: FnOnce() -> T
}
```
`$0` and `$1` are resolved (to `iter::Map<vec::Iter<T>, closure>` and the closure, respectively) and assigned to `A` and `B`, after checking the body of `foo`. `A` and `B` are *never* resolved for user-facing type equality (typeck), but always for the low-level representation and specialization (trans).
The "auto traits" exception is implemented by collecting bounds like `impl Trait: Send` that have failed for the obscure `impl Trait` type (i.e. `A` or `B` above), pretending they succeeded within the function and trying them again after type-checking the whole crate, by replacing `impl Trait` with the real type.
While passing around values which have explicit lifetime parameters (of the function with `-> impl Trait`) in their type *should* work, regionck appears to assign inference variables in *way* too many cases, and never properly resolving them to either explicit lifetime parameters, or `'static`.
We might not be able to handle lifetime parameters in `impl Trait` without changes to lifetime inference, but type parameters can have arbitrary lifetimes in them from the caller, so most type-generic usecases (or not generic at all) should not run into this problem.
cc @rust-lang/lang
The 'cfg' in the Options struct is only the commandline-specified
subset of the crate configuration and it's almost always wrong to
read that instead of the CrateConfig in HIR crate node.
Commandline arguments influence whether incremental compilation
can use its compilation cache and thus their changes relative to
previous compilation sessions need to be taking into account. This
commit makes sure that one has to specify for every commandline
argument whether it influences incremental compilation or not.
Various improvements to the SVH
This fixes a few points for the SVH:
- incorporate resolve results into the SVH;
- don't include nested items.
r? @michaelwoerister
cc #32753 (not fully fixed I don't think)
Turn on new errors and json mode
This PR is a big-switch, but on a well-worn path:
* Turns on new errors by default (and removes old skool)
* Moves json output from behind a flag
The RFC for new errors [landed](https://github.com/rust-lang/rfcs/pull/1644) and as part of that we wanted some bake time. It's now had a few weeks + all the time leading up to the RFC of people banging on it. We've also had [editors updating to the new format](https://github.com/saviorisdead/RustyCode/pull/159) and expect more to follow.
We also have an [issue on old skool](https://github.com/rust-lang/rust/issues/35330) that needs to be fixed as more errors are switched to the new style, but it seems silly to fix old skool errors when we fully intend to throw the switch in the near future.
This makes it lean towards "why not just throw the switch now, rather than waiting a couple more weeks?" I only know of vim that wanted to try to parse the new format but were not sure how, and I think we can reach out to them and work out something in the 8 weeks before this would appear in a stable release.
We've [hashed out](https://github.com/rust-lang/rust/issues/35330) stabilizing JSON output, and it seems like people are relatively happy making what we have v1 and then likely adding to it in the future. The idea is that we'd maintain backward compatibility and just add new fields as needed. We'll also work on a separate output format that'd be better suited for interactive tools like IDES (since JSON message can get a little long depending on the error).
This PR stabilizes JSON mode, allowing its use without `-Z unstable-options`
Combined, this gives editors two ways to support errors going forward: parsing the new error format or using the JSON mode. By moving JSON to stable, we can also add support to Cargo, which plugin authors tell us does help simplify their support story.
r? @nikomatsakis
cc @rust-lang/tools
Closes https://github.com/rust-lang/rust/issues/34826
Address ICEs running w/ incremental compilation and building glium
Fixes for various ICEs I encountered trying to build glium with incremental compilation enabled. Building glium now works. Of the 4 ICEs, I have test cases for 3 of them -- I didn't isolate a test for the last commit and kind of want to go do other things -- most notably, figuring out why incremental isn't saving much *effort*.
But if it seems worthwhile and I can come back and try to narrow down the problem.
r? @michaelwoerister
Fixes#34991Fixes#32015
Per the discussion on #34765, we make one `DepNode::Mir` variant and use
it to represent both the MIR tracking map as well as passes that operate
on MIR. We also track loads of cached MIR (which naturally comes from
metadata).
Note that the "HAIR" pass adds a read of TypeckItemBody because it uses
a myriad of tables that are not individually tracked.
[MIR] Deaggregate structs to enable further optimizations
Currently, we generate MIR like:
```
tmp0 = ...;
tmp1 = ...;
tmp3 = Foo { a: ..., b: ... };
```
This PR implements "deaggregation," i.e.:
```
tmp3.0 = ...
tmp3.1 = ...
```
Currently, the code only deaggregates structs, not enums. My understanding is that we do not have MIR to set the discriminant of an enum.
Better attribute and metaitem encapsulation throughout the compiler
This PR refactors most (hopefully all?) of the `MetaItem` interactions outside of `libsyntax` (and a few inside) to interact with MetaItems through the provided traits instead of directly creating / destruct / matching against them. This is a necessary first step to eventually converting `MetaItem`s to internally use `TokenStream` representations (which will make `MetaItem` interactions much nicer for macro writers once the new macro system is in place).
r? @nrc
Convert built-in targets to JSON
Convert the built-in targets to JSON to ensure that the JSON parser is always fully featured. This follows on #32988 and #32847. The PR includes a number of extra commits that are just intermediate changes necessary for bisectibility and the ability to prove correctness of the change.
We used to use `Name`, but the session outlives the tokenizer, which
means that attempts to read this field after trans has complete
otherwise panic. All reads want an `InternedString` anyhow.
Since we can know which targets are instantiable on a particular host,
it does not make sense to list invalid targets in the target print code.
Filter the list of targets to only include the targets that can be
instantiated.
Simplify librustc_errors
This is part 2 of the error crate refactor, starting with #34403.
In this refactor, I focused on slimming down the error crate to fewer moving parts. As such, I've removed quite a few parts and replaced the with simpler, straight-line code. Specifically, this PR:
* Removes BasicEmitter
* Remove emit from emitter, leaving emit_struct
* Renames emit_struct to emit
* Removes CoreEmitter and focuses on a single Emitter
* Implements the latest changes to error format RFC (#1644)
* Removes (now-unused) code in emitter.rs and snippet.rs
* Moves more tests to the UI tester, removing some duplicate tests in the process
There is probably more that could be done with some additional refactoring, but this felt like it was getting to a good state.
r? @alexcrichton cc: @Manishearth (as there may be breaking changes in stuff I removed/changed)
Simplify the macro hygiene algorithm
This PR removes renaming from the hygiene algorithm and treats differently marked identifiers as unequal.
This change makes the scope of identifiers in `macro_rules!` items empty. That is, identifiers in `macro_rules!` definitions do not inherit any semantics from the `macro_rules!`'s scope.
Since `macro_rules!` macros are items, the scope of their identifiers "should" be the same as that of other items; in particular, the scope should contain only items. Since all items are unhygienic today, this would mean the scope should be empty.
However, the scope of an identifier in a `macro_rules!` statement today is the scope that the identifier would have if it replaced the `macro_rules!` (excluding anything unhygienic, i.e. locals only).
To continue to support this, this PR tracks the scope of each `macro_rules!` and uses it in `resolve` to ensure that an identifier expanded from a `macro_rules!` gets a chance to resolve to the locals in the `macro_rules!`'s scope.
This PR is a pure refactoring. After this PR,
- `syntax::ext::expand` is much simpler.
- We can expand macros in any order without causing problems for hygiene (needed for macro modularization).
- We can deprecate or remove today's `macro_rules!` scope easily.
- Expansion performance improves by 25%, post-expansion memory usage decreases by ~5%.
- Expanding a block is no longer quadratic in the number of `let` statements (fixes#10607).
r? @nrc
Add x86 intrinsics for bit manipulation (BMI 1.0, BMI 2.0, and TBM).
This PR adds the LLVM x86 intrinsics for the bit manipulation instruction sets (BMI 1.0, BMI 2.0, and TBM).
The objective of this pull-request is to allow building a library that implements all the algorithms offered by those instruction sets, using compiler intrinsics for the targets that support them (by means of `target_feature`).
The target features added are:
- `bmi`: Bit Manipulation Instruction Set 1.0, available in Intel >= Haswell and AMD's >= Jaguar/Piledriver,
- `bmi2`: Bit Manipulation Instruction Set 2.0, available in Intel >= Haswell and AMD's >= Excavator,
- `tbm`: Trailing Bit Manipulation, available only in AMD's Piledriver (won't be available in newer CPUs).
The intrinsics added are:
- BMI 1.0:
- `bextr`: Bit field extract (with register).
- BMI 2.0:
- `bzhi`: Zero high bits starting with specified bit position.
- `pdep`: Parallel bits deposit.
- `pext`: Parallel bits extract.
- TBM:
- `bextri`: Bit field extract (with immediate).
To allow these braced macro invocation, this PR removes the optional expression from `ast::Block` and instead uses a `StmtKind::Expr` at the end of the statement list.
Currently, braced macro invocations in blocks can expand into statements (and items) except when they are last in a block, in which case they can only expand into expressions.
For example,
```rust
macro_rules! make_stmt {
() => { let x = 0; }
}
fn f() {
make_stmt! {} //< This is OK...
let x = 0; //< ... unless this line is commented out.
}
```
Fixes#34418.
