This commit is an implementation of [RFC 1681] which adds support to the
compiler for first-class user-define custom `#[derive]` modes with a far more
stable API than plugins have today.
[RFC 1681]: https://github.com/rust-lang/rfcs/blob/master/text/1681-macros-1.1.md
The main features added by this commit are:
* A new `rustc-macro` crate-type. This crate type represents one which will
provide custom `derive` implementations and perhaps eventually flower into the
implementation of macros 2.0 as well.
* A new `rustc_macro` crate in the standard distribution. This crate will
provide the runtime interface between macro crates and the compiler. The API
here is particularly conservative right now but has quite a bit of room to
expand into any manner of APIs required by macro authors.
* The ability to load new derive modes through the `#[macro_use]` annotations on
other crates.
All support added here is gated behind the `rustc_macro` feature gate, both for
the library support (the `rustc_macro` crate) as well as the language features.
There are a few minor differences from the implementation outlined in the RFC,
such as the `rustc_macro` crate being available as a dylib and all symbols are
`dlsym`'d directly instead of having a shim compiled. These should only affect
the implementation, however, not the public interface.
This commit also ended up touching a lot of code related to `#[derive]`, making
a few notable changes:
* Recognized derive attributes are no longer desugared to `derive_Foo`. Wasn't
sure how to keep this behavior and *not* expose it to custom derive.
* Derive attributes no longer have access to unstable features by default, they
have to opt in on a granular level.
* The `derive(Copy,Clone)` optimization is now done through another "obscure
attribute" which is just intended to ferry along in the compiler that such an
optimization is possible. The `derive(PartialEq,Eq)` optimization was also
updated to do something similar.
---
One part of this PR which needs to be improved before stabilizing are the errors
and exact interfaces here. The error messages are relatively poor quality and
there are surprising spects of this such as `#[derive(PartialEq, Eq, MyTrait)]`
not working by default. The custom attributes added by the compiler end up
becoming unstable again when going through a custom impl.
Hopefully though this is enough to start allowing experimentation on crates.io!
syntax-[breaking-change]
Refactor `PathListItem`s
This refactors away variant `Mod` of `ast::PathListItemKind` and refactors the remaining variant `Ident` to a struct `ast::PathListItem_`.
Kicking off libproc_macro
This PR introduces `libproc_macro`, which is currently quite bare-bones (just a few macro construction tools and an initial `quote!` macro).
This PR also introduces a few test cases for it, and an additional `shim` file (at `src/libsyntax/ext/proc_macro_shim.rs` to allow a facsimile usage of Macros 2.0 *today*!
macros: Make metavariables hygienic
This PR makes metavariables hygienic. For example, consider:
```rust
macro_rules! foo {
($x:tt) => { // Suppose that this token tree argument is always a metavariable.
macro_rules! bar { ($x:expr, $y:expr) => { ($x, $y) } }
}
}
fn main() {
foo!($z); // This currently compiles.
foo!($y); // This is an error today but compiles after this PR.
}
```
Today, the `macro_rules! bar { ... }` definition is only valid when the metavariable passed to `foo` is not `$y` (since it unhygienically conflicts with the `$y` in the definition of `bar`) or `$x` (c.f. #35450).
After this PR, the definition of `bar` is always valid (and `bar!(a, b)` always expands to `(a, b)` as expected).
This can break code that was allowed in #34925 (landed two weeks ago). For example,
```rust
macro_rules! outer {
($t:tt) => {
macro_rules! inner { ($i:item) => { $t } }
}
}
outer!($i); // This `$i` should not interact with the `$i` in the definition of `inner!`.
inner!(fn main() {}); // After this PR, this is an error ("unknown macro variable `i`").
```
Due to the severe limitations on nested `macro_rules!` before #34925, this is not a breaking change for stable/beta.
Fixes#35450.
r? @nrc
Better attribute and metaitem encapsulation throughout the compiler
This PR refactors most (hopefully all?) of the `MetaItem` interactions outside of `libsyntax` (and a few inside) to interact with MetaItems through the provided traits instead of directly creating / destruct / matching against them. This is a necessary first step to eventually converting `MetaItem`s to internally use `TokenStream` representations (which will make `MetaItem` interactions much nicer for macro writers once the new macro system is in place).
r? @nrc
Simplify the macro hygiene algorithm
This PR removes renaming from the hygiene algorithm and treats differently marked identifiers as unequal.
This change makes the scope of identifiers in `macro_rules!` items empty. That is, identifiers in `macro_rules!` definitions do not inherit any semantics from the `macro_rules!`'s scope.
Since `macro_rules!` macros are items, the scope of their identifiers "should" be the same as that of other items; in particular, the scope should contain only items. Since all items are unhygienic today, this would mean the scope should be empty.
However, the scope of an identifier in a `macro_rules!` statement today is the scope that the identifier would have if it replaced the `macro_rules!` (excluding anything unhygienic, i.e. locals only).
To continue to support this, this PR tracks the scope of each `macro_rules!` and uses it in `resolve` to ensure that an identifier expanded from a `macro_rules!` gets a chance to resolve to the locals in the `macro_rules!`'s scope.
This PR is a pure refactoring. After this PR,
- `syntax::ext::expand` is much simpler.
- We can expand macros in any order without causing problems for hygiene (needed for macro modularization).
- We can deprecate or remove today's `macro_rules!` scope easily.
- Expansion performance improves by 25%, post-expansion memory usage decreases by ~5%.
- Expanding a block is no longer quadratic in the number of `let` statements (fixes#10607).
r? @nrc
Fix expansion performance regression
**syntax-[breaking-change] cc #31645**
This fixes#34630 by reverting commit 5bf7970 of PR #33943, which landed in #34424.
By removing the `Rc<_>` wrapping around `Delimited` and `SequenceRepetition` in `TokenTree`, 5bf7970 made cloning `TokenTree`s more expensive. While this had no measurable performance impact on the compiler's crates, it caused an order of magnitude performance regression on some macro-heavy code in the wild. I believe this is due to clones of `TokenTree`s in `macro_parser.rs` and/or `macro_rules.rs`.
r? @nrc
This is a spiritual succesor to #34268/8531d581, in which we replaced a
number of matches of None to the unit value with `if let` conditionals
where it was judged that this made for clearer/simpler code (as would be
recommended by Manishearth/rust-clippy's `single_match` lint). The same
rationale applies to matches of None to the empty block.
