This commit starts to lay some groundwork for the stabilization of custom
attribute invocations and general procedural macros. It applies a number of
changes discussed on [internals] as well as a [recent issue][issue], namely:
* The path used to specify a custom attribute must be of length one and cannot
be a global path. This'll help future-proof us against any ambiguities and
give us more time to settle the precise syntax. In the meantime though a bare
identifier can be used and imported to invoke a custom attribute macro. A new
feature gate, `proc_macro_path_invoc`, was added to gate multi-segment paths
and absolute paths.
* The set of items which can be annotated by a custom procedural attribute has
been restricted. Statements, expressions, and modules are disallowed behind
two new feature gates: `proc_macro_expr` and `proc_macro_mod`.
* The input to procedural macro attributes has been restricted and adjusted.
Today an invocation like `#[foo(bar)]` will receive `(bar)` as the input token
stream, but after this PR it will only receive `bar` (the delimiters were
removed). Invocations like `#[foo]` are still allowed and will be invoked in
the same way as `#[foo()]`. This is a **breaking change** for all nightly
users as the syntax coming in to procedural macros will be tweaked slightly.
* Procedural macros (`foo!()` style) can only be expanded to item-like items by
default. A separate feature gate, `proc_macro_non_items`, is required to
expand to items like expressions, statements, etc.
Closes#50038
[internals]: https://internals.rust-lang.org/t/help-stabilize-a-subset-of-macros-2-0/7252
[issue]: https://github.com/rust-lang/rust/issues/50038
This commit is a reorganization of the `proc_macro` crate's public user-facing
API. This is the result of a number of discussions at the recent Rust All-Hands
where we're hoping to get the `proc_macro` crate into ship shape for
stabilization of a subset of its functionality in the Rust 2018 release.
The reorganization here is motivated by experiences from the `proc-macro2`,
`quote`, and `syn` crates on crates.io (and other crates which depend on them).
The main focus is future flexibility along with making a few more operations
consistent and/or fixing bugs. A summary of the changes made from today's
`proc_macro` API is:
* The `TokenNode` enum has been removed and the public fields of `TokenTree`
have also been removed. Instead the `TokenTree` type is now a public enum
(what `TokenNode` was) and each variant is an opaque struct which internally
contains `Span` information. This makes the various tokens a bit more
consistent, require fewer wrappers, and otherwise provides good
future-compatibility as opaque structs are easy to modify later on.
* `Literal` integer constructors have been expanded to be unambiguous as to what
they're doing and also allow for more future flexibility. Previously
constructors like `Literal::float` and `Literal::integer` were used to create
unsuffixed literals and the concrete methods like `Literal::i32` would create
a suffixed token. This wasn't immediately clear to all users (the
suffixed/unsuffixed aspect) and having *one* constructor for unsuffixed
literals required us to pick a largest type which may not always be true. To
fix these issues all constructors are now of the form
`Literal::i32_unsuffixed` or `Literal::i32_suffixed` (for all integral types).
This should allow future compatibility as well as being immediately clear
what's suffixed and what isn't.
* Each variant of `TokenTree` internally contains a `Span` which can also be
configured via `set_span`. For example `Literal` and `Term` now both
internally contain a `Span` rather than having it stored in an auxiliary
location.
* Constructors of all tokens are called `new` now (aka `Term::intern` is gone)
and most do not take spans. Manufactured tokens typically don't have a fresh
span to go with them and the span is purely used for error-reporting
**except** the span for `Term`, which currently affects hygiene. The default
spans for all these constructed tokens is `Span::call_site()` for now.
The `Term` type's constructor explicitly requires passing in a `Span` to
provide future-proofing against possible hygiene changes. It's intended that a
first pass of stabilization will likely only stabilize `Span::call_site()`
which is an explicit opt-in for "I would like no hygiene here please". The
intention here is to make this explicit in procedural macros to be
forwards-compatible with a hygiene-specifying solution.
* Some of the conversions for `TokenStream` have been simplified a little.
* The `TokenTreeIter` iterator was renamed to `token_stream::IntoIter`.
Overall the hope is that this is the "final pass" at the API of `TokenStream`
and most of `TokenTree` before stabilization. Explicitly left out here is any
changes to `Span`'s API which will likely need to be re-evaluated before
stabilization.
All changes in this PR have already been reflected to the [`proc-macro2`],
`quote`, and `syn` crates. New versions of all these crates have also been
published to crates.io.
Once this lands in nightly I plan on making an internals post again summarizing
the changes made here and also calling on all macro authors to give the APIs a
spin and see how they work. Hopefully pending no major issues we can then have
an FCP to stabilize later this cycle!
[`proc-macro2`]: https://docs.rs/proc-macro2/0.3.1/proc_macro2/
Separate the plugin code from non-plugin code to break a potential cycle in crates.
This will allow us to merge the new libproc_macro_tokens into libproc_macro.
This commit blanket renames the `rustc_macro` infrastructure to `proc_macro`,
which reflects the general consensus of #35900. A follow up PR to Cargo will be
required to purge the `rustc-macro` name as well.
