This commit adds explicit imp blocks to ensure that all publicly exported types
(except simple enums) are not `Send` nor `Sync` in the `proc_macro` crate.
cc #38356
proc_macro: Generalize `FromIterator` impl
While never intended to be stable we forgot that trait impls are insta-stable!
This construction of `FromIterator` wasn't our first choice of how to stabilize
the impl but our hands are tied at this point, so revert back to the original
definition of `FromIterator` before #49597Closes#49725
A span covering a single byte, such as for an operator `+` token, should
print as e.g. `80..81` rather than `80...81`. The lo end of the range is
inclusive and the hi end is exclusive.
This commit improves the `fmt::Debug` output of `proc_macro` data structures by
primarily focusing on the representation exposed by `proc_macro` rather than the
compiler's own internal representation. This cuts down quite a bit on assorted
wrapper types and ensure a relatively clean output.
Closes#49720
While never intended to be stable we forgot that trait impls are insta-stable!
This construction of `FromIterator` wasn't our first choice of how to stabilize
the impl but our hands are tied at this point, so revert back to the original
definition of `FromIterator` before #49597Closes#49725
* Expand `!` tokens for inner doc comments
* Trim leading doc comment decoration in the string literal
Both of these should help bring the expansion inline with what `macro_rules!`
already does.
Closes#49655Closes#49656
This commit is a reorganization of the `proc_macro` crate's public user-facing
API. This is the result of a number of discussions at the recent Rust All-Hands
where we're hoping to get the `proc_macro` crate into ship shape for
stabilization of a subset of its functionality in the Rust 2018 release.
The reorganization here is motivated by experiences from the `proc-macro2`,
`quote`, and `syn` crates on crates.io (and other crates which depend on them).
The main focus is future flexibility along with making a few more operations
consistent and/or fixing bugs. A summary of the changes made from today's
`proc_macro` API is:
* The `TokenNode` enum has been removed and the public fields of `TokenTree`
have also been removed. Instead the `TokenTree` type is now a public enum
(what `TokenNode` was) and each variant is an opaque struct which internally
contains `Span` information. This makes the various tokens a bit more
consistent, require fewer wrappers, and otherwise provides good
future-compatibility as opaque structs are easy to modify later on.
* `Literal` integer constructors have been expanded to be unambiguous as to what
they're doing and also allow for more future flexibility. Previously
constructors like `Literal::float` and `Literal::integer` were used to create
unsuffixed literals and the concrete methods like `Literal::i32` would create
a suffixed token. This wasn't immediately clear to all users (the
suffixed/unsuffixed aspect) and having *one* constructor for unsuffixed
literals required us to pick a largest type which may not always be true. To
fix these issues all constructors are now of the form
`Literal::i32_unsuffixed` or `Literal::i32_suffixed` (for all integral types).
This should allow future compatibility as well as being immediately clear
what's suffixed and what isn't.
* Each variant of `TokenTree` internally contains a `Span` which can also be
configured via `set_span`. For example `Literal` and `Term` now both
internally contain a `Span` rather than having it stored in an auxiliary
location.
* Constructors of all tokens are called `new` now (aka `Term::intern` is gone)
and most do not take spans. Manufactured tokens typically don't have a fresh
span to go with them and the span is purely used for error-reporting
**except** the span for `Term`, which currently affects hygiene. The default
spans for all these constructed tokens is `Span::call_site()` for now.
The `Term` type's constructor explicitly requires passing in a `Span` to
provide future-proofing against possible hygiene changes. It's intended that a
first pass of stabilization will likely only stabilize `Span::call_site()`
which is an explicit opt-in for "I would like no hygiene here please". The
intention here is to make this explicit in procedural macros to be
forwards-compatible with a hygiene-specifying solution.
* Some of the conversions for `TokenStream` have been simplified a little.
* The `TokenTreeIter` iterator was renamed to `token_stream::IntoIter`.
Overall the hope is that this is the "final pass" at the API of `TokenStream`
and most of `TokenTree` before stabilization. Explicitly left out here is any
changes to `Span`'s API which will likely need to be re-evaluated before
stabilization.
All changes in this PR have already been reflected to the [`proc-macro2`],
`quote`, and `syn` crates. New versions of all these crates have also been
published to crates.io.
Once this lands in nightly I plan on making an internals post again summarizing
the changes made here and also calling on all macro authors to give the APIs a
spin and see how they work. Hopefully pending no major issues we can then have
an FCP to stabilize later this cycle!
[`proc-macro2`]: https://docs.rs/proc-macro2/0.3.1/proc_macro2/
This commit tweaks the tokenization of a doc comment to use `#[doc = "..."]`
like `macro_rules!` does (instead of treating it as a `Literal` token).
Additionally it fixes treatment of negative literals in the compiler, for
exapmle `Literal::i32(-1)`. The current fix is a bit of a hack around the
current compiler implementation, providing a fix at the proc-macro layer rather
than the libsyntax layer.
Remove experimental -Zremap-path-prefix-from/to, and replace it with
the stabilized --remap-path-prefix=from=to variant.
This is an implementation for issue of #41555.
macros: improve 1.0/2.0 interaction
This PR supports using unhygienic macros from hygienic macros without breaking the latter's hygiene.
```rust
// crate A:
#[macro_export]
macro_rules! m1 { () => {
f(); // unhygienic: this macro needs `f` in its environment
fn g() {} // (1) unhygienic: `g` is usable outside the macro definition
} }
// crate B:
#![feature(decl_macro)]
extern crate A;
use A::m1;
macro m2() {
fn f() {} // (2)
m1!(); // After this PR, `f()` in the expansion resolves to (2), not (3)
g(); // After this PR, this resolves to `fn g() {}` from the above expansion.
// Today, it is a resolution error.
}
fn test() {
fn f() {} // (3)
m2!(); // Today, `m2!()` can see (3) even though it should be hygienic.
fn g() {} // Today, this conflicts with `fn g() {}` from the expansion, even though it should be hygienic.
}
```
Once this PR lands, you can make an existing unhygienic macro hygienic by wrapping it in a hygienic macro. There is an [example](b766fa887d) of this in the tests.
r? @nrc
Proc macro spans serve two mostly unrelated purposes: controlling name
resolution and controlling error messages. It can be useful to mix the
name resolution behavior of one span with the line/column error message
locations of a different span.
In particular, consider the case of a trait brought into scope within
the def_site of a custom derive. I want to invoke trait methods on the
fields of the user's struct. If the field type does not implement the
right trait, I want the error message to underline the corresponding
struct field.
Generating the method call with the def_site span is not ideal -- it
compiles and runs but error messages sadly always point to the derive
attribute like we saw with Macros 1.1.
```
|
4 | #[derive(HeapSize)]
| ^^^^^^^^
```
Generating the method call with the same span as the struct field's
ident or type is not correct -- it shows the right underlines but fails
to resolve to the trait in scope at the def_site.
```
|
7 | bad: std:🧵:Thread,
| ^^^^^^^^^^^^^^^^^^^^^^^^
```
The correct span for the method call is one that combines the def_site's
name resolution with the struct field's line/column.
```
field.span.resolved_at(Span::def_site())
// equivalently
Span::def_site().located_at(field.span)
```
Adding both because which one is more natural will depend on context.
