ba6275b6a9
Implement token-based handling of attributes during expansion This PR modifies the macro expansion infrastructure to handle attributes in a fully token-based manner. As a result: * Derives macros no longer lose spans when their input is modified by eager cfg-expansion. This is accomplished by performing eager cfg-expansion on the token stream that we pass to the derive proc-macro * Inner attributes now preserve spans in all cases, including when we have multiple inner attributes in a row. This is accomplished through the following changes: * New structs `AttrAnnotatedTokenStream` and `AttrAnnotatedTokenTree` are introduced. These are very similar to a normal `TokenTree`, but they also track the position of attributes and attribute targets within the stream. They are built when we collect tokens during parsing. An `AttrAnnotatedTokenStream` is converted to a regular `TokenStream` when we invoke a macro. * Token capturing and `LazyTokenStream` are modified to work with `AttrAnnotatedTokenStream`. A new `ReplaceRange` type is introduced, which is created during the parsing of a nested AST node to make the 'outer' AST node aware of the attributes and attribute target stored deeper in the token stream. * When we need to perform eager cfg-expansion (either due to `#[derive]` or `#[cfg_eval]`), we tokenize and reparse our target, capturing additional information about the locations of `#[cfg]` and `#[cfg_attr]` attributes at any depth within the target. This is a performance optimization, allowing us to perform less work in the typical case where captured tokens never have eager cfg-expansion run. |
||
|---|---|---|
| .. | ||
| assembly | ||
| auxiliary | ||
| codegen | ||
| codegen-units | ||
| debuginfo | ||
| incremental | ||
| mir-opt | ||
| pretty | ||
| run-make | ||
| run-make-fulldeps | ||
| run-pass-valgrind | ||
| rustdoc | ||
| rustdoc-gui | ||
| rustdoc-js | ||
| rustdoc-js-std | ||
| rustdoc-json | ||
| rustdoc-ui | ||
| rustfix | ||
| ui | ||
| ui-fulldeps | ||
| COMPILER_TESTS.md | ||