This extends PR #73293 to handle patterns (Pat). Unlike expressions,
patterns do not support custom attributes, so we only need to capture
tokens during macro_rules! argument parsing.
It has been deny_by_default since 2017 (and warned for some time
before that), so it seems reasonable to promote it.
The specific technical motivation to do this now is to remove a field
from `ParseSess` -- it is a global state, and global state makes
extracting libraries annoying.
Closes#40107
- Move the type parameter from `encode` and `decode` methods to
the trait.
- Remove `UseSpecialized(En|De)codable` traits.
- Remove blanket impls for references.
- Add `RefDecodable` trait to allow deserializing to arena-allocated
references safely.
- Remove ability to (de)serialize HIR.
- Create proc-macros `(Ty)?(En|De)codable` to help implement these new
traits.
By moving `{known,used}_attrs` from `SessionGlobals` to `Session`. This
means they are accessed via the `Session`, rather than via TLS. A few
`Attr` methods and `librustc_ast` functions are now methods of
`Session`.
All of this required passing a `Session` to lots of functions that didn't
already have one. Some of these functions also had arguments removed, because
those arguments could be accessed directly via the `Session` argument.
`contains_feature_attr()` was dead, and is removed.
Some functions were moved from `librustc_ast` elsewhere because they now need
to access `Session`, which isn't available in that crate.
- `entry_point_type()` --> `librustc_builtin_macros`
- `global_allocator_spans()` --> `librustc_metadata`
- `is_proc_macro_attr()` --> `Session`
For consistency with `Attribute::has_name` which doesn't mark the attribute as used either.
Replace all uses of `check_name` with `has_name` outside of rustc
Introduce NonterminalKind for more type-safe mbe parsing
It encapsulate the (part of) the interface between the parser and
macro by example (macro_rules) parser.
The second bit is somewhat more general `parse_ast_fragment`, which is
the reason why we keep some `parse_xxx` functions as public.
It encapsulate the (part of) the interface between the parser and
macro by example (macro_rules) parser.
The second bit is somewhat more general `parse_ast_fragment`, which is
the reason why we keep some `parse_xxx` functions as public.
proc_macro: Add API for tracked access to environment variables
Continuation of https://github.com/rust-lang/rust/pull/71858.
`proc_macro::tracked_env::var` is similar to regular `env::var` called from a proc macro, except that it also adds the accessed variable to depinfo.
Serialize span hygiene data
Fixes#68686Fixes#70963
This PR serializies global hygiene data into both the incremental compilation cache and the crate metadata. This allows hygiene information to be preserved across compilation sessions (both incremental and cross-crate).
When serializing a `SyntaxContext`, we simply write out the raw id from the current compilation session. Whenever we deserialize a `SyntaxContext`, we 'remap' the id to a fresh id in our current compilation session, and load the associated `SyntaxContextData`.
As a result, some 'upstream' `SyntaxContextData` will end up getting duplicated in 'downstream' crates. This only happens when we actually need to use an 'upstream' `SyntaxContext`, which occurs when we deserialize a `Span` that requires it.
We serialize an `ExpnData` into the metadata of the crate which generated it. An `ExpnId` is serialized as a reference into the crate which 'owns' the corresponding `ExpnData`, which avoids duplication in downstream crates.
I've included a macros 2.0 test which requires hygiene serialization to compile successfully.
TODO:
- [x] <strike>Determine how many additional `DefId`s we end up creating for `ExpnId`s - this may be significant for `libcore`, which uses macros heavily. Alternatively, we could try to compute a `DefPathHash` without making a corresponding `DefId` - however, this might significantly complicate the implementation.</strike> (We no longer create `DefId`s)
- [x] Investigate the overhead of duplicating `SyntaxContextData` in crate metadata.
- [x] Investigate how `resolve_crate_root` behaves with deserialized hygiene data - the current logic may be wrong.
- [x] Add additional tests. The effects of this PR are usually only noticeable when working with headache-inducing macro expansions (e.g. macros expanding to macros), so there are lots of corner cases to test.
- [x] Determine what to do about this:
4774f9b523/src/librustc_resolve/build_reduced_graph.rs (L892)
- [x] Determine if we need to do anything here - I think the fact that `src/test/ui/hygiene/cross_crate_hygiene.rs` passes means that this is working.
3d5d0f898c/src/librustc_resolve/imports.rs (L1389-L1392)
It's equivalent to `Ident::from_str_and_span`. The commit also
introduces some more static symbols so that `Ident::new` can be used in
various places instead of `Ident::from_str_and_span`.
The commit also changes `Path::path` from a `&str` to a `Symbol`, which
then allows the lifetime annotation to be removed from `Ty`. Also, the
use of `Symbol` in `Bounds` removes the need for its lifetime
annotation.
There are some structures that are called "globals", but are they global
to a compilation session, and not truly global. I have always found this
highly confusing, so this commit renames them as "session globals" and
adds a comment explaining things.
Also, the commit fixes an unnecessary nesting of `set()` calls
`src/librustc_errors/json/tests.rs`
Rollup of 10 pull requests
Successful merges:
- #73414 (Implement `slice_strip` feature)
- #73564 (linker: Create GNU_EH_FRAME header by default when producing ELFs)
- #73622 (Deny unsafe ops in unsafe fns in libcore)
- #73684 (add spans to injected coverage counters, extract with CoverageData query)
- #73812 (ast_pretty: Pass some token streams and trees by reference)
- #73853 (Add newline to rustc MultiSpan docs)
- #73883 (Compile rustdoc less often.)
- #73885 (Fix wasm32 being broken due to a NodeJS version bump)
- #73903 (Changes required for rustc/cargo to build for iOS targets)
- #73938 (Optimise fast path of checked_ops with `unlikely`)
Failed merges:
r? @ghost
Handle `macro_rules!` tokens consistently across crates
When we serialize a `macro_rules!` macro, we used a 'lowered' `TokenStream` for its body, which has all `Nonterminal`s expanded in-place via `nt_to_tokenstream`. This matters when an 'outer' `macro_rules!` macro expands to an 'inner' `macro_rules!` macro - the inner macro may use tokens captured from the 'outer' macro in its definition.
This means that invoking a foreign `macro_rules!` macro may use a different body `TokenStream` than when the same `macro_rules!` macro is invoked in the same crate. This difference is observable by proc-macros invoked by a `macro_rules!` macro - a `None`-delimited group will be seen in the same-crate case (inserted when convering `Nonterminal`s to the `proc_macro` crate's structs), but no `None`-delimited group in the cross-crate case.
To fix this inconsistency, we now insert `None`-delimited groups when 'lowering' a `Nonterminal` `macro_rules!` body, just as we do in `proc_macro_server`. Additionally, we no longer print extra spaces for `None`-delimited groups - as far as pretty-printing is concerned, they don't exist (only their contents do). This ensures that `Display` output of a `TokenStream` does not depend on which crate a `macro_rules!` macro was invoked from.
This PR is necessary in order to patch the `solana-genesis-programs` for the upcoming hygiene serialization breakage (https://github.com/rust-lang/rust/pull/72121#issuecomment-646924847). The `solana-genesis-programs` crate will need to use a proc macro to re-span certain tokens in a nested `macro_rules!`, which requires us to consistently use a `None`-delimited group.
See `src/test/ui/proc-macro/nested-macro-rules.rs` for an example of the kind of nested `macro_rules!` affected by this crate.
When a `macro_rules!` macro expands to another `macro_rules!` macro, we
may see `None`-delimited groups in odd places when another crate
deserializes the 'inner' macro. This commit 'unwraps' an outer
`None`-delimited group to avoid breaking existing code.
See https://github.com/rust-lang/rust/pull/73569#issuecomment-650860457
for more details.
The proper fix is to handle `None`-delimited groups systematically
throughout the parser, but that will require significant work. In the
meantime, this hack lets us fix important hygiene bugs in macros
expand: More precise locations for expansion-time lints
First commit: a macro expansion doesn't have a `NodeId` associated with it, but it has a parent `DefId` which we can use for linting.
The observable effect is that lints associated with macro expansions can now be `allow`ed at finer-grained level than whole crate.
Second commit: each macro definition has a `NodeId` which we can use for linting, unless that macro definition was decoded from other crate.
Fixes#69977
When we parse a chain of method calls like `foo.a().b().c()`, each
`MethodCallExpr` gets assigned a span that starts at the beginning of
the call chain (`foo`). While this is useful for diagnostics, it means
that `Location::caller` will return the same location for every call
in a call chain.
This PR makes us separately record the span of the function name and
arguments for a method call (e.g. `b()` in `foo.a().b().c()`). This
`Span` is passed through HIR lowering and MIR building to
`TerminatorKind::Call`, where it is used in preference to
`Terminator.source_info.span` when determining `Location::caller`.
This new span is also useful for diagnostics where we want to emphasize
a particular method call - for an example, see
https://github.com/rust-lang/rust/pull/72389#discussion_r436035990
Resolve E0584 conflict
Adds a new error code (`E0761`) to indicate ambiguity in module file names and an accompanying expanded description to resolve a conflict over `E0584`.
Resolves#73116
Add `-Z span-debug` to allow for easier debugging of proc macros
Currently, the `Debug` impl for `proc_macro::Span` just prints out
the byte range. This can make debugging proc macros (either as a crate
author or as a compiler developer) very frustrating, since neither the
actual filename nor the `SyntaxContext` is displayed.
This commit adds a perma-unstable flag `-Z span-debug`. When enabled,
the `Debug` impl for `proc_macro::Span` simply forwards directly to
`rustc_span::Span`. Once #72618 is merged, this will start displaying
actual line numbers.
While `Debug` impls are not subject to Rust's normal stability
guarnatees, we probably shouldn't expose any additional information on
stable until `#![feature(proc_macro_span)]` is stabilized. Otherwise,
we would be providing a 'backdoor' way to access information that's
supposed be behind unstable APIs.
Currently, the `Debug` impl for `proc_macro::Span` just prints out
the byte range. This can make debugging proc macros (either as a crate
author or as a compiler developer) very frustrating, since neither the
actual filename nor the `SyntaxContext` is displayed.
This commit adds a perma-unstable flag `-Z span-debug`. When enabled,
the `Debug` impl for `proc_macro::Span` simply forwards directly to
`rustc_span::Span`. Once #72618 is merged, this will start displaying
actual line numbers.
While `Debug` impls are not subject to Rust's normal stability
guarnatees, we probably shouldn't expose any additional information on
stable until `#![feature(proc_macro_span)]` is stabilized. Otherwise,
we would be providing a 'backdoor' way to access information that's
supposed be behind unstable APIs.
Track devirtualized filenames
Split payload of FileName::Real to track both real and virtualized paths.
(Such splits arise from metadata refs into libstd; the virtualized paths look like `/rustc/1.45.0/src/libstd/io/cursor.rs` rather than `/Users/felixklock/Dev/Mozilla/rust.git/src/libstd/io/cursor.rs`)
This way, we can emit the virtual name into things like the like the StableSourceFileId (as was done back before PR #70642) that ends up in incremental build artifacts, while still using the devirtualized file path when we want to access the file.
Fix#70924
Such splits arise from metadata refs into libstd.
This way, we can (in a follow on commit) continue to emit the virtual name into
things like the like the StableSourceFileId that ends up in incremetnal build
artifacts, while still using the devirtualized file path when we want to access
the file.
Note that this commit is intended to be a refactoring; the actual fix to the bug
in question is in a follow-on commit.
This commit introduces a `Limit` type which is used to ensure that all
comparisons against limits within the compiler are consistent (which can
result in ICEs if they aren't).
Signed-off-by: David Wood <david@davidtw.co>
