Less `TokenTree` cloning
`TokenTreeCursor` has this comment on it:
```
// FIXME: Many uses of this can be replaced with by-reference iterator to avoid clones.
```
This PR completes that FIXME. It doesn't have much perf effect, but at least we now know that.
r? `@petrochenkov`
This is surprising, but the new comment explains why. It's a logical
conclusion in the drive to avoid `TokenTree` clones.
`TokenTreeCursor` is now only used within `Parser`. It's still needed
due to `replace_prev_and_rewind`.
On nightly, dump ICE backtraces to disk
Implement rust-lang/compiler-team#578.
When an ICE is encountered on nightly releases, the new rustc panic handler will also write the contents of the backtrace to disk. If any `delay_span_bug`s are encountered, their backtrace is also added to the file. The platform and rustc version will also be collected.
<img width="1032" alt="Screenshot 2023-03-03 at 2 13 25 PM" src="https://user-images.githubusercontent.com/1606434/222842420-8e039740-4042-4563-b31d-599677171acf.png">
The current behavior will *always* write to disk on nightly builds, regardless of whether the backtrace is printed to the terminal, unless the environment variable `RUSTC_ICE_DISK_DUMP` is set to `0`. This is a compromise and can be changed.
Implement rust-lang/compiler-team#578.
When an ICE is encountered on nightly releases, the new rustc panic
handler will also write the contents of the backtrace to disk. If any
`delay_span_bug`s are encountered, their backtrace is also added to the
file. The platform and rustc version will also be collected.
Previously it removed all other attributes from the crate root.
Now it removes only attributes below itself.
So it becomes possible to configure some global crate properties even for fully unconfigured crates.
Remember names of `cfg`-ed out items to mention them in diagnostics
# Examples
## `serde::Deserialize` without the `derive` feature (a classic beginner mistake)
I had to slightly modify serde so that it uses explicit re-exports instead of a glob re-export. (Update: a serde PR was merged that adds the manual re-exports)
```
error[E0433]: failed to resolve: could not find `Serialize` in `serde`
--> src/main.rs:1:17
|
1 | #[derive(serde::Serialize)]
| ^^^^^^^^^ could not find `Serialize` in `serde`
|
note: crate `serde` has an item named `Serialize` but it is inactive because its cfg predicate evaluated to false
--> /home/gh-Nilstrieb/.cargo/registry/src/index.crates.io-6f17d22bba15001f/serde-1.0.160/src/lib.rs:343:1
|
343 | #[cfg(feature = "serde_derive")]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
344 | pub use serde_derive::{Deserialize, Serialize};
| ^^^^^^^^^
= note: the item is gated behind the `serde_derive` feature
= note: see https://doc.rust-lang.org/cargo/reference/features.html for how to activate a crate's feature
```
(the suggestion is not ideal but that's serde's fault)
I already tested the metadata size impact locally by compiling the `windows` crate without any features. `800k` -> `809k`
r? `@ghost`
`#[cfg]`s are frequently used to gate crate content behind cargo
features. This can lead to very confusing errors when features are
missing. For example, `serde` doesn't have the `derive` feature by
default. Therefore, `serde::Serialize` fails to resolve with a generic
error, even though the macro is present in the docs.
This commit adds a list of all stripped item names to metadata. This is
filled during macro expansion and then, through a fed query, persisted
in metadata. The downstream resolver can then access the metadata to
look at possible candidates for mentioning in the errors.
This slightly increases metadata (800k->809k for the feature-heavy
windows crate), but not enough to really matter.
Each of `{D,Subd}iagnosticMessage::{Str,Eager}` has a comment:
```
// FIXME(davidtwco): can a `Cow<'static, str>` be used here?
```
This commit answers that question in the affirmative. It's not the most
compelling change ever, but it might be worth merging.
This requires changing the `impl<'a> From<&'a str>` impls to `impl
From<&'static str>`, which involves a bunch of knock-on changes that
require/result in call sites being a little more precise about exactly
what kind of string they use to create errors, and not just `&str`. This
will result in fewer unnecessary allocations, though this will not have
any notable perf effects given that these are error paths.
Note that I was lazy within Clippy, using `to_string` in a few places to
preserve the existing string imprecision. I could have used `impl
Into<{D,Subd}iagnosticMessage>` in various places as is done in the
compiler, but that would have required changes to *many* call sites
(mostly changing `&format("...")` to `format!("...")`) which didn't seem
worthwhile.
Error message all end up passing into a function as an `impl
Into<{D,Subd}iagnosticMessage>`. If an error message is creatd as
`&format("...")` that means we allocate a string (in the `format!`
call), then take a reference, and then clone (allocating again) the
reference to produce the `{D,Subd}iagnosticMessage`, which is silly.
This commit removes the leading `&` from a lot of these cases. This
means the original `String` is moved into the
`{D,Subd}iagnosticMessage`, avoiding the double allocations. This
requires changing some function argument types from `&str` to `String`
(when all arguments are `String`) or `impl
Into<{D,Subd}iagnosticMessage>` (when some arguments are `String` and
some are `&str`).
try to downgrade Arc -> Lrc -> Rc -> no-Rc in few places
Expecting this be not slower on non-parallel compiler and probably faster on parallel (checked that this PR builds on it).
Currently a `{D,Subd}iagnosticMessage` can be created from any type that
impls `Into<String>`. That includes `&str`, `String`, and `Cow<'static,
str>`, which are reasonable. It also includes `&String`, which is pretty
weird, and results in many places making unnecessary allocations for
patterns like this:
```
self.fatal(&format!(...))
```
This creates a string with `format!`, takes a reference, passes the
reference to `fatal`, which does an `into()`, which clones the
reference, doing a second allocation. Two allocations for a single
string, bleh.
This commit changes the `From` impls so that you can only create a
`{D,Subd}iagnosticMessage` from `&str`, `String`, or `Cow<'static,
str>`. This requires changing all the places that currently create one
from a `&String`. Most of these are of the `&format!(...)` form
described above; each one removes an unnecessary static `&`, plus an
allocation when executed. There are also a few places where the existing
use of `&String` was more reasonable; these now just use `clone()` at
the call site.
As well as making the code nicer and more efficient, this is a step
towards possibly using `Cow<'static, str>` in
`{D,Subd}iagnosticMessage::{Str,Eager}`. That would require changing
the `From<&'a str>` impls to `From<&'static str>`, which is doable, but
I'm not yet sure if it's worthwhile.
My type ascription
Oh rip it out
Ah
If you think we live too much then
You can sacrifice diagnostics
Don't mix your garbage
Into my syntax
So many weird hacks keep diagnostics alive
Yet I don't even step outside
So many bad diagnostics keep tyasc alive
Yet tyasc doesn't even bother to survive!
Added byte position range for `proc_macro::Span`
Currently, the [`Debug`](https://doc.rust-lang.org/beta/proc_macro/struct.Span.html#impl-Debug-for-Span) implementation for [`proc_macro::Span`](https://doc.rust-lang.org/beta/proc_macro/struct.Span.html#) calls the debug function implemented in the trait implementation of `server::Span` for the type `Rustc` in the `rustc-expand` crate.
The current implementation, of the referenced function, looks something like this:
```rust
fn debug(&mut self, span: Self::Span) -> String {
if self.ecx.ecfg.span_debug {
format!("{:?}", span)
} else {
format!("{:?} bytes({}..{})", span.ctxt(), span.lo().0, span.hi().0)
}
}
```
It returns the byte position of the [`Span`](https://doc.rust-lang.org/beta/proc_macro/struct.Span.html#) as an interpolated string.
Because this is currently the only way to get a spans position in the file, I might lead someone, who is interested in this information, to parsing this interpolated string back into a range of bytes, which I think is a very non-rusty way.
The proposed `position()`, method implemented in this PR, gives the ability to directly get this info.
It returns a [`std::ops::Range`](https://doc.rust-lang.org/std/ops/struct.Range.html#) wrapping the lowest and highest byte of the [`Span`](https://doc.rust-lang.org/beta/proc_macro/struct.Span.html#).
I put it behind the `proc_macro_span` feature flag because many of the other functions that have a similar footprint also are annotated with it, I don't actually know if this is right.
It would be great if somebody could take a look at this, thank you very much in advanced.
Add `rustc_fluent_macro` to decouple fluent from `rustc_macros`
Fluent, with all the icu4x it brings in, takes quite some time to compile. `fluent_messages!` is only needed in further downstream rustc crates, but is blocking more upstream crates like `rustc_index`. By splitting it out, we allow `rustc_macros` to be compiled earlier, which speeds up `x check compiler` by about 5 seconds (and even more after the needless dependency on `serde_json` is removed from `rustc_data_structures`).
Fluent, with all the icu4x it brings in, takes quite some time to
compile. `fluent_messages!` is only needed in further downstream rustc
crates, but is blocking more upstream crates like `rustc_index`. By
splitting it out, we allow `rustc_macros` to be compiled earlier, which
speeds up `x check compiler` by about 5 seconds (and even more after the
needless dependency on `serde_json` is removed from
`rustc_data_structures`).
Improve the error message when forwarding a matched fragment to another macro
Adds a link to [Forwarding a matched fragment](https://doc.rust-lang.org/nightly/reference/macros-by-example.html#forwarding-a-matched-fragment) section of the Rust Reference, and suggests a possible fix (using `:tt` instead in the macro definition).
Also removes typos from the original message, it should be `:lifetime` instead of `$lifetime`.
## Motivation
When trying to write a macro which uses a literal in the matcher from the outer macro, like the following one, using a fragment specified that isn't one of `:ident`, `:lifetime`, or `:tt` currently results in a hard to understand message.
```rs
macro_rules! make_t_for_all_tokens {
($($name:literal as $variant:expr,)*) => {
macro_rules! t {
$(
($name) => {
$variant
};
)*
}
};
}
make_t_for_all_tokens! {
"fn" as Token::Fn,
"return" as Token::Return,
"let" as Token::Let,
}
// This creates
//
// macro_rules! t {
// ("fn") => {
// Token::Fn
// };
// ("return") => {
// Token::Return
// };
// ("let") => {
// Token::Let
// };
// }
t!["fn"];
```
### Before
```
error: no rules expected the token `"fn"`
--> src/main.rs:103:10
|
32 | macro_rules! t {
| -------------- when calling this macro
...
103 | t!["fn"];
| ^^^^ no rules expected this token in macro call
|
note: while trying to match `"fn"`
--> src/main.rs:34:6
|
34 | ($name) => {
| ^^^^^
...
58 | / make_t_for_all_tokens! {
59 | | "fn" as Token::Fn,
60 | | "return" as Token::Return,
61 | | "let" as Token::Let,
62 | | }
| |_- in this macro invocation
= note: captured metavariables except for `$tt`, `$ident` and `$lifetime` cannot be compared to other tokens
= note: this error originates in the macro `make_t_for_all_tokens` (in Nightly builds, run with -Z macro-backtrace for more info)
```
### After
```
error: no rules expected the token `"fn"`
--> src/main.rs:103:10
|
32 | macro_rules! t {
| -------------- when calling this macro
...
103 | t!["fn"];
| ^^^^ no rules expected this token in macro call
|
note: while trying to match `"fn"`
--> src/main.rs:34:6
|
34 | ($name) => {
| ^^^^^
...
58 | / make_t_for_all_tokens! {
59 | | "fn" as Token::Fn,
60 | | "return" as Token::Return,
61 | | "let" as Token::Let,
62 | | }
| |_- in this macro invocation
= note: captured metavariables except for `:tt`, `:ident` and `:lifetime` cannot be compared to other tokens
= note: see https://doc.rust-lang.org/nightly/reference/macros-by-example.html#forwarding-a-matched-fragment for more information
= help: try using `:tt` instead in the macro definition
= note: this error originates in the macro `make_t_for_all_tokens` (in Nightly builds, run with -Z macro-backtrace for more info)
```
## Unresolved questions
- Preferrably the suggestion should be attached to the `$name:literal` part of the outer macro, instead of being in the notes section at the end. But I'm not familiar with how the compiler works at all, and I have no idea how to approach this kind of solution.
- `@Nilstrieb` raised a question that the suggestion of adding `:tt` isn't accurate when there's more than `tt` being matched, for example when the input is an `item`.
Adds a link to the relevant part of The Rust Reference in the eror
message, and suggests a possible fix (replacing the fragment specifier
with :tt in the macro definition).
Fixes typos in the original message.
Signed-off-by: Lena Milizé <me@lvmn.org>
It partially expands crate attributes before the main expansion pass (without modifying the crate), and the produced preliminary crate attribute list is used for querying a few attributes that are required very early.
Crate-level cfg attributes are then expanded normally during the main expansion pass, like attributes on any other nodes.
This makes it easier to open the messages file while developing on features.
The commit was the result of automatted changes:
for p in compiler/rustc_*; do mv $p/locales/en-US.ftl $p/messages.ftl; rmdir $p/locales; done
for p in compiler/rustc_*; do sed -i "s#\.\./locales/en-US.ftl#../messages.ftl#" $p/src/lib.rs; done
Rollup of 9 pull requests
Successful merges:
- #104363 (Make `unused_allocation` lint against `Box::new` too)
- #106633 (Stabilize `nonzero_min_max`)
- #106844 (allow negative numeric literals in `concat!`)
- #108071 (Implement goal caching with the new solver)
- #108542 (Force parentheses around `match` expression in binary expression)
- #108690 (Place size limits on query keys and values)
- #108708 (Prevent overflow through Arc::downgrade)
- #108739 (Prevent the `start_bx` basic block in codegen from having two `Builder`s at the same time)
- #108806 (Querify register_tools and post-expansion early lints)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
Querify register_tools and post-expansion early lints
The 2 extra queries correspond to code that happen before and after macro expansion, and don't need the resolver to exist.
After removing the `map_in_place` method, which isn't much use because
modifying every element in a collection such as a `Vec` can be done
trivially with iteration.
Extend `CodegenBackend` trait with a function returning the translation
resources from the codegen backend, which can be added to the complete
list of resources provided to the emitter.
Signed-off-by: David Wood <david.wood@huawei.com>
Instead of loading the Fluent resources for every crate in
`rustc_error_messages`, each crate generates typed identifiers for its
own diagnostics and creates a static which are pulled together in the
`rustc_driver` crate and provided to the diagnostic emitter.
Signed-off-by: David Wood <david.wood@huawei.com>
Introduce `-Zterminal-urls` to use OSC8 for error codes
Terminals supporting the OSC8 Hyperlink Extension can support inline anchors where the text is user defineable but clicking on it opens a browser to a specified URLs, just like `<a href="URL">` does in HTML.
https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda
Terminals supporting the OSC8 Hyperlink Extension can support inline
anchors where the text is user defineable but clicking on it opens a
browser to a specified URLs, just like `<a href="URL">` does in HTML.
https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda
Fix syntax in `-Zunpretty-expanded` output for derived `PartialEq`.
If you do `derive(PartialEq)` on a packed struct, the output shown by `-Zunpretty=expanded` includes expressions like this:
```
{ self.x } == { other.x }
```
This is invalid syntax. This doesn't break compilation, because the AST nodes are constructed within the compiler. But it does mean anyone using `-Zunpretty=expanded` output as a guide for hand-written impls could get a nasty surprise.
This commit fixes things by instead using this form:
```
({ self.x }) == ({ other.x })
```
r? ``@RalfJung``
If you do `derive(PartialEq)` on a packed struct, the output shown by
`-Zunpretty=expanded` includes expressions like this:
```
{ self.x } == { other.x }
```
This is invalid syntax. This doesn't break compilation, because the AST
nodes are constructed within the compiler. But it does mean anyone using
`-Zunpretty=expanded` output as a guide for hand-written impls could get
a nasty surprise.
This commit fixes things by instead using this form:
```
({ self.x }) == ({ other.x })
```
A recent PR increased the size, which caused regressions. This uses the
existing generic infrastructure to differentiate between the hot path
and the diagnostics path.
Encode spans relative to the enclosing item -- enable on nightly
Follow-up to #84373 with the flag `-Zincremental-relative-spans` set by default.
This PR seeks to remove one of the main shortcomings of incremental: the handling of spans.
Changing the contents of a function may require redoing part of the compilation process for another function in another file because of span information is changed.
Within one file: all the spans in HIR change, so typechecking had to be re-done.
Between files: spans of associated types/consts/functions change, so type-based resolution needs to be re-done (hygiene information is stored in the span).
The flag `-Zincremental-relative-spans` encodes local spans relative to the span of an item, stored inside the `source_span` query.
Trap: stashed diagnostics are referenced by the "raw" span, so stealing them requires to remove the span's parent.
In order to avoid too much traffic in the span interner, span encoding uses the `ctxt_or_tag` field to encode:
- the parent when the `SyntaxContext` is 0;
- the `SyntaxContext` when the parent is `None`.
Even with this, the PR creates a lot of traffic to the Span interner, when a Span has both a LocalDefId parent and a non-root SyntaxContext. They appear in lowering, when we add a parent to all spans, including those which come from macros, and during inlining when we mark inlined spans.
The last commit changes how queries of `LocalDefId` manage their cache. I can put this in a separate PR if required.
Possible future directions:
- validate that all spans are marked in HIR validation;
- mark macro-expanded spans relative to the def-site and not the use-site.
Properly calculate best failure in macro matching
Previously, we used spans. This was not good. Sometimes, the span of the token that failed to match may come from a position later in the file which has been transcribed into a token stream way earlier in the file. If precisely this token fails to match, we think that it was the best match because its span is so high, even though other arms might have gotten further in the token stream.
We now try to properly use the location in the token stream.
This needs a little cleanup as the `best_failure` field is getting out of hand but it should be mostly good to go. I hope I didn't violate too many abstraction boundaries..
Previously, we used spans. This was not good. Sometimes, the span of the
token that failed to match may come from a position later in the file
which has been transcribed into a token stream way earlier in the file.
If precisely this token fails to match, we think that it was the best
match because its span is so high, even though other arms might have
gotten further in the token stream.
We now try to properly use the location in the token stream.
Remove `token::Lit` from `ast::MetaItemLit`.
Currently `ast::MetaItemLit` represents the literal kind twice. This PR removes that redundancy. Best reviewed one commit at a time.
r? `@petrochenkov`
compiler: remove unnecessary imports and qualified paths
Some of these imports were necessary before Edition 2021, others were already in the prelude.
I hope it's fine that this PR is so spread-out across files :/
This migrates everything but the `mbe` and `proc_macro` modules. It also
contains a few cleanups and drive-by/accidental diagnostic improvements
which can be seen in the diff for the UI tests.
This is required to distinguish between cooked and raw byte string
literals in an `ast::LitKind`, without referring to an adjacent
`token::Lit`. It's a prerequisite for the next commit.
There is code for converting `Attribute` (syntactic) to `MetaItem`
(semantic). There is also code for the reverse direction. The reverse
direction isn't really necessary; it's currently only used when
generating attributes, e.g. in `derive` code.
This commit adds some new functions for creating `Attributes`s directly,
without involving `MetaItem`s: `mk_attr_word`, `mk_attr_name_value_str`,
`mk_attr_nested_word`, and
`ExtCtxt::attr_{word,name_value_str,nested_word}`.
These new methods replace the old functions for creating `Attribute`s:
`mk_attr_inner`, `mk_attr_outer`, and `ExtCtxt::attribute`. Those
functions took `MetaItem`s as input, and relied on many other functions
that created `MetaItems`, which are also removed: `mk_name_value_item`,
`mk_list_item`, `mk_word_item`, `mk_nested_word_item`,
`{MetaItem,MetaItemKind,NestedMetaItem}::token_trees`,
`MetaItemKind::attr_args`, `MetaItemLit::{from_lit_kind,to_token}`,
`ExtCtxt::meta_word`.
Overall this cuts more than 100 lines of code and makes thing simpler.
`check_builtin_attribute` calls `parse_meta` to convert an `Attribute`
to a `MetaItem`, which it then checks. However, many callers of
`check_builtin_attribute` start with a `MetaItem`, and then convert it
to an `Attribute` by calling `cx.attribute(meta_item)`. This `MetaItem`
to `Attribute` to `MetaItem` conversion is silly.
This commit adds a new function `check_builtin_meta_item`, which can be
called instead from these call sites. `check_builtin_attribute` also now
calls it. The commit also renames `check_meta` as `check_attr` to better
match its arguments.
Change multiline span ASCII art visual order
Tweak the ASCII art for nested multiline spans so that we minimize line overlaps.
Partially addresses https://github.com/rust-lang/rust/issues/61017.
`Lit::from_included_bytes` calls `Lit::from_lit_kind`, but the two call
sites only need the resulting `token::Lit`, not the full `ast::Lit`.
This commit changes those call sites to use `LitKind::to_token_lit`,
which means `from_included_bytes` can be removed.
Split `MacArgs` in two.
`MacArgs` is an enum with three variants: `Empty`, `Delimited`, and `Eq`. It's used in two ways:
- For representing attribute macro arguments (e.g. in `AttrItem`), where all three variants are used.
- For representing function-like macros (e.g. in `MacCall` and `MacroDef`), where only the `Delimited` variant is used.
In other words, `MacArgs` is used in two quite different places due to them having partial overlap. I find this makes the code hard to read. It also leads to various unreachable code paths, and allows invalid values (such as accidentally using `MacArgs::Empty` in a `MacCall`).
This commit splits `MacArgs` in two:
- `DelimArgs` is a new struct just for the "delimited arguments" case. It is now used in `MacCall` and `MacroDef`.
- `AttrArgs` is a renaming of the old `MacArgs` enum for the attribute macro case. Its `Delimited` variant now contains a `DelimArgs`.
Various other related things are renamed as well.
These changes make the code clearer, avoids several unreachable paths, and disallows the invalid values.
r? `@petrochenkov`
`MacArgs` is an enum with three variants: `Empty`, `Delimited`, and `Eq`. It's
used in two ways:
- For representing attribute macro arguments (e.g. in `AttrItem`), where all
three variants are used.
- For representing function-like macros (e.g. in `MacCall` and `MacroDef`),
where only the `Delimited` variant is used.
In other words, `MacArgs` is used in two quite different places due to them
having partial overlap. I find this makes the code hard to read. It also leads
to various unreachable code paths, and allows invalid values (such as
accidentally using `MacArgs::Empty` in a `MacCall`).
This commit splits `MacArgs` in two:
- `DelimArgs` is a new struct just for the "delimited arguments" case. It is
now used in `MacCall` and `MacroDef`.
- `AttrArgs` is a renaming of the old `MacArgs` enum for the attribute macro
case. Its `Delimited` variant now contains a `DelimArgs`.
Various other related things are renamed as well.
These changes make the code clearer, avoids several unreachable paths, and
disallows the invalid values.
Use `token::Lit` in `ast::ExprKind::Lit`.
Instead of `ast::Lit`.
Literal lowering now happens at two different times. Expression literals are lowered when HIR is crated. Attribute literals are lowered during parsing.
r? `@petrochenkov`
Instead of `ast::Lit`.
Literal lowering now happens at two different times. Expression literals
are lowered when HIR is crated. Attribute literals are lowered during
parsing.
This commit changes the language very slightly. Some programs that used
to not compile now will compile. This is because some invalid literals
that are removed by `cfg` or attribute macros will no longer trigger
errors. See this comment for more details:
https://github.com/rust-lang/rust/pull/102944#issuecomment-1277476773
Show note where the macro failed to match
When feeding the wrong tokens, it used to fail with a very generic error that wasn't very helpful. This change tries to help by noting where specifically the matching went wrong.
```rust
macro_rules! uwu {
(a a a b) => {};
}
uwu! { a a a c }
```
```diff
error: no rules expected the token `c`
--> macros.rs:5:14
|
1 | macro_rules! uwu {
| ---------------- when calling this macro
...
4 | uwu! { a a a c }
| ^ no rules expected this token in macro call
|
+note: while trying to match `b`
+ --> macros.rs:2:12
+ |
+2 | (a a a b) => {};
+ | ^
```
Delay `include_bytes` to AST lowering
Hopefully addresses #65818.
This PR introduces a new `ExprKind::IncludedBytes` which stores the path and bytes of a file included with `include_bytes!()`. We can then create a literal from the bytes during AST lowering, which means we don't need to escape the bytes into valid UTF8 which is the cause of most of the overhead of embedding large binary blobs.
Add the `#[derive_const]` attribute
Closes#102371. This is a minimal patchset for the attribute to work. There are no restrictions on what traits this attribute applies to.
r? `````@oli-obk`````
For now, we only collect the small info for the `best_failure`, but
using this tracker, we can easily extend it in the future to track
things with more performance overhead.
We cannot retry cases where the macro failed with a parser error that
was emitted already, as that would cause us to emit the same error to
the user twice.
This moves out the matching part of expansion into a new function. This
function will try to match the macro and return an error if it failed to
match. A tracker can be used to get more information about the matching.
Track where diagnostics were created.
This implements the `-Ztrack-diagnostics` flag, which uses `#[track_caller]` to track where diagnostics are created. It is meant as a debugging tool much like `-Ztreat-err-as-bug`.
For example, the following code...
```rust
struct A;
struct B;
fn main(){
let _: A = B;
}
```
...now emits the following error message:
```
error[E0308]: mismatched types
--> src\main.rs:5:16
|
5 | let _: A = B;
| - ^ expected struct `A`, found struct `B`
| |
| expected due to this
-Ztrack-diagnostics: created at compiler\rustc_infer\src\infer\error_reporting\mod.rs:2275:31
```
Add flag to forbid recovery in the parser
To start the effort of fixing #103534, this adds a new flag to the parser, which forbids the parser from doing recovery, which it shouldn't do in macros.
This doesn't add any new checks for recoveries yet and is just here to bikeshed the names for the functions here before doing more.
r? `@compiler-errors`
Only apply `ProceduralMasquerade` hack to older versions of `rental`
The latest version of `rental` (v0.5.6) contains a fix that allows it to
compile without relying on the pretty-print back-compat hack.
Hopefully, there are no longer any crates relying on the affected
versions of the (much less popular) `procedural-masquerade` crate. This
should allow us to target the pretty-print back-compat hack specifically
to older versions of `rental`, and specifically mention upgrading to
`rental` v0.5.6 in the lint message.
The latest version of `rental` (v0.5.6) contains a fix that allows it to
compile without relying on the pretty-print back-compat hack.
Hopefully, there are no longer any crates relying on the affected
versions of the (much less popular) `procedural-masquerade` crate. This
should allow us to target the pretty-print back-compat hack specifically
to older versions of `rental`, and specifically mention upgrading to
`rental` v0.5.6 in the lint message.
Remove `TokenStreamBuilder`
`TokenStreamBuilder` is used to combine multiple token streams. It can be removed, leaving the code a little simpler and a little faster.
r? `@Aaron1011`
`TokenStreamBuilder` exists to concatenate multiple `TokenStream`s
together. This commit removes it, and moves the concatenation
functionality directly into `TokenStream`, via two new methods
`push_tree` and `push_stream`. This makes things both simpler and
faster.
`push_tree` is particularly important. `TokenStreamBuilder` only had a
single `push` method, which pushed a stream. But in practice most of the
time we push a single token tree rather than a stream, and `push_tree`
avoids the need to build a token stream with a single entry (which
requires two allocations, one for the `Lrc` and one for the `Vec`).
The main `push_tree` use arises from a change to one of the `ToInternal`
impls in `proc_macro_server.rs`. It now returns a `SmallVec` instead of
a `TokenStream`. This return value is then iterated over by
`concat_trees`, which does `push_tree` on each element. Furthermore, the
use of `SmallVec` avoids more allocations, because there is always only
one or two token trees.
Note: the removed `TokenStreamBuilder::push` method had some code to
deal with a quadratic blowup case from #57735. This commit removes the
code. I tried and failed to reproduce the blowup from that PR, before
and after this change. Various other changes have happened to
`TokenStreamBuilder` in the meantime, so I suspect the original problem
is no longer relevant, though I don't have proof of this. Generally
speaking, repeatedly extending a `Vec` without pre-determining its
capacity is *not* quadratic. It's also incredibly common, within rustc
and many other Rust programs, so if there were performance problems
there you'd think it would show up in other places, too.
`TokenTree::Punct` is handled outside the `match`. This commits moves it
inside the `match`, avoiding the need for the `return`s and making it
easier to read.
