Make `TokenStream` less recursive.
`TokenStream` is currently recursive in *two* ways:
- the `TokenTree` variant contains a `ThinTokenStream`, which can
contain a `TokenStream`;
- the `TokenStream` variant contains a `Vec<TokenStream>`.
The latter is not necessary and causes significant complexity. This
commit replaces it with the simpler `Vec<(TokenTree, IsJoint)>`.
This reduces complexity significantly. In particular, `StreamCursor` is
eliminated, and `Cursor` becomes much simpler, consisting now of just a
`TokenStream` and an index.
The commit also removes the `Extend` impl for `TokenStream`, because it
is only used in tests. (The commit also removes those tests.)
Overall, the commit reduces the number of lines of code by almost 200.
`TokenStream` is currently recursive in *two* ways:
- the `TokenTree` variant contains a `ThinTokenStream`, which can
contain a `TokenStream`;
- the `TokenStream` variant contains a `Vec<TokenStream>`.
The latter is not necessary and causes significant complexity. This
commit replaces it with the simpler `Vec<(TokenTree, IsJoint)>`.
This reduces complexity significantly. In particular, `StreamCursor` is
eliminated, and `Cursor` becomes much simpler, consisting now of just a
`TokenStream` and an index.
The commit also removes the `Extend` impl for `TokenStream`, because it
is only used in tests. (The commit also removes those tests.)
Overall, the commit reduces the number of lines of code by almost 200.
This commit completely removes usage of the `panictry!` macro from
outside libsyntax. The macro causes parse errors to be fatal, so using
it in libsyntax_ext caused parse failures *within* a syntax extension to
be fatal, which is probably not intended.
Furthermore, this commit adds spans to diagnostics emitted by empty
extensions if they were missing, à la #56491.
When a format string has escaped whitespace characters format
arguments were shifted by one per each escaped character. Account
for these escaped characters when synthesizing the spans.
Fix#55155.
- Point at opening mismatched formatting brace
- Account for differences between raw and regular strings
- Account for differences between the code snippet and `InternedString`
- Add more tests
Remove `TokenStream::JointTree`.
This is done by adding a new `IsJoint` field to `TokenStream::Tree`,
which simplifies a lot of `match` statements. And likewise for
`CursorKind`.
The commit also adds a new method `TokenTree:stream()` which can replace
a choice between `.into()` and `.joint()`.
This is done by adding a new `IsJoint` field to `TokenStream::Tree`,
which simplifies a lot of `match` statements. And likewise for
`CursorKind`.
The commit also adds a new method `TokenTree:stream()` which can replace
a choice between `.into()` and `.joint()`.
Remove `tokenstream::Delimited`.
Because it's an extra type layer that doesn't really help; in a couple
of places it actively gets in the way, and overall removing it makes the
code nicer. It does, however, move `tokenstream::TokenTree` further away
from the `TokenTree` in `quote.rs`.
More importantly, this change reduces the size of `TokenStream` from 48
bytes to 40 bytes on x86-64, which is enough to slightly reduce
instruction counts on numerous benchmarks, the best by 1.5%.
Note that `open_tt` and `close_tt` have gone from being methods on
`Delimited` to associated methods of `TokenTree`.
Because it's an extra type layer that doesn't really help; in a couple
of places it actively gets in the way, and overall removing it makes the
code nicer. It does, however, move `tokenstream::TokenTree` further away
from the `TokenTree` in `quote.rs`.
More importantly, this change reduces the size of `TokenStream` from 48
bytes to 40 bytes on x86-64, which is enough to slightly reduce
instruction counts on numerous benchmarks, the best by 1.5%.
Note that `open_tt` and `close_tt` have gone from being methods on
`Delimited` to associated methods of `TokenTree`.
`CrateRoot` -> `PathRoot`, `::` doesn't necessarily mean crate root now
`SelfValue` -> `SelfLower`, `SelfType` -> `SelfUpper`, both `self` and `Self` can be used in type and value namespaces now
Remove not used `DotEq` token
Currently libproc_macro does not use `DotEq` token.
https://github.com/rust-lang/rust/pull/49545 changed libproc_macro
to not generate `DotEq` token.
reword #[test] attribute error on fn items
fix of [#55787](https://github.com/rust-lang/rust/issues/55787)
Reworded error message from "#[test] attribute is only allowed on fn items" to "#[test] attribute is only allowed on non associated functions"
A few tweaks to iterations/collecting
- simplify and speed up `dot::GraphWalk::nodes` for `cfg::CFG`
- `reserve` the capacity for `edges` in `DepGraph::query`
- collect directly to a `HirVec` in `LoweringContext::lower_attrs`
- fix overallocation in `OnDiskCache::serialize`
- preallocate the `new_partitioning` vector in `merge_codegen_units`
- simplify `impl FromHex for str`
- improve the creation of `self_arg_names` in `impl MethodDef`
