Recursively expand `TokenKind::Interpolated` in `probably_equal_for_proc_macro`
Fixes#68430
When comparing the captured and re-parsed `TokenStream` for a `TokenKind::Interpolated`, we currently treat any nested `TokenKind::Interpolated` tokens as unequal. If a `TokenKind::Interpolated` token shows up in the captured `TokenStream` due to a `macro_rules!` expansion, we will throw away the captured `TokenStream`, losing span information.
This PR recursively invokes `nt_to_tokenstream` on nested `TokenKind::Interpolated` tokens, effectively flattening the stream into a sequence of non-interpolated tokens. This allows it to compare equal with the re-parsed stream, allowing us to keep the original captured `TokenStream` (with span information).
This requires all of the `probably_equal_for_proc_macro` methods to be moved from `librustc_ast` to `librustc_parse` so that they can call `nt_to_tokenstream`.
Rollup of 7 pull requests
Successful merges:
- #71289 (Allow using `Self::` in doc)
- #72375 (Improve E0599 explanation)
- #72385 (Add some teams to prioritization exclude_labels)
- #72395 (Allow rust-highfive to label issues it creates.)
- #72453 (Add flag to open docs: x.py doc --open)
- #72459 (Add core::future::IntoFuture)
- #72461 (Clean up E0600 explanation)
Failed merges:
r? @ghost
Update cargo
9 commits in cb06cb2696df2567ce06d1a39b1b40612a29f853..500b2bd01c958f5a33b6aa3f080bea015877b83c
2020-05-08 21:57:44 +0000 to 2020-05-18 17:12:54 +0000
- Handle LTO with an rlib/cdylib crate type (rust-lang/cargo#8254)
- Gracefully handle errors during a build. (rust-lang/cargo#8247)
- Update `im-rc` to 15.0.0 (rust-lang/cargo#8255)
- Fix `cargo update` with unused patch. (rust-lang/cargo#8243)
- Rephrased error message for disallowed sections in virtual workspace (rust-lang/cargo#8200)
- Ignore broken console output in some situations. (rust-lang/cargo#8236)
- Expand error message to explain that a string was found (rust-lang/cargo#8235)
- Add context to some fs errors. (rust-lang/cargo#8232)
- Move SipHasher to an isolated module. (rust-lang/cargo#8233)
update stacker to 0.1.9 to unbreak build on OpenBSD
the version 0.1.8 of stacker (what is currently pinned in Cargo.lock) doesn't build on OpenBSD (see https://github.com/rust-lang/stacker/pull/34).
update the version to 0.1.9
Prevent compiler stack overflow for deeply recursive code
I was unable to write a test that
1. runs in under 1s
2. overflows on my machine without this patch
The following reproduces the issue, but I don't think it's sensible to include a test that takes 30s to compile. We can now easily squash newly appearing overflows by the strategic insertion of calls to `ensure_sufficient_stack`.
```rust
// compile-pass
#![recursion_limit="1000000"]
macro_rules! chain {
(EE $e:expr) => {$e.sin()};
(RECURSE $i:ident $e:expr) => {chain!($i chain!($i chain!($i chain!($i $e))))};
(Z $e:expr) => {chain!(RECURSE EE $e)};
(Y $e:expr) => {chain!(RECURSE Z $e)};
(X $e:expr) => {chain!(RECURSE Y $e)};
(A $e:expr) => {chain!(RECURSE X $e)};
(B $e:expr) => {chain!(RECURSE A $e)};
(C $e:expr) => {chain!(RECURSE B $e)};
// causes overflow on x86_64 linux
// less than 1 second until overflow on test machine
// after overflow has been fixed, takes 30s to compile :/
(D $e:expr) => {chain!(RECURSE C $e)};
(E $e:expr) => {chain!(RECURSE D $e)};
(F $e:expr) => {chain!(RECURSE E $e)};
// more than 10 seconds
(G $e:expr) => {chain!(RECURSE F $e)};
(H $e:expr) => {chain!(RECURSE G $e)};
(I $e:expr) => {chain!(RECURSE H $e)};
(J $e:expr) => {chain!(RECURSE I $e)};
(K $e:expr) => {chain!(RECURSE J $e)};
(L $e:expr) => {chain!(RECURSE L $e)};
}
fn main() {
let x = chain!(D 42.0_f32);
}
```
fixes#55471fixes#41884fixes#40161fixes#34844fixes#32594
cc @alexcrichton @rust-lang/compiler
I looked at all code that checks the recursion limit and inserted stack growth calls where appropriate.
perf: Unify the undo log of all snapshot types
Extracted from #69218 and extended to all the current snapshot types.
Since snapshotting is such a frequent action in the compiler and many of the scopes execute so little work, the act of creating the snapshot and rolling back empty/small snapshots end up showing in perf. By unifying all the logs into one the creation of snapshots becomes significantly cheaper at the cost of some complexity when combining the log with the specific data structures that are being mutated.
Depends on https://github.com/rust-lang-nursery/ena/pull/29
In addition to fixing the toolstate, this also changes the default
compilation model to the out-of-process one, which should hopefully
target considerable memory usage for long-running instances of the RLS.
By merging the undo_log of all structures part of the snapshot the cost
of creating a snapshot becomes much cheaper. Since snapshots with no or
few changes are so frequent this ends up mattering more than the slight
overhead of dispatching on the variants that map to each field.
Implement `confusable_idents` lint.
This collects all identifier symbols into `ParseSession` and examines them within the non-ascii-idents lint.
The skeleton generation part needs to be added to `unicode-security` crate. Will update this PR when the crate is updated.
r? @petrochenkov
EDIT: also included the `concat_idents` part.
We anticipate this to have uses in all sorts of crates and keeping it in
`rustc_data_structures` enables access to it from more locations without
necessarily pulling in the large `librustc` crate.
