rust/compiler/rustc_parse
bors 803c60218f Auto merge of #79978 - Aaron1011:fix/capture-broken-token, r=petrochenkov
Properly capture trailing 'unglued' token

If we try to capture the `Vec<u8>` in `Option<Vec<u8>>`, we'll
need to capture a `>` token which was 'unglued' from a `>>` token.
The processing of unglueing a token for parsing purposes bypasses the
usual capturing infrastructure, so we currently lose the trailing `>`.
As a result, we fall back to the reparsed `TokenStream`, causing us to
lose spans.

This commit makes token capturing keep track of a trailing 'unglued'
token. Note that we don't need to care about unglueing except at the end
of the captured tokens - if we capture both the first and second unglued
tokens, then we'll end up capturing the full 'glued' token, which
already works correctly.
2020-12-13 19:31:06 +00:00
..
src Auto merge of #79978 - Aaron1011:fix/capture-broken-token, r=petrochenkov 2020-12-13 19:31:06 +00:00
Cargo.toml