Rollup merge of #96543 - nnethercote:rm-make_token_stream-hacks, r=Aaron1011

Remove hacks in `make_token_stream`.

`make_tokenstream` has three commented hacks, and a comment at the top
referring to #67062. These hacks have no observable effect, at least as judged
by running the test suite. The hacks were added in #82608, with an explanation
[here](https://github.com/rust-lang/rust/pull/82608#issuecomment-812877329). It
appears that one of the following is true: (a) they never did anything useful,
(b) they do something useful but we have no test coverage for them, or (c)
something has changed in the meantime that means they are no longer necessary.

This commit removes the hacks and the comments, in the hope that (b) is not
true.

r? `@Aaron1011`
This commit is contained in:
Yuki Okushi 2022-05-11 13:16:30 +09:00 committed by GitHub
commit 8fcf113743

View File

@ -79,9 +79,7 @@ fn check_keyword<'a, 'b: 'a>(parser: &'a mut Parser<'b>) -> Option<MacroArg> {
for &keyword in RUST_KW.iter() {
if parser.token.is_keyword(keyword)
&& parser.look_ahead(1, |t| {
t.kind == TokenKind::Eof
|| t.kind == TokenKind::Comma
|| t.kind == TokenKind::CloseDelim(Delimiter::Invisible)
t.kind == TokenKind::Eof || t.kind == TokenKind::Comma
})
{
parser.bump();