6 Commits

Author SHA1 Message Date
Nicholas Nethercote
332dffb1f9 Remove TreeAndSpacing.
A `TokenStream` contains a `Lrc<Vec<(TokenTree, Spacing)>>`. But this is
not quite right. `Spacing` makes sense for `TokenTree::Token`, but does
not make sense for `TokenTree::Delimited`, because a
`TokenTree::Delimited` cannot be joined with another `TokenTree`.

This commit fixes this problem, by adding `Spacing` to `TokenTree::Token`,
changing `TokenStream` to contain a `Lrc<Vec<TokenTree>>`, and removing the
`TreeAndSpacing` typedef.

The commit removes these two impls:
- `impl From<TokenTree> for TokenStream`
- `impl From<TokenTree> for TreeAndSpacing`

These were useful, but also resulted in code with many `.into()` calls
that was hard to read, particularly for anyone not highly familiar with
the relevant types. This commit makes some other changes to compensate:
- `TokenTree::token()` becomes `TokenTree::token_{alone,joint}()`.
- `TokenStream::token_{alone,joint}()` are added.
- `TokenStream::delimited` is added.

This results in things like this:
```rust
TokenTree::token(token::Semi, stmt.span).into()
```
changing to this:
```rust
TokenStream::token_alone(token::Semi, stmt.span)
```
This makes the type of the result, and its spacing, clearer.

These changes also simplifies `Cursor` and `CursorRef`, because they no longer
need to distinguish between `next` and `next_with_spacing`.
2022-07-29 15:52:15 +10:00
Nicholas Nethercote
056d48a2c9 Remove unnecessary sigils around Symbol::as_str() calls. 2021-12-15 17:32:14 +11:00
Hirochika Matsumoto
3818981ca1 Practice diagnostic message convention 2021-10-03 16:16:28 +09:00
Vadim Petrochenkov
85ef265dbe expand: Stop un-interpolating NtIdents before passing them to built-in macros
This was a big hack, and built-in macros should be able to deal with `NtIdents` in the input by themselves like any other parser code.
2020-09-28 23:10:44 +03:00
Aaron Hill
1823dea7df
Attach TokenStream to ast::Ty
A `Ty` does not have outer attributes, so we only capture tokens
when parsing a `macro_rules!` matcher
2020-09-10 17:33:05 -04:00
mark
9e5f7d5631 mv compiler to compiler/ 2020-08-30 18:45:07 +03:00