936d999b52
This common representation for delimeters should make pattern matching easier. Having a separate `token::DelimToken` enum also allows us to enforce the invariant that the opening and closing delimiters must be the same in `ast::TtDelimited`, removing the need to ensure matched delimiters when working with token trees. |
||
|---|---|---|
| .. | ||
| .gitignore | ||
| check.sh | ||
| raw-string-literal-ambiguity.md | ||
| README.md | ||
| RustLexer.g4 | ||
| verify.rs | ||
Reference grammar.
Uses antlr4 and a custom Rust tool to compare
ASTs/token streams generated. You can use the check-syntax make target to
run all of the available tests.
To use manually:
antlr4 RustLexer.g4
javac *.java
rustc -O verify.rs
for file in ../*/**.rs; do
echo $file;
grun RustLexer tokens -tokens < $file | ./verify $file || break
done
Note That the ../*/**.rs glob will match every *.rs file in the above
directory and all of its recursive children. This is a zsh extension.