* Use `make check-lexer` to verify the grammar.
* Extend grammar/README
* Add make clean-grammar rule
* Add target `check-build-lexer-verifier` to `make tidy`, so it will build the verifier with every build and catch future errors
* Search for antlr4 with configure and find
Make sure Name, SyntaxContext and Ident are passed by value
Make sure Idents don't serve as keys (or parts of keys) in maps, Ident comparison is not well defined
* Correctly lex CRLF in string literals
* Update `extern CRATE as NAME` syntax
* Allow leading `::` in view paths
* Allow TySums in type ascriptions and impls
* Allow macros to have visibility and attributes
* Update syntax for qualified path types and expressions
* Allow block expressions to be called () and indexed []
This reverts commit 9c7d5ae57c.
This was wrong... the `continue` was to ignore the latter half of the
tokens file. Another mechanism will have to be used to keep the model
grammar's tokens in sync with the actual grammar's tokens :-/
To prevent the reference grammar from getting out of sync with the real
grammar, panic if RustLexer.tokens contains an unknown token in a
similar way that verify.rs panics if it encounters an unknown binary
operation token.
There were some tokens used in the grammar but not declared. Antlr
doesn't really seem to care and happily uses them, but they appear in
RustLexer.tokens in a potentially-unexpected order.
This appears to not have too much of a detrimental effect, but it
doesn't seem to be what is intended either.
antlr doesn't mind that `PLUS` isn't declared in `tokens` and happily
uses the `PLUS` that appears later in the file, but the generated
RustLexer.tokens had PLUS at the end rather than where it was intended:
NOT=10
TILDE=11
PLUT=12
MINUS=13
...
PLUS=56
This PR moves all `compile-fail` tests that fail at the parsing stage to a `parse-fail` directory, in order to use the tests in the `parse-fail` directory to test if the new LALR parser rejects the same files as the Rust parser. I also adjusted the `testparser.py` script to handle the tests in `parse-fail` differently.
However during working on this, I discovered, that Rust's parser sometimes fails during parsing, but does not return a nonzero return code, e.g. compiling `/test/compile-fail/doc-before-semi.rs` with `-Z parse-only` prints an error message, but returns status code 0. Compiling the same file without `-Z parse-only`, the same error message is displayed, but error code 101 returned. I'll look into that over the next week.
This restructures tidy.py to walk the tree itself,
and improves performance considerably by not loading entire
files into buffers for licenseck.
Splits build rules into 'tidy', 'tidy-basic', 'tidy-binaries',
'tidy-errors', 'tidy-features'.
This adds a new lexer/parser combo for the entire Rust language can be generated with with flex and bison, taken from my project at https://github.com/bleibig/rust-grammar. There is also a testing script that runs the generated parser with all *.rs files in the repository (except for tests in compile-fail or ones that marked as "ignore-test" or "ignore-lexer-test"). If you have flex and bison installed, you can run these tests using the new "check-grammar" make target.
This does not depend on or interact with the existing testing code in the grammar, which only provides and tests a lexer specification.
OS X users should take note that the version of bison that comes with the Xcode toolchain (2.3) is too old to work with this grammar, they need to download and install version 3.0 or later.
The parser builds up an S-expression-based AST, which can be displayed by giving the "-v" argument to parser-lalr (normally it only gives output on error). It is only a rough approximation of what is parsed and doesn't capture every detail and nuance of the program.
Hopefully this should be sufficient for issue #2234, or at least a good starting point.
The regex library was largely used for non-critical aspects of the compiler and
various external tooling. The library at this point is duplicated with its
out-of-tree counterpart and as such imposes a bit of a maintenance overhead as
well as compile time hit for the compiler itself.
The last major user of the regex library is the libtest library, using regexes
for filters when running tests. This removal means that the filtering has gone
back to substring matching rather than using regexes.
macro_rules! is like an item that defines a macro. Other items don't have a
trailing semicolon, or use a paren-delimited body.
If there's an argument for matching the invocation syntax, e.g. parentheses for
an expr macro, then I think that applies more strongly to the *inner*
delimiters on the LHS, wrapping the individual argument patterns.
This removes a large array of deprecated functionality, regardless of how
recently it was deprecated. The purpose of this commit is to clean out the
standard libraries and compiler for the upcoming alpha release.
Some notable compiler changes were to enable warnings for all now-deprecated
command line arguments (previously the deprecated versions were silently
accepted) as well as removing deriving(Zero) entirely (the trait was removed).
The distribution no longer contains the libtime or libregex_macros crates. Both
of these have been deprecated for some time and are available externally.
