Implement a file-path remapping feature in support of debuginfo and reproducible builds
This PR adds the `-Zremap-path-prefix-from`/`-Zremap-path-prefix-to` commandline option pair and is a more general implementation of #41419. As opposed to the previous attempt, this implementation should enable reproducible builds regardless of the working directory of the compiler.
This implementation of the feature is more general in the sense that the re-mapping will affect *all* paths the compiler emits, including the ones in error messages.
r? @alexcrichton
* Use `make check-lexer` to verify the grammar.
* Extend grammar/README
* Add make clean-grammar rule
* Add target `check-build-lexer-verifier` to `make tidy`, so it will build the verifier with every build and catch future errors
* Search for antlr4 with configure and find
Make sure Name, SyntaxContext and Ident are passed by value
Make sure Idents don't serve as keys (or parts of keys) in maps, Ident comparison is not well defined
* Correctly lex CRLF in string literals
* Update `extern CRATE as NAME` syntax
* Allow leading `::` in view paths
* Allow TySums in type ascriptions and impls
* Allow macros to have visibility and attributes
* Update syntax for qualified path types and expressions
* Allow block expressions to be called () and indexed []
This reverts commit 9c7d5ae57c27ebfc019c2c23283bb905d8c3b74f.
This was wrong... the `continue` was to ignore the latter half of the
tokens file. Another mechanism will have to be used to keep the model
grammar's tokens in sync with the actual grammar's tokens :-/
To prevent the reference grammar from getting out of sync with the real
grammar, panic if RustLexer.tokens contains an unknown token in a
similar way that verify.rs panics if it encounters an unknown binary
operation token.
There were some tokens used in the grammar but not declared. Antlr
doesn't really seem to care and happily uses them, but they appear in
RustLexer.tokens in a potentially-unexpected order.
This appears to not have too much of a detrimental effect, but it
doesn't seem to be what is intended either.
antlr doesn't mind that `PLUS` isn't declared in `tokens` and happily
uses the `PLUS` that appears later in the file, but the generated
RustLexer.tokens had PLUS at the end rather than where it was intended:
NOT=10
TILDE=11
PLUT=12
MINUS=13
...
PLUS=56
This PR moves all `compile-fail` tests that fail at the parsing stage to a `parse-fail` directory, in order to use the tests in the `parse-fail` directory to test if the new LALR parser rejects the same files as the Rust parser. I also adjusted the `testparser.py` script to handle the tests in `parse-fail` differently.
However during working on this, I discovered, that Rust's parser sometimes fails during parsing, but does not return a nonzero return code, e.g. compiling `/test/compile-fail/doc-before-semi.rs` with `-Z parse-only` prints an error message, but returns status code 0. Compiling the same file without `-Z parse-only`, the same error message is displayed, but error code 101 returned. I'll look into that over the next week.
This restructures tidy.py to walk the tree itself,
and improves performance considerably by not loading entire
files into buffers for licenseck.
Splits build rules into 'tidy', 'tidy-basic', 'tidy-binaries',
'tidy-errors', 'tidy-features'.
This adds a new lexer/parser combo for the entire Rust language can be generated with with flex and bison, taken from my project at https://github.com/bleibig/rust-grammar. There is also a testing script that runs the generated parser with all *.rs files in the repository (except for tests in compile-fail or ones that marked as "ignore-test" or "ignore-lexer-test"). If you have flex and bison installed, you can run these tests using the new "check-grammar" make target.
This does not depend on or interact with the existing testing code in the grammar, which only provides and tests a lexer specification.
OS X users should take note that the version of bison that comes with the Xcode toolchain (2.3) is too old to work with this grammar, they need to download and install version 3.0 or later.
The parser builds up an S-expression-based AST, which can be displayed by giving the "-v" argument to parser-lalr (normally it only gives output on error). It is only a rough approximation of what is parsed and doesn't capture every detail and nuance of the program.
Hopefully this should be sufficient for issue #2234, or at least a good starting point.
