add suggestion applicabilities to librustc and libsyntax
A down payment on #50723. Interested in feedback on whether my `MaybeIncorrect` vs. `MachineApplicable` judgement calls are well-calibrated (and that we have a consensus on what this means).
r? @Manishearth
cc @killercup @estebank
The `FatalError.raise()` might seem unmotivated (in most places in
the compiler, `err.emit()` suffices), but it's actually used to
maintain behavior (viz., stop lexing, don't emit potentially spurious
errors looking for the next token after the bad Unicodepoint in the
exponent): the previous revision's `self.err_span_` ultimately calls
`Handler::emit`, which aborts if the `Handler`'s continue_after_error
flag is set, which seems to typically be true during lexing (see
`phase_1_parse_input` and and how `CompileController::basic` has
`continue_parse_after_error: false` in librustc_driver).
Also, let's avoid apostrophes in error messages (the present author
would argue that users expect a reassuringly detached, formal,
above-it-all tone from a Serious tool like a compiler), and use an
RLS-friendly structured suggestion.
Resolves#49746.
rustc: Fix joint-ness of stringified token-streams
This commit fixes `StringReader`'s parsing of tokens which have been stringified
through procedural macros. Whether or not a token tree is joint is defined by
span information, but when working with procedural macros these spans are often
dummy and/or overridden which means that they end up considering all operators
joint if they can!
The fix here is to track the raw source span as opposed to the overridden span.
With this information we can more accurately classify `Punct` structs as either
joint or not.
Closes#50700
Consider this a down payment on #50723. To recap, an `Applicability`
enum was recently (#50204) added, to convey to Rustfix and other tools
whether we think it's OK for them to blindly apply the suggestion, or
whether to prompt a human for guidance (because the suggestion might
contain placeholders that we can't infer, or because we think it has a
sufficiently high probability of being wrong even though it's—
presumably—right often enough to be worth emitting in the first place).
When a suggestion is marked as `MaybeIncorrect`, we try to use comments
to indicate precisely why (although there are a few places where we just
say `// speculative` because the present author's subjective judgement
balked at the idea that the suggestion has no false positives).
The `run-rustfix` directive is opporunistically set on some relevant UI
tests (and a couple tests that were in the `test/ui/suggestions`
directory, even if the suggestions didn't originate in librustc or
libsyntax). This is less trivial than it sounds, because a surprising
number of test files aren't equipped to be tested as fixed even when
they contain successfully fixable errors, because, e.g., there are more,
not-directly-related errors after fixing. Some test files need an
attribute or underscore to avoid unused warnings tripping up the "fixed
code is still producing diagnostics" check despite the fixes being
correct; this is an interesting contrast-to/inconsistency-with the
behavior of UI tests (which secretly pass `-A unused`), a behavior which
we probably ought to resolve one way or the other (filed issue #50926).
A few suggestion labels are reworded (e.g., to avoid phrasing it as a
question, which which is discouraged by the style guidelines listed in
`.span_suggestion`'s doc-comment).
This commit fixes `StringReader`'s parsing of tokens which have been stringified
through procedural macros. Whether or not a token tree is joint is defined by
span information, but when working with procedural macros these spans are often
dummy and/or overridden which means that they end up considering all operators
joint if they can!
The fix here is to track the raw source span as opposed to the overridden span.
With this information we can more accurately classify `Punct` structs as either
joint or not.
Closes#50700
Implement edition hygiene for keywords
Determine "keywordness" of an identifier in its hygienic context.
cc https://github.com/rust-lang/rust/pull/49611
I've resurrected `proc` as an Edition-2015-only keyword for testing purposes, but it should probably be buried again. EDIT: `proc` is removed again.
It only has a single use, within code handling indented block comments.
We can replace that with the new `FileMap::col_pos()`, which computes
the col position (BytePos instead of CharPos) based on the record of the
last newline char (which we already record).
This is actually an improvement, because
`trim_whitespace_prefix_and_push_line()` was using `col`, which is a
`CharPos`, as a slice index, which is a byte/char confusion.
It's silly for a hot function like `bump()` to have such an expensive
bounds check. This patch replaces terminator with `end_src_index`.
Note that the `self.terminator` check in `is_eof()` wasn't necessary
because of the way `StringReader` is initialized.
- `source_text` becomes `src`, matching `FileMap::src`.
- `byte_offset()` becomes `src_index()`, which makes it clearer that
it's an index into `src`. (Likewise for variables containing
`byte_offset` in their name.) This function also now returns a `usize`
instead of a `BytePos`, because every callsite immediately converted
the `BytePos` to a `usize`.
This patch removes the "old"/"new" names in favour of "foo"/"next_foo",
which matches the field names.
It also moves the setting of `self.{ch,pos,next_pos}` in the common case
to the end, so that the meaning of "foo"/"next_foo" is consistent until
the end.
Currently ', " and \ are escaped as \', \" and \\ respectively. This
leads to confusing messages such as `error: unknown start of token: \\`
when encountering a single backslash.
Fix by emitting printable ASCII characters directly. This will still
escape \r, \n, \t and Unicode characters.
Fixes#47902
This `horizontal_trim` function strips the leading whitespace from
doc-comments that have a left-asterisk-margin:
/**
* You know what I mean—
*
* comments like this!
*/
The index of the column of asterisks is `i`, and if trimming is deemed
possible, we slice each line from `i+1` to the end of the line. But if, in
particular, `i` was 0 _and_ there was an empty line (as in the example
given in the reporting issue), we ended up panicking trying to slice an
empty string from 0+1 (== 1).
Let's tighten our check to say that we can't trim when `i` is even the same
as the length of the line, not just when it's greater. (Any such cases
would panic trying to slice `line` from `line.len()+1`.)
Resolves#47197.
