A new matcher representation for use in `parse_tt`
By transforming the matcher into a different form, `parse_tt` can run faster and be easier to understand.
r? `@petrochenkov`
Improve method name suggestions
Attempts to improve method name suggestions when a matching method name
is not found. The approach taken is use the Levenshtein distance and
account for substrings having a high distance but can sometimes be very
close to the intended method (eg. empty vs is_empty).
resolves#94747
`parse_tt` currently traverses a `&[TokenTree]` to do matching. But this
is a bad representation for the traversal.
- `TokenTree` is nested, and there's a bunch of expensive and fiddly
state required to handle entering and exiting nested submatchers.
- There are three positions (sequence separators, sequence Kleene ops,
and end of the matcher) that are represented by an index that exceeds
the end of the `&[TokenTree]`, which is clumsy and error-prone.
This commit introduces a new representation called `MatcherLoc` that is
designed specifically for matching. It fixes all the above problems,
making the code much easier to read. A `&[TokenTree]` is converted to a
`&[MatcherLoc]` before matching begins. Despite the cost of the
conversion, it's still a net performance win, because various pieces of
traversal state are computed once up-front, rather than having to be
recomputed repeatedly during the macro matching.
Some improvements worth noting.
- `parse_tt_inner` is *much* easier to read. No more having to compare
`idx` against `len` and read comments to understand what the result
means.
- The handling of `Delimited` in `parse_tt_inner` is now trivial.
- The three end-of-sequence cases in `parse_tt_inner` are now handled in
three separate match arms, and the control flow is much simpler.
- `nameize` is no longer recursive.
- There were two places that issued "missing fragment specifier" errors:
one in `parse_tt_inner()`, and one in `nameize()`. Presumably the
latter was never executed. There's now a single place issuing these
errors, in `compute_locs()`.
- The number of heap allocations done for a `check full` build of
`async-std-1.10.0` (an extreme example of heavy macro use) drops from
11.8M to 2.6M, and most of these occur outside of macro matching.
- The size of `MatcherPos` drops from 64 bytes to 16 bytes. Small enough
that it no longer needs boxing, which partly accounts for the
reduction in allocations.
- The rest of the drop in allocations is due to the removal of
`MatcherKind`, because we no longer need to record anything for the
parent matcher when entering a submatcher.
- Overall it reduces code size by 45 lines.
Do not use `ParamEnv::and` when building a cache key from a param-env and trait eval candidate
Do not use `ParamEnv::and` to cache a param-env with a selection/evaluation candidate.
This is because if the param-env is `RevealAll` mode, and the candidate looks global (i.e. it has erased regions, which can show up when we normalize a projection type under a binder<sup>1</sup>), then when we use `ParamEnv::and` to pair the candidate and the param-env for use as a cache key, we will throw away the param-env's caller bounds, and we'll end up caching a candidate that we inferred from the param-env with a empty param-env, which may cause cache-hit later when we have an empty param-env, and possibly mess with normalization like we see in the referenced issue during codegen.
Not sure how to trigger this with a more structured test, but changing `check-pass` to `build-pass` triggers the case that https://github.com/rust-lang/rust/issues/94903 detected.
<sup>1.</sup> That is, we will replace the late-bound region with a placeholder, which gets canonicalized and turned into an infererence variable, which gets erased during region freshening right before we cache the result. Sorry, it's quite a few steps.
Fixes#94903
r? `@Aaron1011` (or reassign as you see fit)
Don't emit non-asm contents error for naked function composed of errors
## Motivation
For naked functions an error is emitted when they are composed of anything other than a single asm!() block. However, this error triggers in a couple situations in which it adds no additional information or is actively misleading.
One example is if you do have an asm!() block but simply one with a syntax error:
```rust
#[naked]
unsafe extern "C" fn compiler_errors() {
asm!(invalid_syntax)
}
```
This results in two errors, one for the syntax error itself and another telling you that you need an asm block in your function:
```rust
error[E0787]: naked functions must contain a single asm block
--> src/main.rs:6:1
|
6 | / unsafe extern "C" fn naked_compile_error() {
7 | | asm!(blah)
8 | | }
| |_^
```
This issue also comes up when [utilizing `compile_error!()` for improving your diagnostics](https://twitter.com/steveklabnik/status/1509538243020218372), such as raising a compiler error when compiling for an unsupported target.
## Implementation
The rules this PR implements are as follows:
1. If any non-erroneous non-asm statement is included, an error will still occur
2. If multiple asm statements are included, an error will still occur
3. If 0 or 1 asm statements are present, as well as any non-zero number of erroneous statements, then this error will *not* be raised as it is likely either redundant or incorrect
The rule of thumb is effectively "if an error is present and its correction could change things, don't raise an error".
Reduce the cost of loading all built-ins targets
This PR started by measuring the exact slowdown of checking of well known conditional values.
Than this PR implemented some technics to reduce the cost of loading all built-ins targets.
cf. https://github.com/rust-lang/rust/issues/82450#issuecomment-1073992323
Add debug assertions to some unsafe functions
As suggested by https://github.com/rust-lang/rust/issues/51713
~~Some similar code calls `abort()` instead of `panic!()` but aborting doesn't work in a `const fn`, and the intrinsic for doing dispatch based on whether execution is in a const is unstable.~~
This picked up some invalid uses of `get_unchecked` in the compiler, and fixes them.
I can confirm that they do in fact pick up invalid uses of `get_unchecked` in the wild, though the user experience is less-than-awesome:
```
Running unittests (target/x86_64-unknown-linux-gnu/debug/deps/rle_decode_fast-04b7918da2001b50)
running 6 tests
error: test failed, to rerun pass '--lib'
Caused by:
process didn't exit successfully: `/home/ben/rle-decode-helper/target/x86_64-unknown-linux-gnu/debug/deps/rle_decode_fast-04b7918da2001b50` (signal: 4, SIGILL: illegal instruction)
```
~~As best I can tell these changes produce a 6% regression in the runtime of `./x.py test` when `[rust] debug = true` is set.~~
Latest commit (6894d559bd) brings the additional overhead from this PR down to 0.5%, while also adding a few more assertions. I think this actually covers all the places in `core` that it is reasonable to check for safety requirements at runtime.
Thoughts?
Attempts to improve method name suggestions when a matching method name
is not found. The approach taken is use the Levenshtein distance and
account for substrings having a high distance but can sometimes be very
close to the intended method (eg. empty vs is_empty).
make memcmp return a value of c_int_width instead of i32
This is an attempt to fix#32610 and #78022, namely, that `memcmp` always returns an `i32` regardless of the platform. I'm running into some issues and was hoping I could get some help.
Here's what I've been attempting so far:
1. Build the stage0 compiler with all the changes _expect_ for the changes in `library/core/src/slice/cmp.rs` and `compiler/rustc_codegen_llvm/src/context.rs`; this is because `target_c_int_width` isn't passed through and recognized as a valid config option yet. I'm building with `./x.py build --stage 0 library/core library/proc_macro compiler/rustc`
2. Next I add in the `#[cfg(c_int_width = ...)]` params to `cmp.rs` and `context.rs` and build the stage 1 compiler by running `./x.py build --keep-stage 0 --stage 1 library/core library/proc_macro compiler/rustc`. This step now runs successfully.
3. Lastly, I try to build the test program for AVR mentioned in #78022 with `RUSTFLAGS="--emit llvm-ir" cargo build --release`, and look at the resulting llvm IR, which still shows:
```
...
%11 = call addrspace(1) i32 `@memcmp(i8*` nonnull %5, i8* nonnull %10, i16 5) #7, !dbg !1191 %.not = icmp eq i32 %11, 0, !dbg !1191
...
; Function Attrs: nounwind optsize declare i32 `@memcmp(i8*,` i8*, i16) local_unnamed_addr addrspace(1) #4
```
Any ideas what I'm missing here? Alternately, if this is totally the wrong approach I'm open to other suggestions.
cc `@Rahix`
Use DefPathHash instead of HirId to break inlining cycles.
The `DefPathHash` is stable across incremental compilation sessions, so provides a total order on `LocalDefId`. Using it instead of `HirId` ensures the MIR inliner has the same behaviour for incremental and non-incremental compilation.
A downside is that the cycle tie break is not as predictable is with `HirId`.
Suggest `i += 1` when we see `i++` or `++i`
Closes#83502 (for `i++` and `++i`; `--i` should be covered by #82987, and `i--`
is tricky to handle).
This is a continuation of #83536.
r? `@estebank`
Fix multiline attributes handling in doctests
Fixes#55713.
I needed to have access to the `unclosed_delims` field in order to check that the attribute was completely parsed and didn't have missing parts, so I created a getter for it.
r? `@notriddle`
Rollup of 4 pull requests
Successful merges:
- #95587 (Remove need for associated_type_bounds in std.)
- #95589 (Include a header in .rlink files)
- #95593 (diagnostics: add test case for bogus T:Sized suggestion)
- #95597 (Refer to u8 by absolute path in expansion of thread_local)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
Include a header in .rlink files
I couldn't find the right place where to put tests. Is there some location that tests `.rlink` creation and loading?
I only found `src/test/run-make-fulldeps/separate-link/Makefile`, but I'm not sure how to check the error message in the Makefile.
Fixes: https://github.com/rust-lang/rust/issues/95297
r? `@bjorn3`
Make GATs object safe under generic_associated_types_extended feature
Based on #94869
Let's say we have
```rust
trait StreamingIterator {
type Item<'a> where Self: 'a;
}
```
And `dyn for<'a> StreamingIterator<Item<'a> = &'a i32>`.
If we ask `(dyn for<'a> StreamingIterator<Item<'a> = &'a i32>): StreamingIterator`, then we have to prove that `for<'x> (&'x i32): Sized`. So, we generate *new* bound vars to subst for the GAT generics.
Importantly, this doesn't fully verify that these are usable and sound.
r? `@nikomatsakis`
Add error message suggestion for missing noreturn in naked function
I had to google the syntax for inline asm's `noreturn` option when I got this error earlier today, so I figured I'd save others the trouble and add the syntax/fix as a suggestion in the error.
Handle rustc_const_stable attribute in library feature collector
The library feature collector in [compiler/rustc_passes/src/lib_features.rs](551b4fa395/compiler/rustc_passes/src/lib_features.rs) has only been looking at `#[stable(…)]`, `#[unstable(…)]`, and `#[rustc_const_unstable(…)]` attributes, while ignoring `#[rustc_const_stable(…)]`. The consequences of this were:
- When any const feature got stabilized (changing one or more `rustc_const_unstable` to `rustc_const_stable`), users who had previously enabled that unstable feature using `#![feature(…)]` would get told "unknown feature", rather than rustc's nicer "the feature … has been stable since … and no longer requires an attribute to enable".
This can be seen in the way that https://github.com/rust-lang/rust/pull/93957#issuecomment-1079794660 failed after rebase:
```console
error[E0635]: unknown feature `const_ptr_offset`
--> $DIR/offset_from_ub.rs:1:35
|
LL | #![feature(const_ptr_offset_from, const_ptr_offset)]
| ^^^^^^^^^^^^^^^^
```
- We weren't enforcing that a particular feature is either stable everywhere or unstable everywhere, and that a feature that has been stabilized has the same stabilization version everywhere, both of which we enforce for the other stability attributes.
This PR updates the library feature collector to handle `rustc_const_stable`, and fixes places in the standard library and test suite where `rustc_const_stable` was being used in a way that does not meet the rules for a stability attribute.
Rollup of 6 pull requests
Successful merges:
- #95032 (Clean up, categorize and sort unstable features in std.)
- #95260 (Better suggestions for `Fn`-family trait selection errors)
- #95293 (suggest wrapping single-expr blocks in square brackets)
- #95344 (Make `impl Debug for rustdoc::clean::Item` easier to read)
- #95388 (interpret: make isize::MAX the limit for dynamic value sizes)
- #95530 (rustdoc: do not show primitives and keywords as private)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
interpret: make isize::MAX the limit for dynamic value sizes
We are currently enforcing `data_layout.obj_size_bound()` as the maximal dynamic size of a Rust value (including for `size_of_val_raw`), but that does not match the docs.
In particular, Miri currently falsely says that this code has UB:
```rust
#![feature(layout_for_ptr)]
fn main() {
let size = isize::MAX as usize;
// Creating a raw slice of size isize::MAX and asking for its size is okay.
let s = std::ptr::slice_from_raw_parts(1usize as *const u8, size);
assert_eq!(size, unsafe { std::mem::size_of_val_raw(s) });
}
```
Better suggestions for `Fn`-family trait selection errors
1. Suppress suggestions to add `std::ops::Fn{,Mut,Once}` bounds when a type already implements `Fn{,Mut,Once}`
2. Add a note that points out that a type does in fact implement `Fn{,Mut,Once}`, but the arguments vary (either by number or by actual arguments)
3. Add a note that points out that a type does in fact implement `Fn{,Mut,Once}`, but not the right one (e.g. implements `FnMut`, but `Fn` is required).
Fixes#95147
Specialize infinite-type "insert some indirection" suggestion for Option
Suggest `Option<Box<_>>` instead of `Box<Option<_>>` for infinitely-recursive members of a struct.
Not sure if I can get the span of the generic subty of the Option so I can make this a `+++`-style suggestion. The current output is a tiny bit less fancy looking than the original suggestion.
Should I limit the specialization to just `Option<Box<TheOuterStruct>>`? Because right now it applies to all `Option` members in the struct that are returned by `Representability::SelfRecursive`.
Fixes#91402
r? `@estebank`
(since you wrote the original suggestion and are definitely most familiar with it!)
Make lowering pull-based
~Based on https://github.com/rust-lang/rust/pull/90451~
Part of https://github.com/rust-lang/rust/pull/88186
The current lowering code visits all the item-likes in the AST in order, and lowers them one by one.
This PR changes it to index the AST and then proceed to lowering on-demand. This is closer to the logic of query-based lowering.
allow large Size again
This basically reverts most of https://github.com/rust-lang/rust/pull/80042, and instead does the panic in `bits()` with a `#[cold]` function to make sure it does not get inlined.
https://github.com/rust-lang/rust/pull/80042 added a comment about an invariant ("The top 3 bits are ALWAYS zero") that is not actually enforced, and if it were enforced that would be a problem for https://github.com/rust-lang/rust/pull/95388. So I think we should not have that invariant, and I adjusted the code accordingly.
r? `@oli-obk` Cc `@sivadeilra`
It's only used in one place, and there we clone and then make a bunch of
modifications. It's clearer if we duplicate more explicitly, and there's
a symmetry now between `sequence()` and `empty_sequence()`.
`parse_tt` needs a way to get from within submatchers make to the
enclosing submatchers. Currently it has two distinct mechanisms for
this:
- `Delimited` submatchers use `MatcherPos::stack` to record stuff about
the parent (and further back ancestors).
- `Sequence` submatchers use `MatcherPosSequence::parent` to point to
the parent matcher position.
Having two mechanisms is really confusing, and it took me a long time to
understand all this.
This commit eliminates `MatcherPos::stack`, and changes `Delimited`
submatchers to use the same mechanism as sequence submatchers. That
mechanism is also changed a bit: instead of storing the entire parent
`MatcherPos`, we now only store the necessary parts from the parent
`MatcherPos`.
Overall this is a small performance win, with the positives outweighing
the negatives, but it's mostly for clarity.
Spellchecking compiler comments
This PR cleans up the rest of the spelling mistakes in the compiler comments. This PR does not change any literal or code spelling issues.
Don't ICE when opaque types get their hidden type constrained again.
Contrary to popular belief, `codegen_fulfill_obligation` does not get used solely in codegen, so we cannot rely on `param_env` being set to RevealAll and thus revealing the hidden types instead of constraining them.
Fixes#89312 (for real this time)
Restore `impl Future<Output = Type>` to async blocks
I was sad when I undid some of the code I wrote in #91096 in the PR #95225, so I fixed it here to not print `[async output]`.
This PR "manually" normalizes the associated type `<[generator] as Generator>::Return` type which appears very frequently in `impl Future` types that result from async block desugaring.
Currently, we detect an exit from a `Delimited` submatcher when `idx`
exceeds the bounds of the current submatcher *and* there is a `stack`
entry.
This commit changes it to something simpler: just look for a
`CloseDelim` token.
Rollup of 6 pull requests
Successful merges:
- #93901 (Stabilize native library modifier syntax and the `whole-archive` modifier specifically)
- #94806 (Fix `cargo run tidy`)
- #94869 (Add the generic_associated_types_extended feature)
- #95011 (async: Give predictable name to binding generated from .await expressions.)
- #95251 (Reduce max hash in raw strings from u16 to u8)
- #95298 (Fix double drop of allocator in IntoIter impl of Vec)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
async: Give predictable name to binding generated from .await expressions.
This name makes it to debuginfo and allows debuggers to identify such bindings and their captured versions in suspended async fns.
This will be useful for async stack traces, as discussed in https://internals.rust-lang.org/t/async-debugging-logical-stack-traces-setting-goals-collecting-examples/15547.
I don't know if this needs some discussion by ````@rust-lang/compiler,```` e.g. about the name of the binding (`__awaitee`) or about the fact that this PR introduces a (soft) guarantee about a compiler generated name. Although, regarding the later, I think the same reasoning applies here as it does for debuginfo in general.
r? ````@tmandry````
Add the generic_associated_types_extended feature
Right now, this only ignore obligations that reference new placeholders in `poly_project_and_unify_type`. In the future, this might do other things, like allowing object-safe GATs.
**This feature is *incomplete* and quite likely unsound. This is mostly just for testing out potential future APIs using a "relaxed" set of rules until we figure out *proper* rules.**
Also drive by cleanup of adding a `ProjectAndUnifyResult` enum instead of using a `Result<Result<Option>>`.
r? `@nikomatsakis`
Remember mutability in `DefKind::Static`.
This allows to compute the `BodyOwnerKind` from `DefKind` only, and
removes a direct dependency of some MIR queries onto HIR.
As a side effect, it also simplifies metadata, since we don't need 4
flavours of `EntryKind::*Static` any more.
Yet more `parse_tt` improvements
Including lots of comment improvements, and an overhaul of how `matches` work that gives big speedups.
r? `@petrochenkov`
Contrary to popular belief, `codegen_fulfill_obligation` does not get used solely in codegen, so we cannot rely on `param_env` being set to RevealAll and thus revealing the hidden types instead of constraining them.
allow arbitrary inherent impls for builtin types in core
Part of https://github.com/rust-lang/compiler-team/issues/487. Slightly adjusted after some talks with `@m-ou-se` about the requirements of `t-libs-api`.
This adds a crate attribute `#![rustc_coherence_is_core]` which allows arbitrary impls for builtin types in core.
For other library crates impls for builtin types should be avoided if possible. We do have to allow the existing stable impls however. To prevent us from accidentally adding more of these in the future, there is a second attribute `#[rustc_allow_incoherent_impl]` which has to be added to **all impl items**. This only supports impls for builtin types but can easily be extended to additional types in a future PR.
This implementation does not check for overlaps in these impls. Perfectly checking that requires us to check the coherence of these incoherent impls in every crate, as two distinct dependencies may add overlapping methods. It should be easy enough to detect if it goes wrong and the attribute is only intended for use inside of std.
The first two commits are mostly unrelated cleanups.
Strict Provenance MVP
This patch series examines the question: how bad would it be if we adopted
an extremely strict pointer provenance model that completely banished all
int<->ptr casts.
The key insight to making this approach even *vaguely* pallatable is the
ptr.with_addr(addr) -> ptr
function, which takes a pointer and an address and creates a new pointer
with that address and the provenance of the input pointer. In this way
the "chain of custody" is completely and dynamically restored, making the
model suitable even for dynamic checkers like CHERI and Miri.
This is not a formal model, but lots of the docs discussing the model
have been updated to try to the *concept* of this design in the hopes
that it can be iterated on.
See #95228
Rollup of 5 pull requests
Successful merges:
- #95294 (Document Linux kernel handoff in std::io::copy and std::fs::copy)
- #95443 (Clarify how `src/tools/x` searches for python)
- #95452 (fix since field version for termination stabilization)
- #95460 (Spellchecking compiler code)
- #95461 (Spellchecking some comments)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
Lazy type-alias-impl-trait take two
### user visible change 1: RPIT inference from recursive call sites
Lazy TAIT has an insta-stable change. The following snippet now compiles, because opaque types can now have their hidden type set from wherever the opaque type is mentioned.
```rust
fn bar(b: bool) -> impl std::fmt::Debug {
if b {
return 42
}
let x: u32 = bar(false); // this errors on stable
99
}
```
The return type of `bar` stays opaque, you can't do `bar(false) + 42`, you need to actually mention the hidden type.
### user visible change 2: divergence between RPIT and TAIT in return statements
Note that `return` statements and the trailing return expression are special with RPIT (but not TAIT). So
```rust
#![feature(type_alias_impl_trait)]
type Foo = impl std::fmt::Debug;
fn foo(b: bool) -> Foo {
if b {
return vec![42];
}
std::iter::empty().collect() //~ ERROR `Foo` cannot be built from an iterator
}
fn bar(b: bool) -> impl std::fmt::Debug {
if b {
return vec![42]
}
std::iter::empty().collect() // Works, magic (accidentally stabilized, not intended)
}
```
But when we are working with the return value of a recursive call, the behavior of RPIT and TAIT is the same:
```rust
type Foo = impl std::fmt::Debug;
fn foo(b: bool) -> Foo {
if b {
return vec![];
}
let mut x = foo(false);
x = std::iter::empty().collect(); //~ ERROR `Foo` cannot be built from an iterator
vec![]
}
fn bar(b: bool) -> impl std::fmt::Debug {
if b {
return vec![];
}
let mut x = bar(false);
x = std::iter::empty().collect(); //~ ERROR `impl Debug` cannot be built from an iterator
vec![]
}
```
### user visible change 3: TAIT does not merge types across branches
In contrast to RPIT, TAIT does not merge types across branches, so the following does not compile.
```rust
type Foo = impl std::fmt::Debug;
fn foo(b: bool) -> Foo {
if b {
vec![42_i32]
} else {
std::iter::empty().collect()
//~^ ERROR `Foo` cannot be built from an iterator over elements of type `_`
}
}
```
It is easy to support, but we should make an explicit decision to include the additional complexity in the implementation (it's not much, see a721052457cf513487fb4266e3ade65c29b272d2 which needs to be reverted to enable this).
### PR formalities
previous attempt: #92007
This PR also includes #92306 and #93783, as they were reverted along with #92007 in #93893fixes#93411fixes#88236fixes#89312fixes#87340fixes#86800fixes#86719fixes#84073fixes#83919fixes#82139fixes#77987fixes#74282fixes#67830fixes#62742fixes#54895
Currently, matches within a sequence are recorded in a new empty
`matches` vector. Then when the sequence finishes the matches are merged
into the `matches` vector of the parent.
This commit changes things so that a sequence mp inherits the matches
made so far. This means that additional matches from the sequence don't
need to be merged into the parent. `push_match` becomes more
complicated, and the current sequence depth needs to be tracked. But
it's a sizeable performance win because it avoids one or more
`push_match` calls on every iteration of a sequence.
The commit also removes `match_hi`, which is no longer necessary.
Suggest wrapping patterns in enum variants
Structured suggestion to wrap a pattern in a single-field enum or struct:
```diff
struct A;
enum B {
A(A),
}
fn main(b: B) {
match b {
- A => {}
+ B::A(A) => {}
}
}
```
Half of #94942, the other half I'm not exactly sure how to fix.
Also includes two drive-by changes (that I am open to splitting out into another PR, but thought they could be rolled up into this one):
- 07776c111f: Makes sure not to suggest wrapping if it doesn't have tuple field constructor (i.e. has named fields)
- 8f2bbb18fd53e5008bb488302dbd354577698ede: Also suggest wrapping expressions in a tuple struct (not just enum variants)
This allows to compute the `BodyOwnerKind` from `DefKind` only, and
removes a direct dependency of some MIR queries onto HIR.
As a side effect, it also simplifies metadata, since we don't need 4
flavours of `EntryKind::*Static` any more.
