Rewrite `collect_tokens` implementations to use a flattened buffer
Instead of trying to collect tokens at each depth, we 'flatten' the
stream as we go allong, pushing open/close delimiters to our buffer
just like regular tokens. One capturing is complete, we reconstruct a
nested `TokenTree::Delimited` structure, producing a normal
`TokenStream`.
The reconstructed `TokenStream` is not created immediately - instead, it is
produced on-demand by a closure (wrapped in a new `LazyTokenStream` type). This
closure stores a clone of the original `TokenCursor`, plus a record of the
number of calls to `next()/next_desugared()`. This is sufficient to reconstruct
the tokenstream seen by the callback without storing any additional state. If
the tokenstream is never used (e.g. when a captured `macro_rules!` argument is
never passed to a proc macro), we never actually create a `TokenStream`.
This implementation has a number of advantages over the previous one:
* It is significantly simpler, with no edge cases around capturing the
start/end of a delimited group.
* It can be easily extended to allow replacing tokens an an arbitrary
'depth' by just using `Vec::splice` at the proper position. This is
important for PR #76130, which requires us to track information about
attributes along with tokens.
* The lazy approach to `TokenStream` construction allows us to easily
parse an AST struct, and then decide after the fact whether we need a
`TokenStream`. This will be useful when we start collecting tokens for
`Attribute` - we can discard the `LazyTokenStream` if the parsed
attribute doesn't need tokens (e.g. is a builtin attribute).
The performance impact seems to be neglibile (see
https://github.com/rust-lang/rust/pull/77250#issuecomment-703960604). There is a
small slowdown on a few benchmarks, but it only rises above 1% for incremental
builds, where it represents a larger fraction of the much smaller instruction
count. There a ~1% speedup on a few other incremental benchmarks - my guess is
that the speedups and slowdowns will usually cancel out in practice.
rustdoc: Show the correct source filename in page titles, without `.html`
Previously the title would be
lib.rs.html -- source
if `lib.rs` was the actual source filename. Now the title is
lib.rs - source
Improve wording of "cannot multiply" type error
For example, if you had this code:
fn foo(x: i32, y: f32) -> f32 {
x * y
}
You would get this error:
error[E0277]: cannot multiply `f32` to `i32`
--> src/lib.rs:2:7
|
2 | x * y
| ^ no implementation for `i32 * f32`
|
= help: the trait `Mul<f32>` is not implemented for `i32`
However, that's not usually how people describe multiplication. People
usually describe multiplication like how the division error words it:
error[E0277]: cannot divide `i32` by `f32`
--> src/lib.rs:2:7
|
2 | x / y
| ^ no implementation for `i32 / f32`
|
= help: the trait `Div<f32>` is not implemented for `i32`
So that's what this change does. It changes this:
error[E0277]: cannot multiply `f32` to `i32`
--> src/lib.rs:2:7
|
2 | x * y
| ^ no implementation for `i32 * f32`
|
= help: the trait `Mul<f32>` is not implemented for `i32`
To this:
error[E0277]: cannot multiply `i32` by `f32`
--> src/lib.rs:2:7
|
2 | x * y
| ^ no implementation for `i32 * f32`
|
= help: the trait `Mul<f32>` is not implemented for `i32`
Add Pin::static_ref, static_mut.
This adds `Pin::static_ref` and `Pin::static_mut`, which convert a static reference to a pinned static reference.
Static references are effectively already pinned, as what they refer to has to live forever and can never be moved.
---
Context: I want to update the `sys` and `sys_common` mutexes/rwlocks/condvars to use `Pin<&self>` in their functions, instead of only warning in the unsafety comments that they may not be moved. That should make them a little bit less dangerous to use. Putting such an object in a `static` (e.g. through `sys_common::StaticMutex`) fulfills the requirements about never moving it, but right now there's no safe way to get a `Pin<&T>` to a `static`. This solves that.
Rollup of 9 pull requests
Successful merges:
- #78046 (Add codegen test for issue #73827)
- #78061 (Optimize const value interning for ZST types)
- #78070 (we can test std and core panic macros together)
- #78076 (Move orphan module-name/mod.rs files into module-name.rs files)
- #78129 (Wrapping intrinsics doc links update.)
- #78133 (Add some MIR-related regression tests)
- #78144 (Don't update `entries` in `TypedArena` if T does not need drop)
- #78145 (Drop unneeded `mut`)
- #78157 (Remove unused type from librustdoc)
Failed merges:
r? `@ghost`
Drop unneeded `mut`
These parameters don't get modified.
Note that `trailing_comment` is pub and gets exported from `rustc_ast_pretty`. Is that considered to be a stable API? If yes, and you want to reserve the right to modify `self` in `trailing_comment` in the future, that hunk would need to be dropped.
Don't update `entries` in `TypedArena` if T does not need drop
As far as I can tell, `entries` is only used when dropping `TypedArenaChunk`s and their contents. It is already ignored there, if T is not `mem::needs_drop`, this PR just skips updating it's value.
You can see `TypedArenaChunk` ignoring the entry count in L71. The reasoning is similar to what you can find in `DroplessArena`.
r? @oli-obk
Wrapping intrinsics doc links update.
The links in the wrapping intrinsics docs now refer to the `wrapping_*` functions, not the `checked_*` functions.
Disable MatchBranchSimplification
This optimization can result in unsoundness, because it introduces
additional uses of a place holding the discriminant value without
ensuring that it is valid to do so.
Found by validation from #77369 / #78147.
Improve `skip_binder` usage during FlagComputation
It looks like there was previously a bug around `ExistentialPredicate::Projection` here, don't know how to best trigger that one to add a regression test though.
Rollup of 10 pull requests
Successful merges:
- #77612 (BTreeMap: test invariants more thoroughly and more readably)
- #77761 (Assert that pthread mutex initialization succeeded)
- #77778 ([x.py setup] Allow setting up git hooks from other worktrees)
- #77838 (const keyword: brief paragraph on 'const fn')
- #77923 ([net] apply clippy lints)
- #77931 (Fix false positive for `unused_parens` lint)
- #77959 (Tweak ui-tests structure)
- #78105 (change name in .mailmap)
- #78111 (Trait predicate ambiguities are not always in `Self`)
- #78121 (Do not ICE on pattern that uses a binding multiple times in generator)
Failed merges:
r? `@ghost`
Trait predicate ambiguities are not always in `Self`
When reporting ambiguities in trait predicates, the compiler incorrectly assumed the ambiguity was always in the type the trait should be implemented on, and never the generic parameters of the trait. This caused silly suggestions for predicates like `<KnownType as Trait<_>>`, such as giving explicit types to completely unrelated variables that happened to be of type `KnownType`.
This also reverts #73027, which worked around this issue in some cases and does not appear to be necessary any more.
fixes#77982fixes#78055
Tweak ui-tests structure
We have some similar name dirs in ui tests, e.g. `associated-type` and `associated-types` and it can be an issue when we add a test, "which is the right place?". At a glance, it seems they can be merged into one directory so let's merge them to avoid some confusion :)
const keyword: brief paragraph on 'const fn'
`const fn` were mentioned in the title, but called "deterministic functions" which is not their main property (though at least currently it is a consequence of being const-evaluable). This adds a brief paragraph discussing them, also in the hopes of clarifying that they do *not* have any effect on run-time uses.
Assert that pthread mutex initialization succeeded
If pthread mutex initialization fails, the failure will go unnoticed unless
debug assertions are enabled. Any subsequent use of mutex will also silently
fail, since return values from lock & unlock operations are similarly checked
only through debug assertions.
In some implementations the mutex initialization requires a memory
allocation and so it does fail in practice.
Assert that initialization succeeds to ensure that mutex guarantees
mutual exclusion.
Fixes#34966.
This optimization can result in unsoundness, because it introduces
additional uses of a place holding the discriminant value without
ensuring that it is valid to do so.
If pthread mutex initialization fails, the failure will go unnoticed unless
debug assertions are enabled. Any subsequent use of mutex will also silently
fail, since return values from lock & unlock operations are similarly checked
only through debug assertions.
In some implementations the mutex initialization requires a memory
allocation and so it does fail in practice.
Check that initialization succeeds to ensure that mutex guarantees
mutual exclusion.
