Implemented a smarter TokenStream concatenation system
The new algorithm performs 'aggressive compacting' during concatenation as follows:
- If the nodes' combined total total length is less than 32, we copy both of
them into a new vector and build a new leaf node.
- If one node is an internal node and the other is a 'small' leaf (length<32),
we recur down the internal node on the appropriate side.
- Otherwise, we construct a new internal node that points to them as left and
right.
This should produce notably better behavior than the current concatenation implementation.
std: Optimize panic::catch_unwind slightly
The previous implementation of this function was overly conservative with
liberal usage of `Option` and `.unwrap()` which in theory never triggers. This
commit essentially removes the `Option`s in favor of unsafe implementations,
improving the code generation of the fast path for LLVM to see through what's
happening more clearly.
cc #34727
Produce source package in rust-installer format
See rust-lang/rust-buildbot#102
There may be a better way to do this, wasn't sure how to clean-up the `rust-src-image` directory when it's used by multiple make rules.
macros: Make metavariables hygienic
This PR makes metavariables hygienic. For example, consider:
```rust
macro_rules! foo {
($x:tt) => { // Suppose that this token tree argument is always a metavariable.
macro_rules! bar { ($x:expr, $y:expr) => { ($x, $y) } }
}
}
fn main() {
foo!($z); // This currently compiles.
foo!($y); // This is an error today but compiles after this PR.
}
```
Today, the `macro_rules! bar { ... }` definition is only valid when the metavariable passed to `foo` is not `$y` (since it unhygienically conflicts with the `$y` in the definition of `bar`) or `$x` (c.f. #35450).
After this PR, the definition of `bar` is always valid (and `bar!(a, b)` always expands to `(a, b)` as expected).
This can break code that was allowed in #34925 (landed two weeks ago). For example,
```rust
macro_rules! outer {
($t:tt) => {
macro_rules! inner { ($i:item) => { $t } }
}
}
outer!($i); // This `$i` should not interact with the `$i` in the definition of `inner!`.
inner!(fn main() {}); // After this PR, this is an error ("unknown macro variable `i`").
```
Due to the severe limitations on nested `macro_rules!` before #34925, this is not a breaking change for stable/beta.
Fixes#35450.
r? @nrc
Add --test-threads option to test binaries
This change allows parallelism of test runs to be specified by a
command line flag names --test-threads in addition to the existing
environment variable RUST_TEST_THREADS. Fixes#25636.
These tests check for the old error messages "`return` in a function
declared as diverging" and "computation may converge in a function
declared as diverging". The first of these is now invalid as `return` is
permitted in functions that return `!`. The second of these is subsumed
by the "mismatched types" error.
