Make it possible to run `cargo test` for bootstrap
Note that this only runs bootstrap's self-tests, not compiler or library tests.
Helps with https://github.com/rust-lang/rust/issues/94829.
Fix `cargo run` on Windows
Fixes the following error:
```
error: failed to run custom build command for `bootstrap v0.0.0 (C:\Users\Walther\git\rust\src\bootstrap)`
Caused by:
process didn't exit successfully: `C:\Users\Walther\git\rust\target\debug\build\bootstrap-7757a4777dec0f86\build-script-build` (exit code: 101)
--- stdout
cargo:rerun-if-changed=build.rs
cargo:rerun-if-env-changed=RUSTC
cargo:rustc-env=BUILD_TRIPLE=x86_64-pc-windows-msvc
cargo:rerun-if-env-changed=PATH
--- stderr
thread 'main' panicked at 'assertion failed: rustc.is_absolute()', src\bootstrap\build.rs:22:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: build failed
```
The problem was that the `dir.join` check only works with `rustc.exe`, not `rustc`.
Thanks `@Walther` for the help testing the fix!
Helps with https://github.com/rust-lang/rust/issues/94829.
Fix `x test src/librustdoc` with `download-rustc` enabled
The problem was two-fold:
- Bootstrap was hard-coding that unit tests should always run with stage1, not stage2, and
- It hard-coded the sysroot layout in stage1, which puts libLLVM.so in `lib/rustlib/` instead of just `lib/`.
This also takes the liberty of fixing `test src/librustdoc --no-doc`, which has been broken since it was first added. It would be nice at some point to unify this logic with other tests; I opened a Zulip thread: https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/Inconsistency.20in.20.60x.20test.60
Fixes https://github.com/rust-lang/rust/issues/91071.
Fix `x test src/tools/error_index_generator --stage {0,1}`
There were two fixes needed:
1. Use `top_stage` instead of `top_stage - 1`. There was a long and torturous comment about trying to match rustdoc's version, but it works better without the hard-coding than with (before it gave errors that `libtest.so` couldn't be found).
2. Make sure that `ci-llvm/lib` is added to LD_LIBRARY_PATH. Previously the error index would be unable to load LLVM for stage0 builds.
At some point we should probably have a discussion about how rustdoc stages should be numbered;
confusion between 0/1/2 has come up several times in bootstrap now. cc https://github.com/rust-lang/rust/issues/92538
Note that this is still broken when using `download-rustc = true` and `--stage 1`,
but that's *really* a corner case and should affect almost no one. `--stage {0,2}`
work fine with download-rustc.
Fixes https://github.com/rust-lang/rust/issues/80096.
Bump bootstrap compiler to 1.61.0 beta
This PR bumps the bootstrap compiler to the 1.61.0 beta. The first commit changes the stage0 compiler, the second commit applies the "mechanical" changes and the third and fourth commits apply changes explained in the relevant comments.
r? `@Mark-Simulacrum`
[bootstrap] Give a hard error when filtering tests for a file that does not exist
A common issue people run into when running compiletest is that filtering for files that don't exist is only a warning and not an error; running the whole test suite instead.
See for example https://rust-lang.zulipchat.com/#narrow/stream/182449-t-compiler.2Fhelp/topic/Question.20about.20compiletest.
This is especially bad when using `--bless`, which will modify all `.stderr` files.
Change bootstrap to require valid filters instead of discarding invalid filters and continuing.
Before:
```
Warning: Skipping "/home/jnelson/rust-lang/rust/src/test/rustdoc-ui/intra-doc/feature-gate-intra-doc-pointers.r": not a regular file or directory
Check compiletest suite=rustdoc-ui mode=ui (x86_64-unknown-linux-gnu(x86_64-unknown-linux-gnu) -> x86_64-unknown-linux-gnu(x86_64-unknown-linux-gnu))
running 163 tests
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.......................... 100/163
...............................................................
test result: ok. 89 passed; 0 failed; 74 ignored; 0 measured; 0 filtered out; finished in 7.20s
finished in 7.248 seconds
Build completed successfully in 0:00:08
```
After:
```
thread 'main' panicked at 'Invalid test suite filter "/home/jnelson/rust-lang/rust/src/test/rustdoc-ui/intra-doc/feature-gate-intra-doc-pointers.r": file or directory does not exist', src/bootstrap/util.rs:311:
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Build completed unsuccessfully in 0:00:08
```
Fix `x doc --stage 0 compiler`
Eric figured out the fix to this almost 2 years ago, I just didn't read his comment carefully enough at the timme.
The issue was that fake rustc and fake rustdoc were inconsistent about when they passed `--sysroot` to the real compiler.
Change them to consistently only pass it when `--target` is present.
cc https://github.com/rust-lang/rust/issues/74976#issuecomment-667265945
Fixes https://github.com/rust-lang/rust/issues/79980
r? ``@ehuss``
bootstrap.py: nixos check in /etc/os-release with quotes
Per https://www.freedesktop.org/software/systemd/man/os-release.html,
> Variable assignment values must be enclosed in double or single quotes
> if they include spaces, semicolons or other special characters outside
> of A–Z, a–z, 0–9. (Assignments that do not include these special
> characters may be enclosed in quotes too, but this is optional.)
So, past `ID=nixos`, let's also check for `ID='nixos'` and `ID="nixos"`.
One of these is necessary between nixos/nixpkgs#162168 and
nixos/nixpkgs#164068, but this seems more correct either way.
Don't build the full compiler before running unit tests
This has been present since `builder.ensure` was first added in https://github.com/rust-lang/rust/pull/43059.
It's unclear to me why it was added then - I tested these changes locally
with `x test compiler/rustc_data_structures --stage 0` and they worked fine.
Fixes https://github.com/rust-lang/rust/issues/51748.
Fix `cargo run tidy`
When I implemented rust-only bootstrapping in https://github.com/rust-lang/rust/pull/92260,
I neglected to test stage0 tools - it turns out they were broken because
they couldn't find the sysroot of the initial bootstrap compiler.
This fixes stage0 tools by using `rustc --print sysroot` instead of assuming rustc is already in a
sysroot and hard-coding the relative directory.
Fixes https://github.com/rust-lang/rust/issues/94797 (properly, without having to change rustup).
Rollup of 5 pull requests
Successful merges:
- #95294 (Document Linux kernel handoff in std::io::copy and std::fs::copy)
- #95443 (Clarify how `src/tools/x` searches for python)
- #95452 (fix since field version for termination stabilization)
- #95460 (Spellchecking compiler code)
- #95461 (Spellchecking some comments)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
Lazy type-alias-impl-trait take two
### user visible change 1: RPIT inference from recursive call sites
Lazy TAIT has an insta-stable change. The following snippet now compiles, because opaque types can now have their hidden type set from wherever the opaque type is mentioned.
```rust
fn bar(b: bool) -> impl std::fmt::Debug {
if b {
return 42
}
let x: u32 = bar(false); // this errors on stable
99
}
```
The return type of `bar` stays opaque, you can't do `bar(false) + 42`, you need to actually mention the hidden type.
### user visible change 2: divergence between RPIT and TAIT in return statements
Note that `return` statements and the trailing return expression are special with RPIT (but not TAIT). So
```rust
#![feature(type_alias_impl_trait)]
type Foo = impl std::fmt::Debug;
fn foo(b: bool) -> Foo {
if b {
return vec![42];
}
std::iter::empty().collect() //~ ERROR `Foo` cannot be built from an iterator
}
fn bar(b: bool) -> impl std::fmt::Debug {
if b {
return vec![42]
}
std::iter::empty().collect() // Works, magic (accidentally stabilized, not intended)
}
```
But when we are working with the return value of a recursive call, the behavior of RPIT and TAIT is the same:
```rust
type Foo = impl std::fmt::Debug;
fn foo(b: bool) -> Foo {
if b {
return vec![];
}
let mut x = foo(false);
x = std::iter::empty().collect(); //~ ERROR `Foo` cannot be built from an iterator
vec![]
}
fn bar(b: bool) -> impl std::fmt::Debug {
if b {
return vec![];
}
let mut x = bar(false);
x = std::iter::empty().collect(); //~ ERROR `impl Debug` cannot be built from an iterator
vec![]
}
```
### user visible change 3: TAIT does not merge types across branches
In contrast to RPIT, TAIT does not merge types across branches, so the following does not compile.
```rust
type Foo = impl std::fmt::Debug;
fn foo(b: bool) -> Foo {
if b {
vec![42_i32]
} else {
std::iter::empty().collect()
//~^ ERROR `Foo` cannot be built from an iterator over elements of type `_`
}
}
```
It is easy to support, but we should make an explicit decision to include the additional complexity in the implementation (it's not much, see a721052457cf513487fb4266e3ade65c29b272d2 which needs to be reverted to enable this).
### PR formalities
previous attempt: #92007
This PR also includes #92306 and #93783, as they were reverted along with #92007 in #93893fixes#93411fixes#88236fixes#89312fixes#87340fixes#86800fixes#86719fixes#84073fixes#83919fixes#82139fixes#77987fixes#74282fixes#67830fixes#62742fixes#54895
Eric figured out the fix to this almost 2 years ago, I just didn't read his comment carefully enough at the timme.
The issue was that fake rustc and fake rustdoc were inconsistent about when they passed `--sysroot` to the real compiler.
Change them to consistently only pass it when `--target` is present.
There were two fixes needed:
1. Use `top_stage` instead of `top_stage - 1`. There was a long and torturous comment about trying to match rustdoc's version, but it works better without the hard-coding than with.
2. Make sure that `ci-llvm/lib` is added to LD_LIBRARY_PATH. Previously the error index would be unable to load LLVM for stage0 builds.
At some point we should probably have a discussion about how rustdoc stages should be numbered;
confusion between 0/1/2 has come up several times in bootstrap now.
Note that this is still broken when using `download-rustc = true` and `--stage 1`,
but that's *really* a corner case and should affect almost no one. `--stage {0,2}`
work fine with download-rustc.
bootstrap: better error message for no_std docs
Currently if one tries to build std documentation for a no_std target, you get a confusing error message:
`error: The argument '--package [<SPEC>...]' was provided more than once, but cannot be used multiple times`
This is because [`std_cargo`](600ec28483/src/bootstrap/compile.rs (L299-L305)) has a built-in `-p alloc` argument that conflicts with the `cargo rustdoc` command used in the Std doc step.
This just adds a better error message in this scenario. It may be possible to fix this correctly, but that would likely be a bit more of an invasive change that I don't have time for right now.
parallel_compiler: hide dependencies behind feature
Separate dependencies for `parallel_compiler` feature, so they will not be compiled if feature not selected, reducing number of compiled crates from 238 to 224.
[bootstrap] Don't print `Suite not skipped` unless `--verbose` is set
This was so verbose before that it made it hard to see what effect the flag actually had.
Before:
```
Set({test::src/tools/tidy}) not skipped for "bootstrap::test::Tidy" -- not in [src/test/ui, src/test/mir-opt/, src/test/debuginfo, src/test/ui-fulldeps]
Skipping Suite(test::src/test/ui) because it is excluded
Suite(test::src/test/run-pass-valgrind) not skipped for "bootstrap::test::RunPassValgrind" -- not in [src/test/ui, src/test/mir-opt/, src/test/debuginfo, src/test/ui-fulldeps]
Skipping Suite(test::src/test/mir-opt) because it is excluded
Suite(test::src/test/codegen) not skipped for "bootstrap::test::Codegen" -- not in [src/test/ui, src/test/mir-opt/, src/test/debuginfo, src/test/ui-fulldeps]
Suite(test::src/test/codegen-units) not skipped for "bootstrap::test::CodegenUnits" -- not in [src/test/ui, src/test/mir-opt/, src/test/debuginfo, src/test/ui-fulldeps]
Suite(test::src/test/assembly) not skipped for "bootstrap::test::Assembly" -- not in [src/test/ui, src/test/mir-opt/, src/test/debuginfo, src/test/ui-fulldeps]
Suite(test::src/test/incremental) not skipped for "bootstrap::test::Incremental" -- not in [src/test/ui, src/test/mir-opt/, src/test/debuginfo, src/test/ui-fulldeps]
Skipping Suite(test::src/test/debuginfo) because it is excluded
Skipping Suite(test::src/test/ui-fulldeps) because it is excluded
... about 100 more lines ...
```
After:
```
Skipping Suite(test::src/test/ui) because it is excluded
Skipping Suite(test::src/test/mir-opt) because it is excluded
Skipping Suite(test::src/test/debuginfo) because it is excluded
Skipping Suite(test::src/test/ui-fulldeps) because it is excluded
```
This was so verbose before that it made it hard to see what effect the flag actually had.
Before:
```
Set({test::src/tools/tidy}) not skipped for "bootstrap::test::Tidy" -- not in [src/test/ui, src/test/mir-opt/, src/test/debuginfo, src/test/ui-fulldeps]
Skipping Suite(test::src/test/ui) because it is excluded
Suite(test::src/test/run-pass-valgrind) not skipped for "bootstrap::test::RunPassValgrind" -- not in [src/test/ui, src/test/mir-opt/, src/test/debuginfo, src/test/ui-fulldeps]
Skipping Suite(test::src/test/mir-opt) because it is excluded
Suite(test::src/test/codegen) not skipped for "bootstrap::test::Codegen" -- not in [src/test/ui, src/test/mir-opt/, src/test/debuginfo, src/test/ui-fulldeps]
Suite(test::src/test/codegen-units) not skipped for "bootstrap::test::CodegenUnits" -- not in [src/test/ui, src/test/mir-opt/, src/test/debuginfo, src/test/ui-fulldeps]
Suite(test::src/test/assembly) not skipped for "bootstrap::test::Assembly" -- not in [src/test/ui, src/test/mir-opt/, src/test/debuginfo, src/test/ui-fulldeps]
Suite(test::src/test/incremental) not skipped for "bootstrap::test::Incremental" -- not in [src/test/ui, src/test/mir-opt/, src/test/debuginfo, src/test/ui-fulldeps]
Skipping Suite(test::src/test/debuginfo) because it is excluded
Skipping Suite(test::src/test/ui-fulldeps) because it is excluded
... about 100 more lines ...
```
After:
```
Skipping Suite(test::src/test/ui) because it is excluded
Skipping Suite(test::src/test/mir-opt) because it is excluded
Skipping Suite(test::src/test/debuginfo) because it is excluded
Skipping Suite(test::src/test/ui-fulldeps) because it is excluded
```
Previously, it would erroneously try to run the doc-tests anyway and give an error:
```
Doc-tests rustdoc
thread 'main' panicked at 'RUSTDOC_LIBDIR was not set', src/bootstrap/bin/rustdoc.rs:15:48
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
error: test failed, to rerun pass '--doc'
```
The problem was two-fold:
- Bootstrap was hard-coding that unit tests should always run with stage1, not stage2, and
- It hard-coded the sysroot layout in stage1, which puts libLLVM.so in `lib/rustlib/` instead of just `lib/`.
A common issue people run into when running compiletest is that filtering for files that don't exist is only a warning and not an error; running the whole test suite instead.
See for example https://rust-lang.zulipchat.com/#narrow/stream/182449-t-compiler.2Fhelp/topic/Question.20about.20compiletest.
This is especially bad when using `--bless`, which will modify all `.stderr` files.
Change bootstrap to require valid filters instead of discarding invalid filters and continuing.
Before:
```
Warning: Skipping "/home/jnelson/rust-lang/rust/src/test/rustdoc-ui/intra-doc/feature-gate-intra-doc-pointers.r": not a regular file or directory
Check compiletest suite=rustdoc-ui mode=ui (x86_64-unknown-linux-gnu(x86_64-unknown-linux-gnu) -> x86_64-unknown-linux-gnu(x86_64-unknown-linux-gnu))
running 163 tests
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.......................... 100/163
...............................................................
test result: ok. 89 passed; 0 failed; 74 ignored; 0 measured; 0 filtered out; finished in 7.20s
finished in 7.248 seconds
Build completed successfully in 0:00:08
```
After:
```
thread 'main' panicked at 'Invalid test suite filter "/home/jnelson/rust-lang/rust/src/test/rustdoc-ui/intra-doc/feature-gate-intra-doc-pointers.r": file or directory does not exist', src/bootstrap/util.rs:311:
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Build completed unsuccessfully in 0:00:08
```
Fixes the following error:
```
error: failed to run custom build command for `bootstrap v0.0.0 (C:\Users\Walther\git\rust\src\bootstrap)`
Caused by:
process didn't exit successfully: `C:\Users\Walther\git\rust\target\debug\build\bootstrap-7757a4777dec0f86\build-script-build` (exit code: 101)
--- stdout
cargo:rerun-if-changed=build.rs
cargo:rerun-if-env-changed=RUSTC
cargo:rustc-env=BUILD_TRIPLE=x86_64-pc-windows-msvc
cargo:rerun-if-env-changed=PATH
--- stderr
thread 'main' panicked at 'assertion failed: rustc.is_absolute()', src\bootstrap\build.rs:22:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: build failed
```
The problem was that the `dir.join` check only works with rustc.exe, not rustc.
Thanks Walther for the help testing the fix!
Per https://www.freedesktop.org/software/systemd/man/os-release.html,
> Variable assignment values must be enclosed in double or single quotes
> if they include spaces, semicolons or other special characters outside
> of A–Z, a–z, 0–9. (Assignments that do not include these special
> characters may be enclosed in quotes too, but this is optional.)
So, past `ID=nixos`, let's also check for `ID='nixos'` and `ID="nixos"`.
One of these is necessary between nixos/nixpkgs#162168 and
nixos/nixpkgs#164068, but this seems more correct either way.
bootstrap: untangle static-libstdcpp & llvm-tools
Previously, the static-libstdcpp setting was tied to llvm-tools such
that enabling the latter always enabled the latter. This seems
unfortunate, since it is entirely reasonable for someone to want to
_not_ statically link stdc++, but _also_ want to build the llvm-tools.
This patch therefore separates the two settings such that neither
implies the other.
On its own, that would change the default behavior in a way that's
likely to surprise users. Specifically, users who build llvm-tools
_likely_ want those tools to be statically compiled against libstdc++,
since otherwise users with older GLIBCXX will be unable to run the
vended tools. So, we also flip the default for the `static-libstdcpp`
setting such that builds always link statically against libstdc++ by
default, but it's _possible_ to opt out.
See also #94719.
Omit stdarch test crates from the rust-src component
These crates aren't necessary for building the standard library. This saves 30MB of disk space.
Fixes#94906
Enable conditional checking of values in the Rust codebase
This pull-request enable conditional checking of (well known) values in the Rust codebase.
Well known values were added in https://github.com/rust-lang/rust/pull/94362. All the `target_*` values are taken from all the built-in targets which is why some extra values were needed do be added as they are not (yet ?) defined in any built-in targets.
r? `@Mark-Simulacrum`
This reduces binary size from 9.7MiB (5.8MiB for just rustbuild code)
to 9.3MiB (5.3MiB for just rustbuild code).
This doesn't reduce compile time in a statistically significant way.
This reduces binary size from 10.1MiB (6.2MiB for just rustbuild code)
to 9.7MiB (5.8MiB for just rustbuild code).
This also reduces compile time from ~6.1s for incr recompilation to ~5.6s.
There is still a lot of
unnecessary code due to the toml crate monomorphizing every
deserialization impl 5 times.
