Add `download-rustc = "if-unchanged"`
This allows keeping the setting to a fixed value without having to
toggle it when you want to work on the compiler instead of on tools.
This sets `BOOTSTRAP_DOWNLOAD_RUSTC` in bootstrap.py so rustbuild doesn't have to try and replicate its logic.
Helps with https://github.com/rust-lang/rust/issues/81930.
r? `@Mark-Simulacrum` cc `@camelid`
Adds bootstrap rules to support installing rust-demangler.
When compiling with `-Z instrument-coverage`, the coverage reports are
generated by `llvm-cov`. `llvm-cov` includes a built-in demangler for
C++, and an option to supply an alternate demangler. For Rust, we have
`rust-demangler`, currently used in `rustc` coverage tests.
Fuchsia's toolchain for Rust is built via `./x.py install`. Fuchsia is
adding support for Rust coverage, and we need to include the
`rust-demangler` in the installed `bin` directory.
Configured rust-demangler as an in-tree extended tool.
Added tests to support `./x.py test rust-demangler`.
Install with extended tools by default only if `profiler = true`.
config.toml parsing error improvements
Improve error messages for musl-libdir and wasi-root keys. Previously
the parser would panic with `unwrap()`. Now it prints
Target "wasm32-wasi" does not have a "wasi-root" key
(and similar for the `musl-libdir` field, which is used in target that
use musl)
Also update comments around wasi-root field to make it clear that the
field is only valid in wasm32-wasi target and needs to be moved to a
`[target.wasm32-wasi]` section to be valid.
Fixes#82317
---
r? `@Mark-Simulacrum`
Improve error messages for musl-libdir and wasi-root keys. Previously
the parser would panic with `unwrap()`. Now it prints
Target "wasm32-wasi" does not have a "wasi-root" key
(and similar for the `musl-libdir` field, which is used in target that
use musl)
Also update comments around wasi-root field to make it clear that the
field is only valid in wasm32-wasi target and needs to be moved to a
`[target.wasm32-wasi]` section to be valid.
Fixes#82317
- Use the same compiler for stage0 and stage1. This should be fixed at
some point (so bootstrap isn't constantly rebuilt).
- Make sure `x.py build` and `x.py check` work.
- Use `git merge-base` to determine the most recent commit to download.
- Copy stage0 to the various sysroots in `Sysroot`, and delegate to
Sysroot in Assemble. Leave all other code unchanged.
- Rename date -> key
This can also be a commit hash, so 'date' is no longer a good name.
- Add the commented-out option to config.toml.example
- Disable all steps by default when `download-rustc` is enabled
Most steps don't make sense when downloading a compiler, because they'll
be pre-built in the sysroot. Only enable the ones that might be useful,
in particular Rustdoc and all `check` steps.
At some point, this should probably enable other tools, but rustdoc is
enough to test out `download-rustc`.
- Don't print 'Skipping' twice in a row
Bootstrap forcibly enables a dry run if it isn't already set, so
previously it would print the message twice:
```
Skipping bootstrap::compile::Std because it is not enabled for `download-rustc`
Skipping bootstrap::compile::Std because it is not enabled for `download-rustc`
```
Now it correctly only prints once.
## Future work
- Add FIXME about supporting beta commits
- Debug logging will never work. This should be fixed.
`dsymutil` adds time to builds on Apple platforms for no clear benefit, and also
makes it more difficult for debuggers to find debug info. The compiler currently
defaults to running `dsymutil` to preserve its historical default, but when
compiling the compiler itself, we skip it by default since we know it's safe to
do so in that case.
This adds `rust.description` to the config as a descriptive string to be
appended to `rustc --version` output, which is also used in places like
debuginfo `DW_AT_producer`. This may be useful for supplementary build
information, like distro-specific package versions.
For example, in Fedora 33, `gcc --version` outputs:
gcc (GCC) 10.2.1 20201016 (Red Hat 10.2.1-6)
With this change, we can add similar vendor info to `rustc --version`.
Support enable/disable sanitizers/profiler per target
This PR add options under `[target.*]` of `config.toml` which can enable or disable sanitizers/profiler runtime for corresponding target.
If these options are empty, the global options under `[build]` will take effect.
Fix#78329
Update `changelog-seen` in config.toml.example
This got out of sync when the version was bumped last time in #77133
Long-term we may want to find an easier way to maintain this that
doesn't require bumping the version in three different places. Off the
top of my head I can't think of anything, though. It _is_ documented in src/bootstrap/README.md, although I don't know how many people read that.
r? @Mark-Simulacrum
cc @spastorino
This got out of sync when the version was bumped last time.
Long-term we may want to find an easier way to maintain this that
doesn't require bumping the version in three different places. Off the
top of my head I can't think of anything, though.
- Add a changelog and instructions for updating it
- Use `changelog-seen` in `config.toml` and `VERSION` in bootstrap to determine whether the changelog has been read
- Nag people if they haven't read the x.py changelog
+ Print message twice to make sure it's seen
- Give different error messages depending on whether the version needs to be updated or added
- Allow including defaults in `src/bootstrap/defaults` using `profile = "..."`
- Add default config files
- Combine config files using the merge dependency.
- Add comments to default config files
- Add a README asking to open an issue if the defaults are bad
- Give a loud error if trying to merge `.target`, since it's not
currently supported
- Use an exhaustive match
- Use `<none>` in config.toml.example to avoid confusion
- Fix bugs in `Merge` derives
Previously, it would completely ignore the profile defaults if there
were any settings in `config.toml`. I sent an email to the `merge` maintainer
asking them to make the behavior in this commit the default.
This introduces a new dependency on `merge` that hasn't yet been vetted.
I want to improve the output when `include = "x"` isn't found:
```
thread 'main' panicked at 'fs::read_to_string(&file) failed with No such file or directory (os error 2) ("configuration file did not exist")', src/bootstrap/config.rs:522:28
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
failed to run: /home/joshua/rustc/build/bootstrap/debug/bootstrap test tidy
Build completed unsuccessfully in 0:00:00
```
However that seems like it could be fixed in a follow-up.
Make the default stage for x.py configurable
This also allows configuring each sub-command individually.
Possibly #76617 should land before this? I don't feel strongly either way, I don't mind waiting.
Closes https://github.com/rust-lang/rust/issues/76165.
r? `@Mark-Simulacrum`
This allows configuring the default stage for each sub-command individually.
- Normalize the stage as early as possible, so there's no confusion
about which to use.
- Don't add an explicit `stage` option in config.toml
This offers no more flexibility than `*_stage` and makes it confusing
which takes precedence.
- Always give `--stage N` precedence over config.toml
- Fix bootstrap tests
This changes the tests to go through `Config::parse` so that they test
the actual defaults, not the dummy ones provided by `default_opts`. To
make this workable (and independent of the environment), it does not
read `config.toml` for tests.
Add a dedicated debug-logging option to config.toml
`@Mark-Simulacrum` and I were talking in zulip and we found that turning on debug/trace logging in rustc is fairly confusing, as it effectively depends on debug-assertions and is not documented as such. `@Mark-Simulacrum` mentioned that we should probably have a separate option for logging anyways.
this diff adds that, having the option follow debug-assertions (so everyone's existing config.toml should be fine) and if the option is false
to test I ran ./x.py test <something> twice, once with `debug-logging = false` and once with `debug-logging = true` and made sure i only saw trace's when it was true
Previously, the CLI --target/--host definitions and configured options differed
in their effect: when setting these on the CLI, only the passed triples would be
compiled for, while in config.toml we would also compile for the build triple
and any host triples. This is needlessly confusing; users expect --target and
--host to be identical to editing the configuration file.
The new behavior is to respect --host and --target when passed as the *only*
configured triples (no triples are implicitly added). The default for --host is
the build triple, and the default for --target is the host triple(s), either
configured or the default build triple.
rustbuild: Build tests with LLD if `use-lld = true` was passed
Addresses https://github.com/rust-lang/rust/pull/76127#discussion_r479932392.
Our test suite is generally ready to run with an explicitly specified linker (https://github.com/rust-lang/rust/pull/45191),
so LLD specified with `use-lld = true` works as well.
Only 4 tests fail (on `x86_64-pc-windows-msvc`):
```
ui/panic-runtime/lto-unwind.rs
run-make-fulldeps/debug-assertions
run-make-fulldeps/foreign-exceptions
run-make-fulldeps/test-harness
```
All of them are legitimate issues with LLD (or at least with combination Rust+LLD) and manifest in segfaults on access to TLS (https://github.com/rust-lang/rust/pull/76127#issuecomment-683473325). UPD: These issues are caused by https://github.com/rust-lang/rust/issues/72145 and appear because I had `-Ctarget-cpu=native` set.
UPD: Further commits build tests with LLD for non-MSVC targets and propagate LLD to more places when `use-lld` is enabled.
Move `rustllvm` into `compiler/rustc_llvm`
The `rustllvm` directory is not self-contained, it contains C++ code built by a build script of the `rustc_llvm` crate which is then linked into that crate.
So it makes sense to make `rustllvm` a part of `rustc_llvm` and move it into its directory.
I replaced `rustllvm` with more obvious `llvm-wrapper` as the subdirectory name, but something like `llvm-adapter` would work as well, other suggestions are welcome.
To make things more confusing, the Rust side of FFI functions defined in `rustllvm` can be found in `rustc_codegen_llvm` rather than in `rustc_llvm`. Perhaps they need to be moved as well, but this PR doesn't do that.
The presence of multiple LLVM-related directories in `src` (`llvm-project`, `rustllvm`, `librustc_llvm`, `librustc_codegen_llvm` and their predecessors) historically confused me and made me wonder about their purpose.
With this PR we will have LLVM itself (`llvm-project`), a FFI crate (`rustc_llvm`, kind of `llvm-sys`) and a codegen backend crate using LLVM through the FFI crate (`rustc_codegen_llvm`).
- Use prettier syntax for codegen-units defaults
- Remove comment about parallelism that only made sense for specific
values of codegen-units
- Be more specific about what `verbose-tests` does
Ninja substantially improves LLVM build time. On a 96-way system, using
Make took 248s, and using Ninja took 161s, a 35% improvement.
We already require a variety of tools to build Rust. If someone wants to
build without Ninja (for instance, to minimize the set of packages
required to bootstrap a new target), they can easily set `ninja=false`
in `config.toml`. Our defaults should help people build Rust (and LLVM)
faster, to speed up development.
Improve defaults in x.py
- Make the default stage dependent on the subcommand
- Don't build stage1 rustc artifacts with x.py build --stage 1. If this is what you want, use x.py build --stage 2 instead, which gives you a working libstd.
- Change default debuginfo when debug = true from 2 to 1
I tried to fix CI to use `--stage 2` everywhere it currently has no stage, but I might have missed a spot.
This does not update much of the documentation - most of it is in https://github.com/rust-lang/rustc-dev-guide/ or https://github.com/rust-lang/rust-forge and will need a separate PR.
See individual commits for a detailed rationale of each change.
See also the MCP: https://github.com/rust-lang/compiler-team/issues/326
r? @Mark-Simulacrum , but anyone is free to give an opinion.
From [a conversation in discord](https://discordapp.com/channels/442252698964721669/443151243398086667/719200989269327882):
> Linking seems to consume all available RAM, leading to the OS to swap memory to disk and slowing down everything in the process
Compiling itself doesn't seem to take up as much RAM, and I'm only looking to check whether a minimal testcase can be compiled by rustc, where the runtime performance isn't much of an issue
> do you have debug = true or debuginfo-level = 2 in config.toml?
> if so I think that results in over 2GB of debuginfo nowadays and is likely the culprit
> which might mean we're giving out bad advice :(
Anecdotally, this sped up my stage 1 build from 15 to 10 minutes.
This still adds line numbers, it only removes variable and type information.
- Improve wording for debuginfo description
Co-authored-by: Teymour Aldridge <42674621+teymour-aldridge@users.noreply.github.com>
