This commit removes all infrastructure from the repository for our so-called
snapshots to instead bootstrap the compiler from stable releases. Bootstrapping
from a previously stable release is a long-desired feature of distros because
they're not fans of downloading binary stage0 blobs from us. Additionally, this
makes our own CI easier as we can decommission all of the snapshot builders and
start having a regular cadence to when we update the stage0 compiler.
A new `src/etc/get-stage0.py` script was added which shares some code with
`src/bootstrap/bootstrap.py` to read a new file, `src/stage0.txt`, which lists
the current stage0 compiler as well as cargo that we bootstrap from. This script
will download the relevant `rustc` package an unpack it into `$target/stage0` as
we do today.
One problem of bootstrapping from stable releases is that we're not able to
compile unstable code (e.g. all the `#![feature]` directives in libcore/libstd).
To overcome this we employ two strategies:
* The bootstrap key of the previous compiler is hardcoded into `src/stage0.txt`
(enabled as a result of #32731) and exported by the build system. This enables
nightly features in the compiler we download.
* The standard library and compiler are pinned to a specific stage0, which
doesn't change, so we're guaranteed that we'll continue compiling as we start
from a known fixed source.
The process for making a release will also need to be tweaked now to continue to
cadence of bootstrapping from the previous release. This process looks like:
1. Merge `beta` to `stable`
2. Produce a new stable compiler.
3. Change `master` to bootstrap from this new stable compiler.
4. Merge `master` to `beta`
5. Produce a new beta compiler
6. Change `master` to bootstrap from this new beta compiler.
Step 3 above should involve very few changes as `master` was previously
bootstrapping from `beta` which is the same as `stable` at that point in time.
Step 6, however, is where we benefit from removing lots of `#[cfg(stage0)]` and
get to use new features. This also shouldn't slow the release too much as steps
1-5 requires little work other than waiting and step 6 just needs to happen at
some point during a release cycle, it's not time sensitive.
Closes#29555Closes#29557
The performance hit from these checks is significant, but unoptimized
builds are already incredibly slow. Enabling these checks results in
better test coverage since there are bots doing unoptimized builds, and
the cost is relatively small in the context of an unoptimized build.
This also allows using `JEMALLOC_FLAGS` to override the default
configure flags.
The current suite of benchmarks for the standard distribution take a significant
amount of time to run, but it's unclear whether we're gaining any benefit from
running them. Some specific pain points:
* No one is looking at the data generated by the benchmarks. We have no graphs
or analysis of what's happening, so all the data is largely being cast into
the void.
* No benchmark has ever uncovered a bug, they have always run successfully.
* Benchmarks not only take a significant amount of time to run, but also take a
significant amount of time to compile. I don't think we should mitigate this
for now because it's useful to ensure that they do indeed still compile.
This commit disables --bench test runs by default as part of `make check`,
flipping the NO_BENCH environment variable to a PLEASE_BENCH variable which will
manually enable benchmarking. If and when a dedicated bot is set up for
benchmarking, this flag can be enabled on that bot.
The previous dependency calculation was based on an arbitrary set of asterisks
at an arbitrary depth, but using the recursive version should be much more
robust in figuring out what's dependent.
The previous dependency calculation was based on an arbitrary set of asterisks
at an arbitrary depth, but using the recursive version should be much more
robust in figuring out what's dependent.
Closes#13118
Because the build system treats Makefile.in and the .mk files slightly
differently (.in is copied, .mk are included), this makes the system
more uniform. Fewer build system changes will require a complete
reconfigure.
This is blocking a snapshot because the stage0 target compiler comes from a
stage1 host compiler. This means that the stage0 compiler doesn't actually
understand the -Z prefer-dynamic flag and is dying as a result.
This will get added back to stage0 after a snapshot.
It was decided a long, long time ago that libextra should not exist, but rather its modules should be split out into smaller independent libraries maintained outside of the compiler itself. The theory was to use `rustpkg` to manage dependencies in order to move everything out of the compiler, but maintain an ease of usability.
Sadly, the work on `rustpkg` isn't making progress as quickly as expected, but the need for dissolving libextra is becoming more and more pressing. Because of this, we've thought that a good interim solution would be to simply package more libraries with the rust distribution itself. Instead of dissolving libextra into libraries outside of the mozilla/rust repo, we can dissolve libraries into the mozilla/rust repo for now.
Work on this has been excruciatingly painful in the past because the makefiles are completely opaque to all but a few. Adding a new library involved adding about 100 lines spread out across 8 files (incredibly error prone). The first commit of this pull request targets this pain point. It does not rewrite the build system, but rather refactors large portions of it. Afterwards, adding a new library is as simple as modifying 2 lines (easy, right?). The build system automatically keeps track of dependencies between crates (rust *and* native), promotes binaries between stages, tracks dependencies of installed tools, etc, etc.
With this newfound buildsystem power, I chose the `extra::flate` module as the first candidate for removal from libextra. While a small module, this module is relative complex in that is has a C dependency and the compiler requires it (messing with the dependency graph a bit). Albeit I modified more than 2 lines of makefiles to accomodate libflate (the native dependency required 2 extra lines of modifications), but the removal process was easy to do and straightforward.
---
Testing-wise, I've cross-compiled, run tests, built some docs, installed, uninstalled, etc. I'm still working out a few kinks, and I'm sure that there's gonna be built system issues after this, but it should be working well for basic use!
cc #8784
Before this patch, if you wanted to add a crate to the build system you had to
change about 100 lines across 8 separate makefiles. This is highly error prone
and opaque to all but a few. This refactoring is targeted at consolidating this
effort so adding a new crate adds one line in one file in a way that everyone
can understand it.
By default, the compiler and libraries are all still built with rpaths, but this
can be opted out of with --disable-rpath to ./configure or --no-rpath to rustc.
Closes#5219
By default, the compiler and libraries are all still built with rpaths, but this
can be opted out of with --disable-rpath to ./configure or --no-rpath to rustc.
cc #5219
