02538d463a
This commit removes all infrastructure from the repository for our so-called snapshots to instead bootstrap the compiler from stable releases. Bootstrapping from a previously stable release is a long-desired feature of distros because they're not fans of downloading binary stage0 blobs from us. Additionally, this makes our own CI easier as we can decommission all of the snapshot builders and start having a regular cadence to when we update the stage0 compiler. A new `src/etc/get-stage0.py` script was added which shares some code with `src/bootstrap/bootstrap.py` to read a new file, `src/stage0.txt`, which lists the current stage0 compiler as well as cargo that we bootstrap from. This script will download the relevant `rustc` package an unpack it into `$target/stage0` as we do today. One problem of bootstrapping from stable releases is that we're not able to compile unstable code (e.g. all the `#![feature]` directives in libcore/libstd). To overcome this we employ two strategies: * The bootstrap key of the previous compiler is hardcoded into `src/stage0.txt` (enabled as a result of #32731) and exported by the build system. This enables nightly features in the compiler we download. * The standard library and compiler are pinned to a specific stage0, which doesn't change, so we're guaranteed that we'll continue compiling as we start from a known fixed source. The process for making a release will also need to be tweaked now to continue to cadence of bootstrapping from the previous release. This process looks like: 1. Merge `beta` to `stable` 2. Produce a new stable compiler. 3. Change `master` to bootstrap from this new stable compiler. 4. Merge `master` to `beta` 5. Produce a new beta compiler 6. Change `master` to bootstrap from this new beta compiler. Step 3 above should involve very few changes as `master` was previously bootstrapping from `beta` which is the same as `stable` at that point in time. Step 6, however, is where we benefit from removing lots of `#[cfg(stage0)]` and get to use new features. This also shouldn't slow the release too much as steps 1-5 requires little work other than waiting and step 6 just needs to happen at some point during a release cycle, it's not time sensitive. Closes #29555 Closes #29557
Bootstrapping Rust
This is an in-progress README which is targeted at helping to explain how Rust is bootstrapped and in general some of the technical details of the build system.
Note
: This build system is currently under active development and is not intended to be the primarily used one just yet. The makefiles are currently the ones that are still "guaranteed to work" as much as possible at least.
Using the new build system
When configuring Rust via ./configure, pass the following to enable building
via this build system:
./configure --enable-rustbuild
...
Directory Layout
This build system houses all output under the target directory, which looks
like this:
# Root folder of all output. Everything is scoped underneath here
build/
# Location where the stage0 compiler downloads are all cached. This directory
# only contains the tarballs themselves as they're extracted elsewhere.
cache/
2015-12-19/
2016-01-15/
2016-01-21/
...
# Output directory for building this build system itself. The stage0
# cargo/rustc are used to build the build system into this location.
bootstrap/
debug/
release/
# Each remaining directory is scoped by the "host" triple of compilation at
# hand.
x86_64-unknown-linux-gnu/
# The build artifacts for the `compiler-rt` library for the target this
# folder is under. The exact layout here will likely depend on the platform,
# and this is also built with CMake so the build system is also likely
# different.
compiler-rt/build/
# Output folder for LLVM if it is compiled for this target
llvm/
# build folder (e.g. the platform-specific build system). Like with
# compiler-rt this is compiled with CMake
build/
# Installation of LLVM. Note that we run the equivalent of 'make install'
# for LLVM to setup these folders.
bin/
lib/
include/
share/
...
# Location where the stage0 Cargo and Rust compiler are unpacked. This
# directory is purely an extracted and overlaid tarball of these two (done
# by the bootstrapy python script). In theory the build system does not
# modify anything under this directory afterwards.
stage0/
# These to build directories are the cargo output directories for builds of
# the standard library and compiler, respectively. Internally these may also
# have other target directories, which represent artifacts being compiled
# from the host to the specified target.
#
# Essentially, each of these directories is filled in by one `cargo`
# invocation. The build system instruments calling Cargo in the right order
# with the right variables to ensure these are filled in correctly.
stageN-std/
stageN-rustc/
# This is a special case of the above directories, **not** filled in via
# Cargo but rather the build system itself. The stage0 compiler already has
# a set of target libraries for its own host triple (in its own sysroot)
# inside of stage0/. When we run the stage0 compiler to bootstrap more
# things, however, we don't want to use any of these libraries (as those are
# the ones that we're building). So essentially, when the stage1 compiler is
# being compiled (e.g. after libstd has been built), *this* is used as the
# sysroot for the stage0 compiler being run.
#
# Basically this directory is just a temporary artifact use to configure the
# stage0 compiler to ensure that the libstd we just built is used to
# compile the stage1 compiler.
stage0-rustc/lib/
# These output directories are intended to be standalone working
# implementations of the compiler (corresponding to each stage). The build
# system will link (using hard links) output from stageN-{std,rustc} into
# each of these directories.
#
# In theory there is no extra build output in these directories.
stage1/
stage2/
stage3/