In Linux distributions, it is often necessary to rebuild packages for
cases like applying new patches or linking against new system libraries.
In this scenario, the rustc in the distro build environment may already
match the current release that we're trying to rebuild. Thus we don't
want to use the prior release's bootstrap key, nor `--cfg stage0` for
the prior unstable features.
The new `configure --enable-local-rebuild` option specifies that we are
rebuilding from the current release. The current bootstrap key is used
for the local rustc, and current stage1 features are also assumed.
This commit removes all infrastructure from the repository for our so-called
snapshots to instead bootstrap the compiler from stable releases. Bootstrapping
from a previously stable release is a long-desired feature of distros because
they're not fans of downloading binary stage0 blobs from us. Additionally, this
makes our own CI easier as we can decommission all of the snapshot builders and
start having a regular cadence to when we update the stage0 compiler.
A new `src/etc/get-stage0.py` script was added which shares some code with
`src/bootstrap/bootstrap.py` to read a new file, `src/stage0.txt`, which lists
the current stage0 compiler as well as cargo that we bootstrap from. This script
will download the relevant `rustc` package an unpack it into `$target/stage0` as
we do today.
One problem of bootstrapping from stable releases is that we're not able to
compile unstable code (e.g. all the `#![feature]` directives in libcore/libstd).
To overcome this we employ two strategies:
* The bootstrap key of the previous compiler is hardcoded into `src/stage0.txt`
(enabled as a result of #32731) and exported by the build system. This enables
nightly features in the compiler we download.
* The standard library and compiler are pinned to a specific stage0, which
doesn't change, so we're guaranteed that we'll continue compiling as we start
from a known fixed source.
The process for making a release will also need to be tweaked now to continue to
cadence of bootstrapping from the previous release. This process looks like:
1. Merge `beta` to `stable`
2. Produce a new stable compiler.
3. Change `master` to bootstrap from this new stable compiler.
4. Merge `master` to `beta`
5. Produce a new beta compiler
6. Change `master` to bootstrap from this new beta compiler.
Step 3 above should involve very few changes as `master` was previously
bootstrapping from `beta` which is the same as `stable` at that point in time.
Step 6, however, is where we benefit from removing lots of `#[cfg(stage0)]` and
get to use new features. This also shouldn't slow the release too much as steps
1-5 requires little work other than waiting and step 6 just needs to happen at
some point during a release cycle, it's not time sensitive.
Closes#29555Closes#29557
Starting with the 1.10.0 release we would like to bootstrap all compilers from
the previous stable release. For example the 1.10.0 compiler should bootstrap
from the literal 1.9.0 release artifacts. To do this, however, we need a way to
enable unstable features temporarily in a stable compiler (as the released
compiler is stable), but it turns out we already have a way to do that!
At compile time the configure script selects a `CFG_BOOTSTRAP_KEY` variable
value and then exports it into the makefiles. If the `RUSTC_BOOTSTRAP_KEY`
environment variable is set to this value, then the compiler is allowed to
"cheat" and use unstable features.
This method of choosing the bootstrap key, however, is problematic for the
intention of bootstrapping from the previous release. Each time a 1.9.0 compiler
is created, a new bootstrap key will be selected. That means that the 1.10.0
compiler will only compile from *our* literal release artifacts. Instead
distributions would like to bootstrap from their own compilers, so instead we
simply hardcode the bootstrap key for each release.
This patch uses the same `CFG_FILENAME_EXTRA` value (a hash of the release
string) as the bootstrap key. Consequently all 1.9.0 compilers, no matter where
they are compiled, will have the same bootstrap key. Additionally we won't need
to keep updating this as it'll be based on the release number anyway.
Once the 1.9.0 beta has been created, we can update the 1.10.0 nightly sources
(the `master` branch at that time) to bootstrap from that release using this
hard-coded bootstrap key. We will likely just hardcode into the makefiles what
the previous bootstrap key was and we'll change that whenever the stage0
compiler is updated.
The change in b20e748 had the unintended consequence of breaking cross-host
builds as we apparently relied on the incorrect definition of this variable in
the makefiles. That change, however, was required to get tests passing so we
couldn't just revert it.
This commit fixes the underlying bug by leaving the "more correct" definition of
`LD_LIBRARY_PATH_ENV_TARGETDIR` (also fixing it with a hardcoded reference to
`CFG_BUILD`) and updating the `RPATH_VAR` definition below. Turned out we
already had special-casing logic for passing `--cfg stage1` during the
well-we-print-this-as-stage0 build of a cross-host. That logic was just updated
to pull from a different variable as opposed to relying on the definition of
that variable to accommodate this.
Closes#32568
The `--disable-jemalloc` configure option has a failure mode where it will
create a distribution that is not compatible with other compilers. For example
the nightly for Linux will assume that it will link to jemalloc by default as
an allocator for executable crates. If, however, a standard library is used
which was built via `./configure --disable-jemalloc` then this will fail
because the jemalloc crate wasn't built.
While this seems somewhat reasonable as a niche situation, the same mechanism is
used for disabling jemalloc for platforms that just don't support it. For
example if the rumprun target is compiled then the sibiling Linux target *also*
doesn't have jemalloc. This is currently a problem for our cross-build nightlies
which build many targets. If rumprun is also built, it will disable jemalloc for
all targets, which isn't desired.
This commit moves the platform-specific disabling of jemalloc as hardcoded logic
into the makefiles that is scoped per-platform. This way when configuring
multiple targets **without the `--disable-jemalloc` option specified** all
targets will get jemalloc as they should.
This mixes in additional information into the hash that is
passed to -C extra-filename. It can be used to further distinguish
the standard libraries if they must be installed next to each
other.
Closes#29559
Frankly, I'm not sure if this solves a real problem. It's meant to help with side-by-side and overlapping installations where there are two sets of libs in /usr, but there are other potential issues there as well, including that some of our artifacts don't use this extra-filename munging, and it's not something our installers can support at all.
cc @jauhien Do you still think this helps the Gentoo case?
This mixes in additional information into the hash that is
passed to -C extra-filename. It can be used to further distinguish
the standard libraries if they must be installed next to each
other.
Closes#29559
This handles cases when the LLVM used isn't configured will the 'usual'
targets. Also, cases where LLVM is shared are also handled (ie with
`LD_LIBRARY_PATH` etc).
This is to handle the case where CFG_LIBDIR is not a direct child of
CFG_PREFIX (in other words, where CFG_LIBDIR_RELATIVE has more than
one component).
This fixes the case where we try to re-build & re-install rust to the
same prefix (without uninstalling) while using an llvm-root that is the
same as the prefix.
Without this, builds like that fail with:
'error: multiple dylib candidates for `std` found'
See https://github.com/jmesmon/meta-rust/issues/6 for some details.
May also be related to #20342.
