Sanity check Python on OSX for LLDB tests
Two primary changes:
* Don't get past the configure stage if `python` isn't coming from `/usr/bin`
* Call `debugger.Terminate()` to prevent segfaults on newer versions of LLDB.
Closes#32994
This uncovered a lot of bugs in compiletest and also some shortcomings
of our existing JSON output. We had to add information to the JSON
output, such as suggested text and macro backtraces. We also had to fix
various bugs in the existing tests.
Joint work with jntrnr.
Compute `target_feature` from LLVM
This is a work-in-progress fix for #31662.
The logic that computes the target features from the command line has been replaced with queries to the `TargetMachine`.
This commit removes all infrastructure from the repository for our so-called
snapshots to instead bootstrap the compiler from stable releases. Bootstrapping
from a previously stable release is a long-desired feature of distros because
they're not fans of downloading binary stage0 blobs from us. Additionally, this
makes our own CI easier as we can decommission all of the snapshot builders and
start having a regular cadence to when we update the stage0 compiler.
A new `src/etc/get-stage0.py` script was added which shares some code with
`src/bootstrap/bootstrap.py` to read a new file, `src/stage0.txt`, which lists
the current stage0 compiler as well as cargo that we bootstrap from. This script
will download the relevant `rustc` package an unpack it into `$target/stage0` as
we do today.
One problem of bootstrapping from stable releases is that we're not able to
compile unstable code (e.g. all the `#![feature]` directives in libcore/libstd).
To overcome this we employ two strategies:
* The bootstrap key of the previous compiler is hardcoded into `src/stage0.txt`
(enabled as a result of #32731) and exported by the build system. This enables
nightly features in the compiler we download.
* The standard library and compiler are pinned to a specific stage0, which
doesn't change, so we're guaranteed that we'll continue compiling as we start
from a known fixed source.
The process for making a release will also need to be tweaked now to continue to
cadence of bootstrapping from the previous release. This process looks like:
1. Merge `beta` to `stable`
2. Produce a new stable compiler.
3. Change `master` to bootstrap from this new stable compiler.
4. Merge `master` to `beta`
5. Produce a new beta compiler
6. Change `master` to bootstrap from this new beta compiler.
Step 3 above should involve very few changes as `master` was previously
bootstrapping from `beta` which is the same as `stable` at that point in time.
Step 6, however, is where we benefit from removing lots of `#[cfg(stage0)]` and
get to use new features. This also shouldn't slow the release too much as steps
1-5 requires little work other than waiting and step 6 just needs to happen at
some point during a release cycle, it's not time sensitive.
Closes#29555Closes#29557
This commit adds support in rustbuild for running all of the compiletest test
suites as part of `make check`. The `compiletest` program was moved to
`src/tools` (like `rustbook` and others) and is now just compiled like any other
old tool. Each test suite has a pretty standard set of dependencies and just
tweaks various parameters to the final compiletest executable.
Note that full support is lacking in terms of:
* Once a test suite has passed, that's not remembered. When a test suite is
requested to be run, it's always run.
* The arguments to compiletest probably don't work for every possible
combination of platforms and testing environments just yet. There will likely
need to be future updates to tweak various pieces here and there.
* Cross compiled test suites probably don't work just yet, support for that will
come in a follow-up patch.
This verifies that the crates listed in the `[dependencies]` section of
`Cargo.toml` are a subset of the crates listed in `lib.rs` for our in-tree
crates. This should help ensure that when we refactor crates over time we keep
these dependency lists in sync.
This commit rewrites all of the tidy checks we have, namely:
* featureck
* errorck
* tidy
* binaries
into Rust under a new `tidy` tool inside of the `src/tools` directory. This at
the same time deletes all the corresponding Python tidy checks so we can be sure
to only have one source of truth for all the tidy checks.
cc #31590
mk: Hardcode the bootstrap key for each release
Starting with the 1.10.0 release we would like to bootstrap all compilers from
the previous stable release. For example the 1.10.0 compiler should bootstrap
from the literal 1.9.0 release artifacts. To do this, however, we need a way to
enable unstable features temporarily in a stable compiler (as the released
compiler is stable), but it turns out we already have a way to do that!
At compile time the configure script selects a `CFG_BOOTSTRAP_KEY` variable
value and then exports it into the makefiles. If the `RUSTC_BOOTSTRAP_KEY`
environment variable is set to this value, then the compiler is allowed to
"cheat" and use unstable features.
This method of choosing the bootstrap key, however, is problematic for the
intention of bootstrapping from the previous release. Each time a 1.9.0 compiler
is created, a new bootstrap key will be selected. That means that the 1.10.0
compiler will only compile from *our* literal release artifacts. Instead
distributions would like to bootstrap from their own compilers, so instead we
simply hardcode the bootstrap key for each release.
This patch uses the same `CFG_FILENAME_EXTRA` value (a hash of the release
string) as the bootstrap key. Consequently all 1.9.0 compilers, no matter where
they are compiled, will have the same bootstrap key. Additionally we won't need
to keep updating this as it'll be based on the release number anyway.
Once the 1.9.0 beta has been created, we can update the 1.10.0 nightly sources
(the `master` branch at that time) to bootstrap from that release using this
hard-coded bootstrap key. We will likely just hardcode into the makefiles what
the previous bootstrap key was and we'll change that whenever the stage0
compiler is updated.
Starting with the 1.10.0 release we would like to bootstrap all compilers from
the previous stable release. For example the 1.10.0 compiler should bootstrap
from the literal 1.9.0 release artifacts. To do this, however, we need a way to
enable unstable features temporarily in a stable compiler (as the released
compiler is stable), but it turns out we already have a way to do that!
At compile time the configure script selects a `CFG_BOOTSTRAP_KEY` variable
value and then exports it into the makefiles. If the `RUSTC_BOOTSTRAP_KEY`
environment variable is set to this value, then the compiler is allowed to
"cheat" and use unstable features.
This method of choosing the bootstrap key, however, is problematic for the
intention of bootstrapping from the previous release. Each time a 1.9.0 compiler
is created, a new bootstrap key will be selected. That means that the 1.10.0
compiler will only compile from *our* literal release artifacts. Instead
distributions would like to bootstrap from their own compilers, so instead we
simply hardcode the bootstrap key for each release.
This patch uses the same `CFG_FILENAME_EXTRA` value (a hash of the release
string) as the bootstrap key. Consequently all 1.9.0 compilers, no matter where
they are compiled, will have the same bootstrap key. Additionally we won't need
to keep updating this as it'll be based on the release number anyway.
Once the 1.9.0 beta has been created, we can update the 1.10.0 nightly sources
(the `master` branch at that time) to bootstrap from that release using this
hard-coded bootstrap key. We will likely just hardcode into the makefiles what
the previous bootstrap key was and we'll change that whenever the stage0
compiler is updated.
Our `codegen` test suite requires the LLVM `FileCheck` utility but unfortunately
this isn't always available in all custom LLVM roots (e.g. those specified via
`--llvm-root`). This commit adds a `./configure` option called
`--disable-codegen-tests` which will manually disable running these tests. In
the case that this option is passed we can forgo the need for the `FileCheck`
executable. Note that we still require `FileCheck` by default as we will attempt
to run these tests.
Closes#28667
mk: A few build fixes for i586-pc-windows-msvc
Detect the triple in the configure script for probing MSVC shenanigans and also
be sure to use `llvm-config` from the build host and not the target when
configuring compiler-rt.
mk: Fix cross-host builds
The change in b20e748 had the unintended consequence of breaking cross-host
builds as we apparently relied on the incorrect definition of this variable in
the makefiles. That change, however, was required to get tests passing so we
couldn't just revert it.
This commit fixes the underlying bug by leaving the "more correct" definition of
`LD_LIBRARY_PATH_ENV_TARGETDIR` (also fixing it with a hardcoded reference to
`CFG_BUILD`) and updating the `RPATH_VAR` definition below. Turned out we
already had special-casing logic for passing `--cfg stage1` during the
well-we-print-this-as-stage0 build of a cross-host. That logic was just updated
to pull from a different variable as opposed to relying on the definition of
that variable to accommodate this.
Closes#32568
Detect the triple in the configure script for probing MSVC shenanigans and also
be sure to use `llvm-config` from the build host and not the target when
configuring compiler-rt.
The change in b20e748 had the unintended consequence of breaking cross-host
builds as we apparently relied on the incorrect definition of this variable in
the makefiles. That change, however, was required to get tests passing so we
couldn't just revert it.
This commit fixes the underlying bug by leaving the "more correct" definition of
`LD_LIBRARY_PATH_ENV_TARGETDIR` (also fixing it with a hardcoded reference to
`CFG_BUILD`) and updating the `RPATH_VAR` definition below. Turned out we
already had special-casing logic for passing `--cfg stage1` during the
well-we-print-this-as-stage0 build of a cross-host. That logic was just updated
to pull from a different variable as opposed to relying on the definition of
that variable to accommodate this.
Closes#32568
This should re-enable all external builds of crates with the same name. Right
now Cargo doesn't pass `-C metadata` for the top-level library being compiled,
so if that library is called `libc`, for example, then it won't be able to link
to the standard library which *also* has a `libc` library compiled without `-C
metadata`. This can result in naming conflicts which need to be resolved.
By passing `-C metadata` to the in-tree crates we ship it should add some extra
salt to all symbol names to ensure that they don't collide.
emit (via debug!) scary message from `fn borrowck_mir` until basic
prototype is in place.
Gather children of move paths and set their kill bits in
dataflow. (Each node has a link to the child that is first among its
siblings.)
Hooked in libgraphviz based rendering, including of borrowck dataflow
state.
doing this well required some refactoring of the code, so I cleaned it
up more generally (adding comments to explain what its trying to do
and how it is doing it).
Update: this newer version addresses most review comments (at least
the ones that were largely mechanical changes), but I left the more
interesting revisions to separate followup commits (in this same PR).
typestrong const integers
~~It would be great if someone could run crater on this PR, as this has a high danger of breaking valid code~~ Crater ran. Good to go.
----
So this PR does a few things:
1. ~~const eval array values when const evaluating an array expression~~
2. ~~const eval repeat value when const evaluating a repeat expression~~
3. ~~const eval all struct and tuple fields when evaluating a struct/tuple expression~~
4. remove the `ConstVal::Int` and `ConstVal::Uint` variants and replace them with a single enum (`ConstInt`) which has variants for all integral types
* `usize`/`isize` are also enums with variants for 32 and 64 bit. At creation and various usage steps there are assertions in place checking if the target bitwidth matches with the chosen enum variant
5. enum discriminants (`ty::Disr`) are now `ConstInt`
6. trans has its own `Disr` type now (newtype around `u64`)
This obviously can't be done without breaking changes (the ones that are noticable in stable)
We could probably write lints that find those situations and error on it for a cycle or two. But then again, those situations are rare and really bugs imo anyway:
```rust
let v10 = 10 as i8;
let v4 = 4 as isize;
assert_eq!(v10 << v4 as usize, 160 as i8);
```
stops compiling because 160 is not a valid i8
```rust
struct S<T, S> {
a: T,
b: u8,
c: S
}
let s = S { a: 0xff_ff_ff_ffu32, b: 1, c: 0xaa_aa_aa_aa as i32 };
```
stops compiling because `0xaa_aa_aa_aa` is not a valid i32
----
cc @eddyb @pnkfelix
related: https://github.com/rust-lang/rfcs/issues/1071
Add a link validator to rustbuild
This commit was originally targeted at just adding a link checking script to the rustbuild system. This ended up snowballing a bit to extend rustbuild to be amenable to various tools we have as part of the build system in general.
There's a new `src/tools` directory which has a number of scripts/programs that are purely intended to be used as part of the build system and CI of this repository. This is currently inhabited by rustbook, the error index generator, and a new linkchecker script added as part of this PR. I suspect that more tools like compiletest, tidy scripts, snapshot scripts, etc will migrate their way into this directory over time.
The commit which adds the error index generator shows the steps necessary to add new tools to the build system, namely:
1. New steps are defined for building the tool and running the tool
2. The dependencies are configured
3. The steps are implemented
In terms of the link checker, these commits do a few things:
* A new `src/tools/linkchecker` script is added. This will read an entire documentation tree looking for broken relative links (HTTP links aren't followed yet).
* A large number of broken links throughout the documentation were fixed. Many of these were just broken when viewed from core as opposed to std, but were easily fixed.
* A few rustdoc bugs here and there were fixed
rustc: Add an i586-pc-windows-msvc target
Similarly to #31629 where an i586-unknown-linux-gnu target was added, there is
sometimes a desire to compile for x86 Windows as well where SSE2 is disabled.
This commit mirrors the i586-unknown-linux-gnu target and simply adds a variant
for Windows as well.
This is motivated by a recent [Gecko bug][ff] where crashes were seen on 32-bit
Windows due to users having CPUs that don't support SSE2 instructions. It was
requested that we could have non-SSE2 builds of the standard library available
so they could continue to use vanilla releases and nightlies.
[ff]: https://bugzilla.mozilla.org/show_bug.cgi?id=1253202
mk: Distribute fewer TARGET_CRATES
Right now everything in TARGET_CRATES is built by default for all non-fulldeps
tests and is distributed by default for all target standard library packages.
Currenly this includes a number of unstable crates which are rarely used such as
`graphviz` and `rbml`>
This commit trims down the set of `TARGET_CRATES`, moves a number of tests to
`*-fulldeps` as a result, and trims down the dependencies of libtest so we can
distribute fewer crates in the `rust-std` packages.
Right now everything in TARGET_CRATES is built by default for all non-fulldeps
tests and is distributed by default for all target standard library packages.
Currenly this includes a number of unstable crates which are rarely used such as
`graphviz` and `rbml`>
This commit trims down the set of `TARGET_CRATES`, moves a number of tests to
`*-fulldeps` as a result, and trims down the dependencies of libtest so we can
distribute fewer crates in the `rust-std` packages.
