rustbuild: Fix LC_ID_DYLIB directives on OSX
Currently libraries installed by rustbuild on OSX have an incorrect
`LC_ID_DYLIB` directive located in the dynamic libraries that are
installed. The directive we expect looks like:
@rpath/libstd.dylib
Which means that if you want to find that dynamic library you should
look at the dylib's other `@rpath` directives. Typically our `@rpath`
directives look like `@loader_path/../lib` for the compiler as that's
where the installed libraries will be located. Currently, though,
rustbuild produces dylibs with the directive that looks like:
/Users/rustbuild/src/rust-buildbot/slave/nightly-dist-rustc-mac/build/build/x86_64-apple-darwin/stage1-std/x86_64-apple-darwin/release/deps/libstd-713ad88203512705.dylib
In other words, the build directory is encoded erroneously. The compiler
already [knows how] to change this directive, but it only passes that
argument when `-C rpath` is also passed. The rustbuild system, however,
explicitly [does not pass] this option explicitly and instead bakes its
own. This logic then also erroneously didn't pass `-Wl,-install_name`
like the compiler.
[knows how]: 4a008cccaa/src/librustc_trans/back/linker.rs (L210-L214)
[does not pass]: 4a008cccaa/src/bootstrap/bin/rustc.rs (L133-L158)
To fix this regression this patch introduces a new `-Z` flag, `-Z
osx-rpath-install-name` which basically just forces the compiler to take
the previous `-install_name` branch when creating a dynamic library.
Hopefully we can sort out a better rpath story in the future, but for
now this "hack" should suffice in getting our nightly builds back to the
same state as before.
Closes#38430
rustbuild: Create directories in mingw dist
Previously we accidentally relied on the mingw dist step running last, but the
step just needed to ensure the directories were created.
Currently libraries installed by rustbuild on OSX have an incorrect
`LC_ID_DYLIB` directive located in the dynamic libraries that are
installed. The directive we expect looks like:
@rpath/libstd.dylib
Which means that if you want to find that dynamic library you should
look at the dylib's other `@rpath` directives. Typically our `@rpath`
directives look like `@loader_path/../lib` for the compiler as that's
where the installed libraries will be located. Currently, though,
rustbuild produces dylibs with the directive that looks like:
/Users/rustbuild/src/rust-buildbot/slave/nightly-dist-rustc-mac/build/build/x86_64-apple-darwin/stage1-std/x86_64-apple-darwin/release/deps/libstd-713ad88203512705.dylib
In other words, the build directory is encoded erroneously. The compiler
already [knows how] to change this directive, but it only passes that
argument when `-C rpath` is also passed. The rustbuild system, however,
explicitly [does not pass] this option explicitly and instead bakes its
own. This logic then also erroneously didn't pass `-Wl,-install_name`
like the compiler.
[knows how]: 4a008cccaa/src/librustc_trans/back/linker.rs (L210-L214)
[does not pass]: 4a008cccaa/src/bootstrap/bin/rustc.rs (L133-L158)
To fix this regression this patch introduces a new `-Z` flag, `-Z
osx-rpath-install-name` which basically just forces the compiler to take
the previous `-install_name` branch when creating a dynamic library.
Hopefully we can sort out a better rpath story in the future, but for
now this "hack" should suffice in getting our nightly builds back to the
same state as before.
Closes#38430
rustbuild: Add sccache support
This commit adds support for sccache, a ccache-like compiler which works on MSVC
and stores results into an S3 bucket. This also switches over all Travis and
AppVeyor automation to using sccache to ensure a shared and unified cache over
time which can be shared across builders.
The support for sccache manifests as a new `--enable-sccache` option which
instructs us to configure LLVM differently to use a 'sccache' binary instead of
a 'ccache' binary. All docker images for Travis builds are updated to download
Mozilla's tooltool builds of sccache onto various containers and systems.
Additionally a new `rust-lang-ci-sccache` bucket is configured to hold all of
our ccache goodies.
---
Note that this does not currently change Windows [due to previously written up issues](https://github.com/rust-lang/rust/issues/38119#issuecomment-266631585). Despite that, however, I was curious to get timings for the builds on Travis to see what ranges we're working with. As a result, this is a WIP PR I'm using to gauge build times and such.
This erroneously truncated files when the destination already existed and was an
existing hard link to the source. This in turn caused weird bugs!
Closes#37745
rustbuild: Package rust-mingw by default
This fixes the `make dist` step on MinGW to package the `rust-mingw` component
by default. This should hopefully be the last step in fixing nightlies.
rustbuild: Add cli option --keep-stage
This option is intended to be used like:
./x.py build --stage 1 --keep-stage 0
Which skips all stage 0 steps, so that stage 1 can be recompiled
directly (even if for example libcore has changes).
This is useful when working on `cfg(not(stage0))` parts of the
libraries or when re-running stage 1 tests in libraries in general.
Fixes#38326
This commit adds support for sccache, a ccache-like compiler which works on MSVC
and stores results into an S3 bucket. This also switches over all Travis and
AppVeyor automation to using sccache to ensure a shared and unified cache over
time which can be shared across builders.
The support for sccache manifests as a new `--enable-sccache` option which
instructs us to configure LLVM differently to use a 'sccache' binary instead of
a 'ccache' binary. All docker images for Travis builds are updated to download
Mozilla's tooltool builds of sccache onto various containers and systems.
Additionally a new `rust-lang-ci-sccache` bucket is configured to hold all of
our ccache goodies.
Document --test-args for rustbuild
There are three changes:
* Replace --filter with --test-args
* Delete `./x.py test src/test/run-pass/assert-*` example, which doesn't work
* As driveby, update Buildbot URLs to https
Fix#38275.
r? @alexcrichton
This commit skips a few more dist tragets during compilation which shouldn't be
necessary.
* First, when packaging std we only take action when the host target is the
build target. Otherwise we package the same artifacts a number of times, which
shouldn't be necessary.
* Next, we apply the same logic to the save-analysis build. This is actually
required for correctness as the build compiler is the only one which actually
has save analysis information. This should fix an error seen on nightlies.
This option is intended to be used like:
./x.py build --stage 1 --keep-stage 0
Which skips all stage 0 steps, so that stage 1 can be recompiled
directly (even if for example libcore has changes).
This is useful when working on `cfg(not(stage0))` parts of the
libraries, or when re-running stage 1 tests in libraries in general.
Git worktrees have this as a file and typically won't work inside docker
containers, but that's ok, so instead of just checking for existence check for a
directory to see if the git commands will succeed.
Make configure message re x.py not assume build dir == src dir
Fix#38251 but perhaps not BEST fix for it.
As driveby, fix copyright year in `Makefile.in`
Prevent Windows from displaying UI on errors.
Otherwise tests like run-pass/out-of-stack get wedged on Windows error reporting dialog (unless error reporting has been disabled, of course).
This commit implements the `distcheck` target for rustbuild which is only ever
run on our nightly bots. This essentially just creates a tarball, un-tars it,
and then runs a full build, validating that the release tarballs do indeed have
everything they need to build Rust.
rustbuild: Use src/rustc for assembled compilers
The `src/rustc` path is intended for assembling a compiler (e.g. the bare bones)
not actually compiling the whole compiler itself. This path was accidentally
getting hijacked to represent the whole compiler being compiled, so let's
redirect that elsewhere for that particular cargo project.
Closes#38039
This commit switches the default build system for Rust from the makefiles to
rustbuild. The rustbuild build system has been in development for almost a year
now and has become quite mature over time. This commit is an implementation of
the proposal on [internals] which slates deletion of the makefiles on
2016-01-02.
[internals]: https://internals.rust-lang.org/t/proposal-for-promoting-rustbuild-to-official-status/4368
This commit also updates various documentation in `README.md`,
`CONTRIBUTING.md`, `src/bootstrap/README.md`, and throughout the source code of
rustbuild itself.
Closes#37858
The `src/rustc` path is intended for assembling a compiler (e.g. the bare bones)
not actually compiling the whole compiler itself. This path was accidentally
getting hijacked to represent the whole compiler being compiled, so let's
redirect that elsewhere for that particular cargo project.
Closes#38039
Add rustbuild command `bench`
Add command bench to rustbuild, so that `./x.py bench <path>` can compile and run benchmarks.
`./x.py bench --stage 1 src/libcollections` and `./x.py bench --stage 1 src/libstd` should both compile well. Just `./x.py bench` runs all benchmarks for the libstd crates.
Fixes#37897
with this feature disabled, you can (Cargo) compile std with
"panic=abort"
rustbuild will build std with this feature enabled, to maintain the
status quo
fixes#37252
rustbuild: allow dynamically linking LLVM
The makefiles and `mklldeps.py` called `llvm-config --shared-mode` to
find out if LLVM defaulted to shared or static libraries, and just went
with that. But under rustbuild, `librustc_llvm/build.rs` was assuming
that LLVM should be static, and even forcing `--link-static` for 3.9+.
Now that build script also uses `--shared-mode` to learn the default,
which should work better for pre-3.9 configured for dynamic linking, as
it wasn't possible back then to choose differently via `llvm-config`.
Further, the configure script now has a new `--enable-llvm-link-shared`
option, which allows one to manually override `--link-shared` on 3.9+
instead of forcing static.
Update: There are now four static/shared scenarios that can happen
for the supported LLVM versions:
- 3.9+: By default use `llvm-config --link-static`
- 3.9+ and `--enable-llvm-link-shared`: Use `--link-shared` instead.
- 3.8: Use `llvm-config --shared-mode` and go with its answer.
- 3.7: Just assume static, maintaining the status quo.
rustbuild: Allow configuration of python interpreter
Add a configuration key to `config.toml`, read it from `./configure`, and add
auto-detection if none of those were specified.
Closes#35760
The makefiles and `mklldeps.py` called `llvm-config --shared-mode` to
find out if LLVM defaulted to shared or static libraries, and just went
with that. But under rustbuild, `librustc_llvm/build.rs` was assuming
that LLVM should be static, and even forcing `--link-static` for 3.9+.
Now that build script also uses `--shared-mode` to learn the default,
which should work better for pre-3.9 configured for dynamic linking, as
it wasn't possible back then to choose differently via `llvm-config`.
Further, the configure script now has a new `--enable-llvm-link-shared`
option, which allows one to manually override `--link-shared` on 3.9+
instead of forcing static.
Add llvm debuginfo configure option
CC @nnethercote @Mark-Simulacrum
We add a new configure option, `--enable-llvm-debuginfo`, to do exactly what you'd think.
Re: #31033Fixes#37738
If a rule is flagged with `default(true)` then the pseudo-rule `default:foo`
will include that. If a rule is also flagged with `.host(true)`, however, then
the rule shouldn't be included for targets that aren't in the host array. This
adds a filter to ensure we don't pull in host rules for targets by accident.
