This commit switches the rustbuild build system to compiling the
compiler twice for a normal bootstrap rather than the historical three
times.
Rust is a bootstrapped language which means that a previous version of
the compiler is used to build the next version of the compiler. Over
time, however, we change many parts of compiler artifacts such as the
metadata format, symbol names, etc. These changes make artifacts from
one compiler incompatible from another compiler. Consequently if a
compiler wants to be able to use some artifacts then it itself must have
compiled the artifacts.
Historically the rustc build system has achieved this by compiling the
compiler three times:
* An older compiler (stage0) is downloaded to kick off the chain.
* This compiler now compiles a new compiler (stage1)
* The stage1 compiler then compiles another compiler (stage2)
* Finally, the stage2 compiler needs libraries to link against, so it
compiles all the libraries again.
This entire process amounts in compiling the compiler three times.
Additionally, this process always guarantees that the Rust source tree
can compile itself because the stage2 compiler (created by a freshly
created compiler) would successfully compile itself again. This
property, ensuring Rust can compile itself, is quite important!
In general, though, this third compilation is not required for general
purpose development on the compiler. The third compiler (stage2) can
reuse the libraries that were created during the second compile. In
other words, the second compilation can produce both a compiler and the
libraries that compiler will use. These artifacts *must* be compatible
due to the way plugins work today anyway, and they were created by the
same source code so they *should* be compatible as well.
So given all that, this commit switches the default build process to
only compile the compiler three times, avoiding this third compilation
by copying artifacts from the previous one. Along the way a new entry in
the Travis matrix was also added to ensure that our full bootstrap can
succeed. This entry does not run tests, though, as it should not be
necessary.
To restore the old behavior of a full bootstrap (three compiles) you can
either pass:
./configure --enable-full-bootstrap
or if you're using config.toml:
[build]
full-bootstrap = true
Overall this will hopefully be an easy 33% win in build times of the
compiler. If we do 33% less work we should be 33% faster! This in turn
should affect cycle times and such on Travis and AppVeyor positively as
well as making it easier to work on the compiler itself.
This commit adds support for sccache, a ccache-like compiler which works on MSVC
and stores results into an S3 bucket. This also switches over all Travis and
AppVeyor automation to using sccache to ensure a shared and unified cache over
time which can be shared across builders.
The support for sccache manifests as a new `--enable-sccache` option which
instructs us to configure LLVM differently to use a 'sccache' binary instead of
a 'ccache' binary. All docker images for Travis builds are updated to download
Mozilla's tooltool builds of sccache onto various containers and systems.
Additionally a new `rust-lang-ci-sccache` bucket is configured to hold all of
our ccache goodies.
This commit switches the default build system for Rust from the makefiles to
rustbuild. The rustbuild build system has been in development for almost a year
now and has become quite mature over time. This commit is an implementation of
the proposal on [internals] which slates deletion of the makefiles on
2016-01-02.
[internals]: https://internals.rust-lang.org/t/proposal-for-promoting-rustbuild-to-official-status/4368
This commit also updates various documentation in `README.md`,
`CONTRIBUTING.md`, `src/bootstrap/README.md`, and throughout the source code of
rustbuild itself.
Closes#37858
rustbuild: allow dynamically linking LLVM
The makefiles and `mklldeps.py` called `llvm-config --shared-mode` to
find out if LLVM defaulted to shared or static libraries, and just went
with that. But under rustbuild, `librustc_llvm/build.rs` was assuming
that LLVM should be static, and even forcing `--link-static` for 3.9+.
Now that build script also uses `--shared-mode` to learn the default,
which should work better for pre-3.9 configured for dynamic linking, as
it wasn't possible back then to choose differently via `llvm-config`.
Further, the configure script now has a new `--enable-llvm-link-shared`
option, which allows one to manually override `--link-shared` on 3.9+
instead of forcing static.
Update: There are now four static/shared scenarios that can happen
for the supported LLVM versions:
- 3.9+: By default use `llvm-config --link-static`
- 3.9+ and `--enable-llvm-link-shared`: Use `--link-shared` instead.
- 3.8: Use `llvm-config --shared-mode` and go with its answer.
- 3.7: Just assume static, maintaining the status quo.
The makefiles and `mklldeps.py` called `llvm-config --shared-mode` to
find out if LLVM defaulted to shared or static libraries, and just went
with that. But under rustbuild, `librustc_llvm/build.rs` was assuming
that LLVM should be static, and even forcing `--link-static` for 3.9+.
Now that build script also uses `--shared-mode` to learn the default,
which should work better for pre-3.9 configured for dynamic linking, as
it wasn't possible back then to choose differently via `llvm-config`.
Further, the configure script now has a new `--enable-llvm-link-shared`
option, which allows one to manually override `--link-shared` on 3.9+
instead of forcing static.
Fix grammar verification
* Use make check-lexer to verify the grammar.
* Extend grammar/README
* Add make clean-grammar rule
* Add target check-build-lexer-verifier to make tidy, so it will build the verifier with every build and catch future errors
This is the continuation of #34994
r? @steveklabnik @jonathandturner @alexcrichton
* Use `make check-lexer` to verify the grammar.
* Extend grammar/README
* Add make clean-grammar rule
* Add target `check-build-lexer-verifier` to `make tidy`, so it will build the verifier with every build and catch future errors
* Search for antlr4 with configure and find
Add llvm debuginfo configure option
CC @nnethercote @Mark-Simulacrum
We add a new configure option, `--enable-llvm-debuginfo`, to do exactly what you'd think.
Re: #31033Fixes#37738
A few changes are included here:
* The `winapi` and `url` dependencies were dropped. The source code for these
projects is pretty weighty, and we're about to vendor them, so let's not
commit to that intake just yet. If necessary we can vendor them later but for
now it shouldn't be necessary.
* The `--frozen` flag is now always passed to Cargo, obviating the need for
tidy's `cargo_lock` check.
* Tidy was updated to not check the vendor directory
Closes#34687
armv7l is armv7 architecture and CFG_CPUTYPE should be armv7 in order to end up
with armv7-unknown-linux-gnueabihf.mk rather than
arm-unknown-linux-gnueabihf.mk
This commit enables by default passing the `-C debuginfo=1` argument to the
compiler for the stable, beta, and nightly release channels. A new configure
option was also added, `--enable-debuginfo-lines`, to enable this behavior in
developer builds as well.
Closes#36452
configure: Add options for separate musl roots
This allows using the `./configure` script to enable rustbuild to compile
multiple musl targets at once. We'll hopefully use this soon on our bots to
produce a bunch of targets.
This allows using the `./configure` script to enable rustbuild to compile
multiple musl targets at once. We'll hopefully use this soon on our bots to
produce a bunch of targets.
Allow setting --docdir
This will allow setting `--docdir` during configure, this is useful because not all linux distributions install documentation to `/usr/share/doc`. For example in Slackware documentation is installed to `/usr/doc/$PRGNAM-$VERSION` and `/usr/share/doc` is a symlink to `/usr/doc`.
To use this `./configure --docdir=/usr/doc/$PRGNAM-$VERSION` can be used.
Add s390x support
This adds support for building the Rust compiler and standard
library for s390x-linux, allowing a full cross-bootstrap sequence
to complete. This includes:
- Makefile/configure changes to allow native s390x builds
- Full Rust compiler support for the s390x C ABI
(only the non-vector ABI is supported at this point)
- Port of the standard library to s390x
- Update the liblibc submodule to a version including s390x support
- Testsuite fixes to allow clean "make check" on s390x
Caveats:
- Resets base cpu to "z10" to bring support in sync with the default
behaviour of other compilers on the platforms. (Usually, upstream
supports all older processors; a distribution build may then chose
to require a more recent base version.) (Also, using zEC12 causes
failures in the valgrind tests since valgrind doesn't fully support
this CPU yet.)
- z13 vector ABI is not yet supported. To ensure compatible code
generation, the -vector feature is passed to LLVM. Note that this
means that even when compiling for z13, no vector instructions
will be used. In the future, support for the vector ABI should be
added (this will require common code support for different ABIs
that need different data_layout strings on the same platform).
- Two test cases are (temporarily) ignored on s390x to allow passing
the test suite. The underlying issues still need to be fixed:
* debuginfo/simd.rs fails because of incorrect debug information.
This seems to be a LLVM bug (also seen with C code).
* run-pass/union/union-basic.rs simply seems to be incorrect for
all big-endian platforms.
Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
This adds support for building the Rust compiler and standard
library for s390x-linux, allowing a full cross-bootstrap sequence
to complete. This includes:
- Makefile/configure changes to allow native s390x builds
- Full Rust compiler support for the s390x C ABI
(only the non-vector ABI is supported at this point)
- Port of the standard library to s390x
- Update the liblibc submodule to a version including s390x support
- Testsuite fixes to allow clean "make check" on s390x
Caveats:
- Resets base cpu to "z10" to bring support in sync with the default
behaviour of other compilers on the platforms. (Usually, upstream
supports all older processors; a distribution build may then chose
to require a more recent base version.) (Also, using zEC12 causes
failures in the valgrind tests since valgrind doesn't fully support
this CPU yet.)
- z13 vector ABI is not yet supported. To ensure compatible code
generation, the -vector feature is passed to LLVM. Note that this
means that even when compiling for z13, no vector instructions
will be used. In the future, support for the vector ABI should be
added (this will require common code support for different ABIs
that need different data_layout strings on the same platform).
- Two test cases are (temporarily) ignored on s390x to allow passing
the test suite. The underlying issues still need to be fixed:
* debuginfo/simd.rs fails because of incorrect debug information.
This seems to be a LLVM bug (also seen with C code).
* run-pass/union/union-basic.rs simply seems to be incorrect for
all big-endian platforms.
Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
Use libraries from local-rust-root directory in configure when using …
…--enable-local-rebuild
When using --enable-local-rebuild configure options, the configure
script will test rustc version. But when running it, it will not use the
libraries in the local-rust-root directory.
So use `LD_LIBRARY_PATH` environment variable to correct it.
Under OpenBSD, we use `--enable-local-rebuild` for rebuilding rustc-1.11.0 using rustc-1.11.0. But as it is in use in -current (the developpment branch of OpenBSD), system libraries could easily have ABI/API changes. The (unofficial) bootstrap (rustc-1.11.0 used for building rustc-1.11.0) include several system libraries to permit it to run. The build system use libraries in stage0/lib directory. But the configure script doesn't.
Due to my special use of `--enable-local-rebuild`, I dunno if this PR is suitable for inclusion. But it corrects a difference between build and configure use of the binary.
r? @alexcrichton
Remove --{enable|disable}-orbit from configure.
Fixes#35956.
r? @eddyb
There are only two buildbots left, though they are both failing. Is there something to be done there other than wait?
When using --enable-local-rebuild configure options, the configure
script will test rustc version. But when running it, it will not use the
libraries in the local-rust-root directory.
So use `LD_LIBRARY_PATH` environment variable to correct it.
