The compiler-rt build system has been a never ending cause of pain for Rust
unfortunately:
* The build system is very difficult to invoke and configure to only build
compiler-rt, especially across platforms.
* The standard build system doesn't actually do what we want, not working for
some of our platforms and requiring a significant number of patches on our end
which are difficult to apply when updating compiler-rt.
* Compiling compiler-rt requires LLVM to be compiled, which... is a big
dependency! This also means that over time compiler-rt is not guaranteed to
build against older versions of LLVM (or newer versions), and we often want to
work with multiple versions of LLVM simultaneously.
The makefiles and rustbuild already know how to compile C code, the code here is
far from the *only* C code we're compiling. This patch jettisons all logic to
work with compiler-rt's build system and just goes straight to the source. We
just list all files manually (copied from compiler-rt's
lib/builtins/CMakeLists.txt) and compile them into an archive.
It's likely that this means we'll fail to pick up new files when we upgrade
compiler-rt, but that seems like a much less significant cost to pay than what
we're currently paying.
cc #34400, first steps towards that
llvm, rt: build using the Ninja generator if available
The Ninja generator generally builds much faster than make. It may also
be used on Windows to have a vast speed improvement over the Visual
Studio generators.
Currently hidden behind an `--enable-ninja` flag because it does not
obey the top-level `-j` or `-l` flags given to `make`.
If local-rust is the same as the current version, then force a local-rebuild
In Debian, we would like the option to build/rebuild the current release from
*either* the current or previous stable release. So we use enable-local-rust
instead of enable-local-rebuild, and read the bootstrap key dynamically from
whatever is installed locally.
In general, it does not make much sense to allow enable-local-rust without also
setting the bootstrap key, since the build would fail otherwise.
(The way I detect "the bootstrap key of [the local] rustc installation" is a bit hacky, suggestions welcome.)
The Ninja generator generally builds much faster than make. It may also
be used on Windows to have a vast speed improvement over the Visual
Studio generators.
Currently hidden behind an `--enable-ninja` flag because it does not
obey the top-level `-j` or `-l` flags given to `make`.
rustbuild: Remove the `build` directory
The organization in rustbuild was a little odd at the moment where the `lib.rs`
was quite small but the binary `main.rs` was much larger. Unfortunately as well
there was a `build/` directory with the implementation of the build system, but
this directory was ignored by GitHub on the file-search prompt which was a
little annoying.
This commit reorganizes rustbuild slightly where all the library files (the
build system) is located directly inside of `src/bootstrap` and all the binaries
now live in `src/bootstrap/bin` (they're small). Hopefully this should allow
GitHub to index and allow navigating all the files while maintaining a
relatively similar layout to the other libraries in `src/`.
The organization in rustbuild was a little odd at the moment where the `lib.rs`
was quite small but the binary `main.rs` was much larger. Unfortunately as well
there was a `build/` directory with the implementation of the build system, but
this directory was ignored by GitHub on the file-search prompt which was a
little annoying.
This commit reorganizes rustbuild slightly where all the library files (the
build system) is located directly inside of `src/bootstrap` and all the binaries
now live in `src/bootstrap/bin` (they're small). Hopefully this should allow
GitHub to index and allow navigating all the files while maintaining a
relatively similar layout to the other libraries in `src/`.
Build: Shows total time taken to build the compiler
Fixes#34600
Prints the total time taken to build rustc by executing `src/bootstrap/bootstrap.py`; also includes time taken to download `stage0` compiler and deps.
r? @alexcrichton
Support more python 2.7 versions in bootstrap.py
It seems python broke compatability between 2.7.9 and 2.7.12 as on the former a WindowsError was raised while on the latter a subprocess.CalledProcessError was raised while testing for the existence of uname.
As a WindowsError being thrown obviously indicates we're running on windows, this should probably be accepted too.
rustbuild: Implement testing for Android
This commit enhances the rustbuild support for testing Android to the same level
of parity as the makefiles. This involved:
* A new step to copy the standard library and other shared objects to the
emulator. This is injected as a dependency of all test suites for Android.
* Appropriate arguments are now passed through to compiletest to ensure that it
can run tests.
* When testing the standard library the test executables are probed for and
shipped to the emulator to run for each test.
* Fixing compilation of compiler-rt a bit
All support added here is modeled after what's found in the makefiles, just
translating one strategy to another. As an added bonus this commit adds support
for the "check" step to automatically run tests for all targets, and the
"check-target" step now runs all tests for a particular target, automatically
filtering the tests if the target is detected as a cross-compile.
Note that we don't (and probably won't) have a bot which is actually going to
exercise any of this just yet, but all tests have passed locally for me at
least.
mk: Don't consider LLVM done until it's done
Currently if an LLVM build is interrupted *after* it creates the llvm-config
binary but before it's done it puts us in an inconsistent state where we think
LLVM is compiled but it's not actually. This tweaks our logic to only consider
LLVM done building once it's actually done building.
This should hopefully alleviate problems on the bots where if we interrupt at
the wrong time it doesn't corrupt the build directory.
Right now we generate error index information into this directory, but it's
never cleaned out. This means that if a build *bounces* because of something in
this directory it'll continue to cause all future builds to fail because the
relevant files are never removed.
Currently if an LLVM build is interrupted *after* it creates the llvm-config
binary but before it's done it puts us in an inconsistent state where we think
LLVM is compiled but it's not actually. This tweaks our logic to only consider
LLVM done building once it's actually done building.
This should hopefully alleviate problems on the bots where if we interrupt at
the wrong time it doesn't corrupt the build directory.
This commit enhances the rustbuild support for testing Android to the same level
of parity as the makefiles. This involved:
* A new step to copy the standard library and other shared objects to the
emulator. This is injected as a dependency of all test suites for Android.
* Appropriate arguments are now passed through to compiletest to ensure that it
can run tests.
* When testing the standard library the test executables are probed for and
shipped to the emulator to run for each test.
* Fixing compilation of compiler-rt a bit
All support added here is modeled after what's found in the makefiles, just
translating one strategy to another. As an added bonus this commit adds support
for the "check" step to automatically run tests for all targets, and the
"check-target" step now runs all tests for a particular target, automatically
filtering the tests if the target is detected as a cross-compile.
Note that we don't (and probably won't) have a bot which is actually going to
exercise any of this just yet, but all tests have passed locally for me at
least.
rustbuild: Clean more on `make clean`
Be sure to not use the old build cache for the bootstrap build system nor the
old caches of the compiler/cargo extractions (in case something went wrong).
Closes#33986
Be sure to not use the old build cache for the bootstrap build system nor the
old caches of the compiler/cargo extractions (in case something went wrong).
Closes#33986
Currently rustbuild isn't detecting the `-O` flag for tests via the
`--disable-optimize-tests` or not command line flag to `./configure`, and this
commit patches up the support to pass `-O` by default.
After a comparison with the current set of tests run it was discovered that
rustbuild accidentally wasn't running a few test suites. This commit adds back a
few more test suites:
* rfail-full
* pretty-rpass
* pretty-rpass-full
* pretty-rpass-valgrind
* pretty-rfail
* pretty-rfail-full
* librustc_bitflags unit tests
This commit adds support to rustbuild to run crate unit tests (those defined by
`#[test]`) as well as documentation tests. All tests are powered by `cargo test`
under the hood.
Each step requires the `libtest` library is built for that corresponding stage.
Ideally the `test` crate would be a dev-dependency, but for now it's just easier
to ensure that we sequence everything in the right order.
Currently no filtering is implemented, so there's not actually a method of
testing *only* libstd or *only* libcore, but rather entire swaths of crates are
tested all at once.
A few points of note here are:
* The `coretest` and `collectionstest` crates are just listed as `[[test]]`
entires for `cargo test` to naturally pick up. This mean that `cargo test -p
core` actually runs all the tests for libcore.
* Libraries that aren't tested all mention `test = false` in their `Cargo.toml`
* Crates aren't currently allowed to have dev-dependencies due to
rust-lang/cargo#860, but we can likely alleviate this restriction once
workspaces are implemented.
cc #31590
This commit is an implementation of [RFC 1513] which allows applications to
alter the behavior of panics at compile time. A new compiler flag, `-C panic`,
is added and accepts the values `unwind` or `panic`, with the default being
`unwind`. This model affects how code is generated for the local crate, skipping
generation of landing pads with `-C panic=abort`.
[RFC 1513]: https://github.com/rust-lang/rfcs/blob/master/text/1513-less-unwinding.md
Panic implementations are then provided by crates tagged with
`#![panic_runtime]` and lazily required by crates with
`#![needs_panic_runtime]`. The panic strategy (`-C panic` value) of the panic
runtime must match the final product, and if the panic strategy is not `abort`
then the entire DAG must have the same panic strategy.
With the `-C panic=abort` strategy, users can expect a stable method to disable
generation of landing pads, improving optimization in niche scenarios,
decreasing compile time, and decreasing output binary size. With the `-C
panic=unwind` strategy users can expect the existing ability to isolate failure
in Rust code from the outside world.
Organizationally, this commit dismantles the `sys_common::unwind` module in
favor of some bits moving part of it to `libpanic_unwind` and the rest into the
`panicking` module in libstd. The custom panic runtime support is pretty similar
to the custom allocator support with the only major difference being how the
panic runtime is injected (takes the `-C panic` flag into account).
rustbuild: Document many more parts of the build
This commit expands the bootstrap build system's `README.md` as well as ensuring
that all API documentation is present and up-to-date. Additionally a new
`config.toml.example` file is checked in with commented out versions of all
possible configuration values.
Get a file during bootstrap to a temp location first.
When downloading a file in the bootstrap phase - get it to a temp
location first. Verify it there and only if downloaded properly move it
to the `cache` directory.
This should prevent `make` being stuck if the download was interrupted
or otherwise corrupted, as per discussion in #32834
The temporary files are deleted in case of an exception.
I was looking for some unit/integration tests around this and couldn't find any - presumably because this is being tested by just Travis launching it ? Let me know if it would be good to try to write tests around this. Thanks !
Add armv7-linux-androideabi target
This PR adds `armv7-linux-androideabi` target that matches `armeabi-v7a` Android ABI, ~~downscales `arm-linux-androideabi` target to match `armeabi` Android ABI~~ (TBD later if needed).
This should allow us to get the best performance from every [Android ABI level](http://developer.android.com/ndk/guides/abis.html).
Currently existing target `arm-linux-androideabi` started gaining features out of the supported range of [android `armeabi`](http://developer.android.com/ndk/guides/abis.html). While android compiler does not use a different target for later supported `armv7` architecture, it has distinct ABI name `armeabi-v7a`. We decided to add rust target `armv7-linux-androideabi` to match it.
Note that `NEON`, `VFPv3-D32`, and `ThumbEE` instruction sets are not added, because not all android devices are guaranteed to support all or some of these, and [their availability should be checked at runtime](http://developer.android.com/ndk/guides/abis.html#v7a).
~~This reduces performance of existing `arm-linux-androideabi` and may make it _much_ slower (we are talking more than order of magnitude in some random ad-hoc fp benchmark that I did).~~
Part of #33278.
