I noticed these while reading through the build system
documentation. They're hardly worth fixing, but I'm also using this to
get my feet wet with the rustc contribution system.
Add an option to run rustbuild on low priority on Windows and Unix
This is a resurrection of #40776, combining their Windows setup with an additional setup on Unix to set the program group's *nice*ness to +10 (low-but-not-lowest priority, mirroring the priority in the Windows setup) when the `low_priority` option is on.
Introduce a new Installer object that hold a reference to all the
configured paths for installation
Signed-off-by: Marc-Antoine Perennou <Marc-Antoine@Perennou.com>
Unify tools building
Close#41601
Time saving for up to 10 minutes. Cargo is now only compiled once.
Downsides:
- Out of tree Cargo.lock maintenance
- Cargo.toml `[replace]` version maintenance
This is a resurrection of #40776, combining their Windows setup with an
additional setup on Unix to set the program group's niceness to +10
(low-but-not-lowest priority) when the `low_priority` option is on.
There's no shell interpreting the file paths under the new Rusty
rust-installer, so we don't need to use `sanitize_sh` for it. Plus,
the drive-letter transformation is actually harmful for the now-native
Windows rust-installer to understand those paths.
config.toml.example: point out that optimize = false won't speed up a full bootstrap
Originally I've learned about this by @eddyb pointing this out to me over IRC, and after having told someone today the same over IRC I've thought that this is a common mistake and should be prevented by a note in config.toml.example
r? @eddyb
rustbuild: Sanity-check cmake for sanitizers too
It's possible to build the sanitizers when using an external LLVM, but
we still need cmake for this. Extend the sanity check to look for cmake
whenever sanitizers are enabled too.
It's possible to build the sanitizers when using an external LLVM, but
we still need cmake for this. Extend the sanity check to look for cmake
whenever sanitizers are enabled too.
Add a distcheck for rust-src completeness
This is for the last commit of #41546. For some reason, @bors only saw the first two commits, and wouldn't approve the last even when explicitly directed so.
r? @alexcrichton
We've got a freshly minted beta compiler, let's update to use that on nightly!
This has a few other changes associated with it as well
* A bump to the rustc version number (to 1.19.0)
* Movement of the `cargo` and `rls` submodules to their "proper" location in
`src/tools/{cargo,rls}`. Now that Cargo workspaces support the `exclude`
option this can work.
* Updates of the `cargo` and `rls` submodules to their master branches.
* Tweak to the `src/stage0.txt` format to be more amenable for Cargo version
numbers. On the beta channel Cargo will bootstrap from a different version
than rustc (e.g. the version numbers are different), so we need different
configuration for this.
* Addition of `dev` as a readable key in the `src/stage0.txt` format. If present
then stage0 compilers are downloaded from `dev-static.rust-lang.org` instead
of `static.rust-lang.org`. This is added to accomodate our updated release
process with Travis and AppVeyor.
Currently our slowest test suite on android, run-pass, takes over 5 times longer
than the x86_64 component (~400 -> ~2200s). Typically QEMU emulation does indeed
add overhead, but not 5x for this kind of workload. One of the slowest parts of
the Android process is that *compilation* happens serially. Tests themselves
need to run single-threaded on the emulator (due to how the test harness works)
and this forces the compiles themselves to be single threaded.
Now Travis gives us more than one core per machine, so it'd be much better if we
could take advantage of them! The emulator itself is still fundamentally
single-threaded, but we should see a nice speedup by sending binaries for it to
run much more quickly.
It turns out that we've already got all the tools to do this in-tree. The
qemu-test-{server,client} that are in use for the ARM Linux testing are a
perfect match for the Android emulator. This commit migrates the custom adb
management code in compiletest/rustbuild to the same qemu-test-{server,client}
implementation that ARM Linux uses.
This allows us to lift the parallelism restriction on the compiletest test
suites, namely run-pass. Consequently although we'll still basically run the
tests themselves in single threaded mode we'll be able to compile all of them in
parallel, keeping the pipeline much more full and using more cores for the work
at hand. Additionally the architecture here should be a bit speedier as it
should have less overhead than adb which is a whole new process on both the host
and the emulator!
Locally on an 8 core machine I've seen the run-pass test suite speed up from
taking nearly an hour to only taking 6 minutes. I don't think we'll see quite a
drastic speedup on Travis but I'm hoping this change can place the Android tests
well below 2 hours instead of just above 2 hours.
Because the client/server here are now repurposed for more than just QEMU,
they've been renamed to `remote-test-{server,client}`.
Note that this PR does not currently modify how debuginfo tests are executed on
Android. While parallelizable it wouldn't be quite as easy, so that's left to
another day. Thankfully that test suite is much smaller than the run-pass test
suite.
As a final fix I discovered that the ARM and Android test suites were actually
running all library unit tests (e.g. stdtest, coretest, etc) twice. I've
corrected that to only run tests once which should also give a nice boost in
overall cycle time here.
Now that the final bug fixes have been merged into sccache we can start
leveraging sccache on the MSVC builders on AppVeyor instead of relying on the
ad-hoc caching strategy of trigger files and whatnot.
Shrink the rust-src component
Before this change, the installable rust-src component had essentially the same contents as the rustc-src dist tarball, just additionally wrapped in a rust-installer. As discussed on [internals], rust-src is only meant to support uses for the standard library, so it doesn't really need the rest of the compiler sources.
Now rust-src only contains libstd and its path dependencies, which roughly matches the set of crates that have rust-analysis data. The result is **significantly** smaller, from 36MB to 1.3MB compressed, and from 247MB to 8.5MB uncompressed.
[internals]: https://internals.rust-lang.org/t/minimizing-the-rust-src-component/5117
Add Hexagon support
This requires an updated LLVM with https://reviews.llvm.org/D31999 and https://reviews.llvm.org/D32000 to build libcore.
A basic hello world builds and runs successfully on the hexagon simulator. libcore is fine with LLVM fixes, but libstd requires a lot more work since there's a custom rtos running on most hexagon cores. Running Linux sounds possible though, so maybe getting linux + musl going would be easier.
Here's the target file I've been using for testing
```
{
"arch": "hexagon",
"llvm-target": "hexagon-unknown-elf",
"os": "none",
"target-endian": "little",
"target-pointer-width": "32",
"data-layout": "e-m:e-p:32:32:32-a:0-n16:32-i64:64:64-i32:32:32-i16:16:16-i1:8:8-f32:32:32-f64:64:64-v32:32:32-v64:64:64-v512:512:512-v1024:1024:1024-v2048:2048:2048",
"linker": "hexagon-clang",
"linker-flavor": "gcc",
"executables": true,
"cpu": "hexagonv60"
}
```
