1137 Commits

Author SHA1 Message Date
Marc-Antoine Perennou
a8c6ba9c6e rustbuild: only build cargo for host
Signed-off-by: Marc-Antoine Perennou <Marc-Antoine@Perennou.com>
2017-04-28 10:48:49 +02:00
Alex Crichton
7bc2cbf5db travis: Parallelize tests on Android
Currently our slowest test suite on android, run-pass, takes over 5 times longer
than the x86_64 component (~400 -> ~2200s). Typically QEMU emulation does indeed
add overhead, but not 5x for this kind of workload. One of the slowest parts of
the Android process is that *compilation* happens serially. Tests themselves
need to run single-threaded on the emulator (due to how the test harness works)
and this forces the compiles themselves to be single threaded.

Now Travis gives us more than one core per machine, so it'd be much better if we
could take advantage of them! The emulator itself is still fundamentally
single-threaded, but we should see a nice speedup by sending binaries for it to
run much more quickly.

It turns out that we've already got all the tools to do this in-tree. The
qemu-test-{server,client} that are in use for the ARM Linux testing are a
perfect match for the Android emulator. This commit migrates the custom adb
management code in compiletest/rustbuild to the same qemu-test-{server,client}
implementation that ARM Linux uses.

This allows us to lift the parallelism restriction on the compiletest test
suites, namely run-pass. Consequently although we'll still basically run the
tests themselves in single threaded mode we'll be able to compile all of them in
parallel, keeping the pipeline much more full and using more cores for the work
at hand. Additionally the architecture here should be a bit speedier as it
should have less overhead than adb which is a whole new process on both the host
and the emulator!

Locally on an 8 core machine I've seen the run-pass test suite speed up from
taking nearly an hour to only taking 6 minutes. I don't think we'll see quite a
drastic speedup on Travis but I'm hoping this change can place the Android tests
well below 2 hours instead of just above 2 hours.

Because the client/server here are now repurposed for more than just QEMU,
they've been renamed to `remote-test-{server,client}`.

Note that this PR does not currently modify how debuginfo tests are executed on
Android. While parallelizable it wouldn't be quite as easy, so that's left to
another day. Thankfully that test suite is much smaller than the run-pass test
suite.

As a final fix I discovered that the ARM and Android test suites were actually
running all library unit tests (e.g. stdtest, coretest, etc) twice. I've
corrected that to only run tests once which should also give a nice boost in
overall cycle time here.
2017-04-27 20:20:13 -07:00
Alex Crichton
2e72bcb934 appveyor: Use Ninja/sccache on MSVC
Now that the final bug fixes have been merged into sccache we can start
leveraging sccache on the MSVC builders on AppVeyor instead of relying on the
ad-hoc caching strategy of trigger files and whatnot.
2017-04-27 07:19:34 -07:00
Marc-Antoine Perennou
7b1fb89641 rustbuild: install rustc after cargo and rls
This way its files take precedence (e.g. README.md and stuff)

Signed-off-by: Marc-Antoine Perennou <Marc-Antoine@Perennou.com>
2017-04-27 15:05:29 +02:00
Marc-Antoine Perennou
03c5797552 rustbuild: install cargo and rls when extended build is enabled
Signed-off-by: Marc-Antoine Perennou <Marc-Antoine@Perennou.com>
2017-04-27 12:05:34 +02:00
Marc-Antoine Perennou
a2bab1ddf5 rustbuild: add rls_package_vers
Signed-off-by: Marc-Antoine Perennou <Marc-Antoine@Perennou.com>
2017-04-27 11:52:58 +02:00
Marc-Antoine Perennou
c4c3b5a443 rustbuild: pass version number as param in install phase
Signed-off-by: Marc-Antoine Perennou <Marc-Antoine@Perennou.com>
2017-04-27 11:52:58 +02:00
Marc-Antoine Perennou
f5929037cc rustbuild: build cargo and rls as part of extended build
Build them directly in the `./x.py build` phase, don't wait for
`./x.py dist`

Signed-off-by: Marc-Antoine Perennou <Marc-Antoine@Perennou.com>
2017-04-27 11:52:57 +02:00
bors
6e0c5af9de Auto merge of - arielb1:rollup, r=arielb1
Rollup of 9 pull requests

- Successful merges: , , , , , , , , 
- Failed merges:
2017-04-27 00:04:03 +00:00
Ariel Ben-Yehuda
60837b1e8d Rollup merge of - jessicah:updated-num-cpus-dep, r=alexcrichton
Update num_cpus dependency to 1.x (1.4.0)
2017-04-26 22:11:10 +00:00
Ariel Ben-Yehuda
95c2f06162 Rollup merge of - cuviper:reduced-rust-src, r=alexcrichton
Shrink the rust-src component

Before this change, the installable rust-src component had essentially the same contents as the rustc-src dist tarball, just additionally wrapped in a rust-installer.  As discussed on [internals], rust-src is only meant to support uses for the standard library, so it doesn't really need the rest of the compiler sources.

Now rust-src only contains libstd and its path dependencies, which roughly matches the set of crates that have rust-analysis data.  The result is **significantly** smaller, from 36MB to 1.3MB compressed, and from 247MB to 8.5MB uncompressed.

[internals]: https://internals.rust-lang.org/t/minimizing-the-rust-src-component/5117
2017-04-26 22:11:08 +00:00
Ariel Ben-Yehuda
c6ff8326ec Rollup merge of - michaelwu:basic-hexagon, r=alexcrichton
Add Hexagon support

This requires an updated LLVM with https://reviews.llvm.org/D31999 and https://reviews.llvm.org/D32000 to build libcore.

A basic hello world builds and runs successfully on the hexagon simulator. libcore is fine with LLVM fixes, but libstd requires a lot more work since there's a custom rtos running on most hexagon cores. Running Linux sounds possible though, so maybe getting linux + musl going would be easier.

Here's the target file I've been using for testing
```
{
    "arch": "hexagon",
    "llvm-target": "hexagon-unknown-elf",
    "os": "none",
    "target-endian": "little",
    "target-pointer-width": "32",

    "data-layout": "e-m:e-p:32:32:32-a:0-n16:32-i64:64:64-i32:32:32-i16:16:16-i1:8:8-f32:32:32-f64:64:64-v32:32:32-v64:64:64-v512:512:512-v1024:1024:1024-v2048:2048:2048",
    "linker": "hexagon-clang",
    "linker-flavor": "gcc",
    "executables": true,
    "cpu": "hexagonv60"
}
```
2017-04-26 22:11:07 +00:00
Ariel Ben-Yehuda
639000ea2e Rollup merge of - jessicah:haiku-support, r=alexcrichton
Haiku: fix initial platform support
2017-04-26 22:11:03 +00:00
Ariel Ben-Yehuda
a3dde0b457 Rollup merge of - malbarbo:android-bootstrap, r=alexcrichton
Add bootstrap support for android
2017-04-26 22:11:02 +00:00
Aidan Hobson Sayers
367e90775b Make sure openssl compiles with only one core
Fixes 
2017-04-26 21:21:27 +01:00
Josh Stone
c5cd4cb51e Add a distcheck for rust-src completeness 2017-04-26 12:37:12 -07:00
Michael Wu
22eb3c69b9 Enable building the LLVM Hexagon target 2017-04-26 12:00:16 -04:00
bors
0ee56f686d Auto merge of - kennytm:macos-sanitizers, r=alexcrichton
Support AddressSanitizer and ThreadSanitizer on x86_64-apple-darwin

[ASan](https://clang.llvm.org/docs/AddressSanitizer.html#supported-platforms) and [TSan](https://clang.llvm.org/docs/ThreadSanitizer.html#supported-platforms) are supported on macOS, and this commit enables their support.

The sanitizers are always built as `*.dylib` on Apple platforms, so they cannot be statically linked into the corresponding `rustc_?san.rlib`. The dylibs are directly copied to `lib/rustlib/x86_64-apple-darwin/lib/` instead.

Note, although Xcode also ships with their own copies of ASan/TSan dylibs, we cannot use them due to version mismatch.

----

~~There is a caveat: the sanitizer libraries are linked as `@rpath/` (due to https://reviews.llvm.org/D6018), so the user needs to additionally pass `-C rpath`:~~

**Edit:** Passing rpath is now automatic.
2017-04-26 12:34:45 +00:00
Jessica Hamilton
85c2ff1c46 Update num_cpus dependency to 1.x (1.4.0) 2017-04-26 15:25:03 +12:00
Corey Farwell
27fbe8cbca Rollup merge of - AndrewGaspar:rust-windbg, r=brson
Adds rust-windbg.cmd script

Adds rust-gdb/rust-lldb equivalent for windbg that loads the Rust .natvis files on start.

This change modifies the bootstrap code to add rust-windbg to bin and the .natvis files to lib/rustlib/etc.

Example usage from cmd or PowerShell:
```
rust-windbg -c "bu rs_f442289d74765418!rs::main;g" target\debug\rs.exe
```
2017-04-25 23:05:54 -04:00
Josh Stone
b9bdb1766d Reduce the contents of the rust-src component 2017-04-25 16:13:46 -07:00
Josh Stone
7248f67f1b Build the rustc-src tarball before the rust-src component 2017-04-25 16:13:45 -07:00
bors
2b4c911581 Auto merge of - alexcrichton:run-cargot-ests, r=aturon
Run tests for the cargo submodule in tree

Previously the `cargotest` suite would run some arbitrary revision of Cargo's
test suite, but now that we're bundling it in tree we should be running the
Cargo submodule's test suite instead.
2017-04-25 15:51:40 +00:00
kennytm
00dff0aa59
Support AddressSanitizer and ThreadSanitizer on x86_64-apple-darwin.
ASan and TSan are supported on macOS, and this commit enables their
support.

The sanitizers are always built as *.dylib on Apple platforms, so they
cannot be statically linked into the corresponding `rustc_?san.rlib`. The
dylibs are directly copied to `lib/rustlib/x86_64-apple-darwin/lib/`
instead.

Note, although Xcode also ships with their own copies of ASan/TSan dylibs,
we cannot use them due to version mismatch.

There is a caveat: the sanitizer libraries are linked as @rpath, so the
user needs to additionally pass `-C rpath`:

    rustc -Z sanitizer=address -C rpath file.rs
                               ^~~~~~~~

Otherwise there will be a runtime error:

    dyld: Library not loaded: @rpath/libclang_rt.asan_osx_dynamic.dylib
      Referenced from: /path/to/executable
      Reason: image not found
    Abort trap: 6

The next commit includes a temporary change in compiler to force the linker
to emit a usable @rpath.
2017-04-25 10:31:01 +08:00
kennytm
93d57d64b8
Pass --format-version 1 to cargo metadata.
Suppress warning introduced by .
2017-04-25 10:31:00 +08:00
Andrew Gaspar
70e673952e Adds rust-windbg.cmd script 2017-04-24 16:26:00 -07:00
bors
0777c757a6 Auto merge of - TimNN:llvm40, r=alexcrichton
LLVM 4.0 Upgrade

Since nobody has done this yet, I decided to get things started:

**Todo:**

* [x] push the relevant commits to `rust-lang/llvm` and `rust-lang/compiler-rt`
* [x] cleanup `.gitmodules`
* [x] Verify if there are any other commits from `rust-lang/llvm` which need backporting
* [x] Investigate / fix debuginfo ("`<optimized out>`") failures
* [x] Use correct emscripten version in docker image

---

Closes .

---

**Test results:**

Everything is green 🎉
2017-04-24 22:18:16 +00:00
Alex Crichton
009f45f8f1 Run tests for the cargo submodule in tree
Previously the `cargotest` suite would run some arbitrary revision of Cargo's
test suite, but now that we're bundling it in tree we should be running the
Cargo submodule's test suite instead.
2017-04-24 08:08:40 -07:00
Jessica Hamilton
0d63f13378 Haiku: add missing cases of using LIBRARY_PATH 2017-04-24 14:29:39 +00:00
Tim Neumann
f3dda17469 FIN: Compile LLVM with -fno-omit-frame-pointer on 32bit MinGW builds
to work around an apparently bad optimization.
2017-04-23 22:00:03 +02:00
bors
1785bca513 Auto merge of - frewsxcv:rollup, r=frewsxcv
Rollup of 4 pull requests

- Successful merges: , , , 
- Failed merges:
2017-04-21 16:36:41 +00:00
Eduard-Mihai Burtescu
89bd3f39ca Update #[no_core] users with the "freeze" lang item. 2017-04-21 15:48:35 +03:00
Corey Farwell
d5bbeb1a0a Rollup merge of - malbarbo:android-x86_64, r=alexcrichton
Add x86_64-linux-android target
2017-04-21 08:40:35 -04:00
Marco A L Barbosa
51cc0e38e3 Add x86_64-linux-android target 2017-04-20 16:29:59 -03:00
Corey Farwell
80fa79821a Rollup merge of - durka:patch-40, r=petrochenkov
remove disclaimer from bootstrap/README.md
2017-04-19 21:55:40 -04:00
bors
452bf0852e Auto merge of - alexcrichton:tool-no-debug, r=aturon
Disable debuginfo when compiling tools

Currently the Cargo binary has jumped from 14M to 34M on the beta channel, which
appears to be due to the fact that we're compiling tools with debug information
inside them. This additionally means that the `rls` binary is 62M right now!

This wasn't an intentional change, so be sure to disable debuginfo when
compiling tools as it's just intended for the standard library and compile for
now.
2017-04-19 11:07:08 +00:00
bors
5f22d46e4b Auto merge of - Nashenas88:master, r=alexcrichton
Use the existing path when removing the prefix fails

This allows the use of out-of-tree paths to be specified. I found this while trying to build with a modified version of `rls-data`, which is currently pointing to a version on crates.io.

cc @alexcrichton

Also, it wasn't clear if I needed to add a test for this (or how). I didn't see any tests that took paths into consideration.
2017-04-19 05:54:23 +00:00
Alex Burka
d6b8d9f75d remove disclaimer from bootstrap/README.md 2017-04-19 01:45:34 -04:00
Marco A L Barbosa
a42c0257c7 Add bootstrap support for android 2017-04-18 13:50:29 -03:00
Richo Healey
1dbc72b28e bootstrap: Don't workaround uname -m on Darwin
This no longer manifests on any versions of OSX that I could find.
2017-04-17 19:30:18 -07:00
Alex Crichton
e3ad1b58f5 Disable debuginfo when compiling tools
Currently the Cargo binary has jumped from 14M to 34M on the beta channel, which
appears to be due to the fact that we're compiling tools with debug information
inside them. This additionally means that the `rls` binary is 62M right now!

This wasn't an intentional change, so be sure to disable debuginfo when
compiling tools as it's just intended for the standard library and compile for
now.
2017-04-17 18:16:53 -07:00
Eduard-Mihai Burtescu
c5979945da [rustbuild] Side-step HashMap iteration to preserve command-line step order. 2017-04-15 03:33:12 +03:00
Alex Crichton
2a33559207 rustbuild: Fix recompilation of stage0 tools dir
This commit knocks out a longstanding FIXME in rustbuild which should correctly
recompile stage0 compiletest and such whenever libstd itself changes. The
solution implemented here was to implement a notion of "order only" dependencies
and then add a new dependency stage for clearing out the tools dir, using
order-only deps to ensure that it happens correctly.

The dependency drawing for tools is a bit wonky now but I think this'll get the
job done.

Closes 
2017-04-13 09:47:00 -07:00
Alex Crichton
cdedecb7ba travis: Enable rust-analysis package for more targets
This commit enables the `rust-analysis` package to be produced for all targets
that are part of the `dist-*` suite of docker images on Travis. Currently
these packages are showing up with `available = false` in the
`channel-rust-nightly.toml` manifest where we'd prefer to have them show up for
all targets.

Unfortunately rustup isn't handling the `available = false` section well right
now, so this should also inadvertently fix the nightly regression.
2017-04-12 20:48:18 -07:00
bors
8c6e2ff452 Auto merge of - nrc:rls-submod, r=alexcrichton
Add the RLS as a submodule and build a package out of it

r? @brson (and cc @alexcrichton) Please review closely, I am not at all convinced I've done the right things here. I did run `x.py dist` and it makes an rls package which looks right to my eyes, but I haven't tested on non-linux platforms nor am I really sure what it should look like.

This does not attempt to run tests for the RLS yet.
2017-04-12 05:26:10 +00:00
Alex Crichton
13d008d1e8 Touch up rls integration
* Use the right version when building combined installer
* Update dependencies of rls as it depends on rustc and plugins
* Fix build-manifest and the versions it uses for the rls
2017-04-11 10:47:53 -07:00
Aidan Hobson Sayers
f297767b2c Make sccache a bit quieter 2017-04-11 00:22:26 +01:00
Nick Cameron
5766d526a2 Remove --enable-save-analysis configure flag 2017-04-10 08:46:20 +12:00
Nick Cameron
223b280f31 Reviewer changes 2017-04-10 08:30:34 +12:00
Nick Cameron
c55325e0f7 Build an RLS package as part of the dist target 2017-04-10 08:30:34 +12:00