Commit Graph

671 Commits

Author SHA1 Message Date
bors
bad58f2791 Auto merge of - alexcrichton:aarch64-ci, r=aidanhs
Add a disabled builder for aarch64 emulated tests

This commit adds a disabled builder which will run all tests for the standard
library for aarch64 in a QEMU instance. Once we get enough capacity to run this
on Travis this can be used to boost our platform coverage of AArch64
2017-07-26 03:53:46 +00:00
Alex Crichton
8ef3f69ffb Add a disabled builder for aarch64 emulated tests
This commit adds a disabled builder which will run all tests for the standard
library for aarch64 in a QEMU instance. Once we get enough capacity to run this
on Travis this can be used to boost our platform coverage of AArch64
2017-07-24 16:24:52 -07:00
Ximin Luo
ffae5deaa7 configure: allow distros to disable debuginfo-only-std 2017-07-17 22:29:09 +02:00
Thomas Lively
447297ce59 Make wasm32 buildbot test LLVM backend
This adds the experimental targets option to configure so it can be used
by the builders and changes the wasm32 Dockerfile accordingly. Instead
of using LLVM from the emsdk, the builder's emscripten tools now uses
the Rust in-tree LLVM, since this is the one built with wasm support.
2017-06-22 18:34:56 -07:00
Marco Castelluccio
ecba8d6a23 Merge branch 'profiling' of github.com:whitequark/rust into profiling 2017-06-04 15:54:39 +01:00
Marc-Antoine Perennou
0a8b8febb0 configure: allow configuring bindir
Signed-off-by: Marc-Antoine Perennou <Marc-Antoine@Perennou.com>
2017-05-16 08:21:27 +02:00
Ralph Giles
19cab639fb Remove obsolete --disable-elf-tls configure switch.
Support for disabling ELF-style thread local storage in
the standard library at configure time was removed in
pulls  and , in favour of a member in
the TargetOptions database. The new method respects
MACOSX_DEPLOYMENT_TARGET on macOS, addressing the
original use case for this configure option.

However, those commits left the configure option itself
in place. It's no longer referenced anywhere and can
be removed.
2017-05-02 11:08:10 -07:00
whitequark
42754ce710 Add profiling support, through the rustc -Z profile flag.
When -Z profile is passed, the GCDAProfiling LLVM pass is added
to the pipeline, which uses debug information to instrument the IR.
After compiling with -Z profile, the $(OUT_DIR)/$(CRATE_NAME).gcno
file is created, containing initial profiling information.
After running the program built, the $(OUT_DIR)/$(CRATE_NAME).gcda
file is created, containing branch counters.

The created *.gcno and *.gcda files can be processed using
the "llvm-cov gcov" and "lcov" tools. The profiling data LLVM
generates does not faithfully follow the GCC's format for *.gcno
and *.gcda files, and so it will probably not work with other tools
(such as gcov itself) that consume these files.
2017-05-01 09:16:20 +00:00
Marco A L Barbosa
51cc0e38e3 Add x86_64-linux-android target 2017-04-20 16:29:59 -03:00
Nick Cameron
5766d526a2 Remove --enable-save-analysis configure flag 2017-04-10 08:46:20 +12:00
bors
0aeb9c1297 Auto merge of - ishitatsuyuki:easy-dist-analysis, r=alexcrichton
rustbuild: Make save-analysis an option

This makes save-analysis an option independent from the release channel.

The CI build scripts have been modified to enable the flag.

*Merge with caution.* I haven't tested this, and this can cause nightly breakage.
2017-03-15 13:50:13 +00:00
Vadim Petrochenkov
362aa9a317 Default llvm.clean-rebuild to false 2017-03-12 14:02:49 +03:00
Alex Crichton
f8ca805422 configure: Remove --build detection
This commit removes detection of CFG_OSTYPE and CFG_CPUTYPE from the configure
script, which means that the default value of `--build` is no longer present in
the configure script. All this logic is now available in rustbuild itself, so
there's no need to duplicate it.
2017-03-11 08:41:36 -08:00
Alex Crichton
51a4238dcc configure: Remove misc unused vars
None of this is used by rustbuild any more
2017-03-11 08:41:11 -08:00
Alex Crichton
748e2de7cc configure: Remove SUPPORTED_TARGET support
This is not used any more
2017-03-11 08:41:11 -08:00
Alex Crichton
e82bcabf07 configure: Remove some lldb-specific logic
All of this should already be vendored in rustbuild if necessary or otherwise
it's just not used.
2017-03-11 08:41:10 -08:00
Alex Crichton
e019f133a0 configure: Remove miscellaneous program probes
All of these should be handled by rustbuild in sanity.rs right now.
2017-03-11 08:41:10 -08:00
Alex Crichton
b326bd31e5 configure: Remove md5 probing logic
This is all not used any more
2017-03-11 08:41:10 -08:00
Alex Crichton
3308bd54b2 configure: Remove git probing logic
This is all in rustbuild already.
2017-03-11 08:41:10 -08:00
Tatsuyuki Ishi
e31e264c55 rustbuild: Make save-analysis an option 2017-03-11 20:00:05 +09:00
Ximin Luo
0a55c8e659 Support armhf abi on 64-bit ARM cpus
They report their `uname -m` as armv8l rather than aarch64.

Patch originally by Matthias Klose <doko@debian.org>
2017-03-10 08:13:21 -08:00
Alex Crichton
44a01b8a54 rustbuild: Add support for compiling Cargo
This commit adds support to rustbuild for compiling Cargo as part of the release
process. Previously rustbuild would simply download a Cargo snapshot and
repackage it. With this change we should be able to turn off artifacts from the
rust-lang/cargo repository and purely rely on the artifacts Cargo produces here.

The infrastructure added here is intended to be extensible to other components,
such as the RLS. It won't exactly be a one-line addition, but the addition of
Cargo didn't require too much hooplah anyway.

The process for release Cargo will now look like:

* The rust-lang/rust repository has a Cargo submodule which is used to build a
  Cargo to pair with the rust-lang/rust release
* Periodically we'll update the cargo submodule as necessary on rust-lang/rust's
  master branch
* When branching beta we'll create a new branch of Cargo (as we do today), and
  the first commit to the beta branch will be to update the Cargo submodule to
  this exact revision.
* When branching stable, we'll ensure that the Cargo submodule is updated and
  then make a stable release.

Backports to Cargo will look like:

* Send a PR to cargo's master branch
* Send a PR to cargo's release branch (e.g. rust-1.16.0)
* Send a PR to rust-lang/rust's beta branch updating the submodule
* Eventually send a PR to rust-lang/rust's master branch updating the submodule

For reference, the process to add a new component to the rust-lang/rust release
would look like:

* Add `$foo` as a submodule in `src/tools`
* Add a `tool-$foo` step which compiles `$foo` with the specified compiler,
  likely mirroring what Cargo does.
* Add a `dist-$foo` step which uses `src/tools/$foo` and the `tool-$foo` output
  to create a rust-installer package for `$foo` likely mirroring what Cargo
  does.
* Update the `dist-extended` step with a new dependency on `dist-$foo`
* Update `src/tools/build-manifest` for the new component.
2017-03-03 07:29:31 -08:00
Alex Crichton
1902488228 travis: Disable source tarballs on most builders
Currently we create a source tarball on almost all of the `DEPLOY=1` builders
but this has the adverse side effect of all source tarballs overriding
themselves in the S3 bucket. Normally this is ok but unfortunately a source
tarball created on Windows is not buildable on Unix.

On Windows the vendored sources contain paths with `\` characters in them which
when interpreted on Unix end up in "file not found" errors.

Instead of this overwriting behavior, whitelist just one linux builder for
producing tarballs and avoid producing tarballs on all other hosts.
2017-02-15 18:07:16 -08:00
bors
48bc08247a Auto merge of - eddyb:vendeur-tres-bien, r=alexcrichton
Automate vendoring by invoking cargo-vendor when building src dist tarballs.

This avoids  bringing the `src/vendor` checked into git by , past 200,000 lines of code.

I believe the strategy of having rustbuild run `cargo vendor` during the `dist src` step is sound.

However, the only way to be sure `cargo-vendor` exists is to run `cargo install --force cargo-vendor`, which will recompile it every time (not passing `--force` means you can't tell between "already exists" and "build error"). ~~This is quite suboptimal and I'd like to somehow do it in each `Dockerfile` that would need it.~~

* [ ] Cache `CARGO_HOME` (i.e. `~/.cargo`) between CI runs
  * `bin/cargo-vendor` and the actual caches are the relevant bits
* [x] Do not build `cargo-vendor` all the time
  * ~~Maybe detect `~/.cargo/bin/cargo-vendor` already exists?~~
  * ~~Could also try to build it in a `Dockerfile` but do we have `cargo`/`rustc` there?~~
  * Final solution: check `cargo install --list` for a line starting with `cargo-vendor `

cc @rust-lang/tools
2017-02-14 07:06:25 +00:00
Eduard-Mihai Burtescu
d29f0bc8fa Automatically vendor Cargo deps when building the source tarballs. 2017-02-14 01:52:03 +02:00
Corey Farwell
6194a7643c Remove '--disable-rustbuild' option from configure script.
Fixes https://github.com/rust-lang/rust/issues/39762.
2017-02-12 19:16:22 -05:00
Corey Farwell
3053494a9a Rollup merge of - japaric:lsan, r=alexcrichton
LeakSanitizer, ThreadSanitizer, AddressSanitizer and MemorySanitizer support

```
$ cargo new --bin leak && cd $_

$ edit Cargo.toml && tail -n3 $_
```

``` toml
[profile.dev]
opt-level = 1
```

```
$ edit src/main.rs && cat $_
```

``` rust
use std::mem;

fn main() {
    let xs = vec![0, 1, 2, 3];
    mem::forget(xs);
}
```

```
$ RUSTFLAGS="-Z sanitizer=leak" cargo run --target x86_64-unknown-linux-gnu; echo $?
    Finished dev [optimized + debuginfo] target(s) in 0.0 secs
     Running `target/debug/leak`

=================================================================
==10848==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 16 byte(s) in 1 object(s) allocated from:
     0x557c3488db1f in __interceptor_malloc /shared/rust/checkouts/lsan/src/compiler-rt/lib/lsan/lsan_interceptors.cc:55
     0x557c34888aaa in alloc::heap::exchange_malloc::h68f3f8b376a0da42 /shared/rust/checkouts/lsan/src/liballoc/heap.rs:138
     0x557c34888afc in leak::main::hc56ab767de6d653a $PWD/src/main.rs:4
     0x557c348c0806 in __rust_maybe_catch_panic ($PWD/target/debug/leak+0x3d806)

SUMMARY: LeakSanitizer: 16 byte(s) leaked in 1 allocation(s).
23
```

```
$ cargo new --bin racy && cd $_

$ edit src/main.rs && cat $_
```

``` rust
use std::thread;

static mut ANSWER: i32 = 0;

fn main() {
    let t1 = thread::spawn(|| unsafe { ANSWER = 42 });
    unsafe {
        ANSWER = 24;
    }
    t1.join().ok();
}
```

```
$ RUSTFLAGS="-Z sanitizer=thread" cargo run --target x86_64-unknown-linux-gnu; echo $?
==================
WARNING: ThreadSanitizer: data race (pid=12019)
  Write of size 4 at 0x562105989bb4 by thread T1:
     racy::main::_$u7b$$u7b$closure$u7d$$u7d$::hbe13ea9e8ac73f7e $PWD/src/main.rs:6 (racy+0x000000010e3f)
     _$LT$std..panic..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h2e466a92accacc78 /shared/rust/checkouts/lsan/src/libstd/panic.rs:296 (racy+0x000000010cc5)
     std::panicking::try::do_call::h7f4d2b38069e4042 /shared/rust/checkouts/lsan/src/libstd/panicking.rs:460 (racy+0x00000000c8f2)
     __rust_maybe_catch_panic <null> (racy+0x0000000b4e56)
     std::panic::catch_unwind::h31ca45621ad66d5a /shared/rust/checkouts/lsan/src/libstd/panic.rs:361 (racy+0x00000000b517)
     std:🧵:Builder::spawn::_$u7b$$u7b$closure$u7d$$u7d$::hccfc37175dea0b01 /shared/rust/checkouts/lsan/src/libstd/thread/mod.rs:357 (racy+0x00000000c226)
     _$LT$F$u20$as$u20$alloc..boxed..FnBox$LT$A$GT$$GT$::call_box::hd880bbf91561e033 /shared/rust/checkouts/lsan/src/liballoc/boxed.rs:605 (racy+0x00000000f27e)
     std::sys:👿🧵:Thread:🆕:thread_start::hebdfc4b3d17afc85 <null> (racy+0x0000000abd40)

  Previous write of size 4 at 0x562105989bb4 by main thread:
     racy::main::h23e6e5ca46d085c3 $PWD/src/main.rs:8 (racy+0x000000010d7c)
     __rust_maybe_catch_panic <null> (racy+0x0000000b4e56)
     __libc_start_main <null> (libc.so.6+0x000000020290)

  Location is global 'racy::ANSWER::h543d2b139f819b19' of size 4 at 0x562105989bb4 (racy+0x0000002f8bb4)

  Thread T1 (tid=12028, running) created by main thread at:
     pthread_create /shared/rust/checkouts/lsan/src/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:902 (racy+0x00000001aedb)
     std::sys:👿🧵:Thread:🆕:hce44187bf4a36222 <null> (racy+0x0000000ab9ae)
     std:🧵:spawn::he382608373eb667e /shared/rust/checkouts/lsan/src/libstd/thread/mod.rs:412 (racy+0x00000000b5aa)
     racy::main::h23e6e5ca46d085c3 $PWD/src/main.rs:6 (racy+0x000000010d5c)
     __rust_maybe_catch_panic <null> (racy+0x0000000b4e56)
     __libc_start_main <null> (libc.so.6+0x000000020290)

SUMMARY: ThreadSanitizer: data race $PWD/src/main.rs:6 in racy::main::_$u7b$$u7b$closure$u7d$$u7d$::hbe13ea9e8ac73f7e
==================
ThreadSanitizer: reported 1 warnings
66
```

```
$ cargo new --bin oob && cd $_

$ edit src/main.rs && cat $_
```

``` rust
fn main() {
    let xs = [0, 1, 2, 3];
    let y = unsafe { *xs.as_ptr().offset(4) };
}
```

```
$ RUSTFLAGS="-Z sanitizer=address" cargo run --target x86_64-unknown-linux-gnu; echo $?
=================================================================
==13328==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fff29f3ecd0 at pc 0x55802dc6bf7e bp 0x7fff29f3ec90 sp 0x7fff29f3ec88
READ of size 4 at 0x7fff29f3ecd0 thread T0
     0x55802dc6bf7d in oob::main::h0adc7b67e5feb2e7 $PWD/src/main.rs:3
     0x55802dd60426 in __rust_maybe_catch_panic ($PWD/target/debug/oob+0xfe426)
     0x55802dd58dd9 in std::rt::lang_start::hb2951fc8a59d62a7 ($PWD/target/debug/oob+0xf6dd9)
     0x55802dc6c002 in main ($PWD/target/debug/oob+0xa002)
     0x7fad8c3b3290 in __libc_start_main (/usr/lib/libc.so.6+0x20290)
     0x55802dc6b719 in _start ($PWD/target/debug/oob+0x9719)

Address 0x7fff29f3ecd0 is located in stack of thread T0 at offset 48 in frame
     0x55802dc6bd5f in oob::main::h0adc7b67e5feb2e7 $PWD/src/main.rs:1

  This frame has 1 object(s):
    [32, 48) 'xs' <== Memory access at offset 48 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow $PWD/src/main.rs:3 in oob::main::h0adc7b67e5feb2e7
Shadow bytes around the buggy address:
  0x1000653dfd40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1000653dfd50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1000653dfd60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1000653dfd70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1000653dfd80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x1000653dfd90: 00 00 00 00 f1 f1 f1 f1 00 00[f3]f3 00 00 00 00
  0x1000653dfda0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1000653dfdb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1000653dfdc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1000653dfdd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1000653dfde0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==13328==ABORTING
1
```

```
$ cargo new --bin uninit && cd $_

$ edit src/main.rs && cat $_
```

``` rust
use std::mem;

fn main() {
    let xs: [u8; 4] = unsafe { mem::uninitialized() };
    let y = xs[0] + xs[1];
}
```

```
$ RUSTFLAGS="-Z sanitizer=memory" cargo run; echo $?
==30198==WARNING: MemorySanitizer: use-of-uninitialized-value
     0x563f4b6867da in uninit::main::hc2731cd4f2ed48f8 $PWD/src/main.rs:5
     0x563f4b7033b6 in __rust_maybe_catch_panic ($PWD/target/debug/uninit+0x873b6)
     0x563f4b6fbd69 in std::rt::lang_start::hb2951fc8a59d62a7 ($PWD/target/debug/uninit+0x7fd69)
     0x563f4b6868a9 in main ($PWD/target/debug/uninit+0xa8a9)
     0x7fe844354290 in __libc_start_main (/usr/lib/libc.so.6+0x20290)
     0x563f4b6864f9 in _start ($PWD/target/debug/uninit+0xa4f9)

SUMMARY: MemorySanitizer: use-of-uninitialized-value $PWD/src/main.rs:5 in uninit::main::hc2731cd4f2ed48f8
Exiting
77
```
2017-02-08 23:55:43 -05:00
Jorge Aparicio
775a93646c build/test the sanitizers only when --enable-sanitizers is used 2017-02-08 18:51:43 -05:00
Corey Farwell
6fb57bf13f Rollup merge of - alexcrichton:no-more-makefiles, r=brson
Delete the makefile build system

This PR deletes the makefile build system in favor of the rustbuild build system. The beta has now been branched so 1.16 will continue to be buildable from the makefiles, but going forward 1.17 will only be buildable with rustbuild.

Rustbuild has been the default build system [since 1.15.0](https://github.com/rust-lang/rust/pull/37817) and the makefiles were [proposed for deletion](https://internals.rust-lang.org/t/proposal-for-promoting-rustbuild-to-official-status/4368) at this time back in November of last year.

And now with the deletion of these makefiles we can start getting those sweet sweet improvements of using crates.io crates in the compiler!
2017-02-07 22:54:25 -05:00
Corey Farwell
370b63f386 Rollup merge of - alexcrichton:arm-cross-test, r=brson
Add support for test suites emulated in QEMU

This commit adds support to the build system to execute test suites that cannot
run natively but can instead run inside of a QEMU emulator. A proof-of-concept
builder was added for the `arm-unknown-linux-gnueabihf` target to show off how
this might work.

In general the architecture is to have a server running inside of the emulator
which a local client connects to. The protocol between the server/client
supports compiling tests on the host and running them on the target inside the
emulator.

Closes 
2017-02-07 22:54:23 -05:00
Alex Crichton
9b0e6af110 Delete swaths of the configure script
This commit deletes swaths of the configure script related to the old build
system which are now no longer needed when using rustbuild.
2017-02-06 08:42:54 -08:00
Corey Farwell
b3518aff90 Rollup merge of - dumbbell:support-aarch64-unknown-freebsd, r=alexcrichton
Support aarch64-unknown-freebsd
2017-02-05 12:45:06 -05:00
Jean-Sébastien Pédron
f6c6b31d26
aarch64 CPU type is called arm64 on FreeBSD 2017-02-03 09:11:33 +01:00
Alex Crichton
1747ce25ad Add support for test suites emulated in QEMU
This commit adds support to the build system to execute test suites that cannot
run natively but can instead run inside of a QEMU emulator. A proof-of-concept
builder was added for the `arm-unknown-linux-gnueabihf` target to show off how
this might work.

In general the architecture is to have a server running inside of the emulator
which a local client connects to. The protocol between the server/client
supports compiling tests on the host and running them on the target inside the
emulator.

Closes 
2017-01-29 14:16:41 -08:00
Alex Crichton
f3dfcae202 rustbuild: Start building --enable-extended
This commit adds a new flag to the configure script,
`--enable-extended`, which is intended for specifying a desire to
compile the full suite of Rust tools such as Cargo, the RLS, etc. This
is also an indication that the build system should create combined
installers such as the pkg/exe/msi artifacts.

Currently the `--enable-extended` flag just indicates that combined
installers should be built, and Cargo is itself not compiled just yet
but rather only downloaded from its location. The intention here is to
quickly get to feature parity with the current release process and then
we can start improving it afterwards.

All new files in this PR inside `src/etc/installer` are copied from the
rust-packaging repository.
2017-01-24 14:48:03 -08:00
Alex Crichton
0618580235 travis: Pass --release-channel=nightly on deploy
This commit passes the `--release-channel=nightly` flag to all images which have
the `DEPLOY` flag set. This means that we'll name artifacts and the compiler
appropriately.

This reworks a bit how arguments are passed, but for now doesn't change what's
already being passed. Eventually we'll want to avoid enabling debug assertions
and llvm assertions for *all* releases, but I figure we can tackle that a little
bit more down the road.
2017-01-16 22:26:21 -08:00
Ximin Luo
5740f94fed Detect mips CPUs in ./configure
This mirrors existing logic already in src/bootstrap/bootstrap.py
2017-01-12 19:33:45 +01:00
Alex Crichton
099e7cb120 rustbuild: Don't enable debuginfo in rustc
In  we enabled line number debugging information in release artifacts,
primarily to close out  where debugging information was critical for MSVC
builds of Rust to be useful in production. This commit, however, apparently had
some unfortunate side effects.

Namely it was noticed in  that if `RUST_BACKTRACE=1` was set then any
compiler error would take a very long time for the compiler to exit. The cause
of the problem here was somewhat deep:

* For all compiler errors, the compiler will `panic!` with a known value. This
  tears down the main compiler thread and allows cleaning up all the various
  resources. By default, however, this panic output is suppressed for "normal"
  compiler errors.
* When `RUST_BACKTRACE=1` was set this caused every compiler error to generate a
  backtrace.
* The libbacktrace library hits a pathological case where it spends a very long
  time in its custom allocation function, `backtrace_alloc`, because the
  compiler has so much debugging information. More information about this can be
  found in  with a summary at the end of .

To solve this problem this commit simply removes debuginfo from the compiler but
not from the standard library. This should allow us to keep  closed while
also closing . I've measured the difference to be orders of magnitude
faster than it was before, so we should see a much quicker time-to-exit after a
compile error when `RUST_BACKTRACE=1` is set.

Closes 
Closes 
2017-01-10 20:33:12 -08:00
Clar Charr
8ffc3e7790 Reword 'stupid' and 'crazy' in docs. 2017-01-02 16:29:19 -05:00
Seo Sanghyeon
b14785d3d0 Merge branch 'master' into sparc64 2017-01-01 12:40:10 +09:00
Jonathan A. Kollasch
982849535d further enable the Sparc LLVM backend 2016-12-29 21:30:01 -05:00
Alex Crichton
7046fea5be rustbuild: Compile rustc twice, not thrice
This commit switches the rustbuild build system to compiling the
compiler twice for a normal bootstrap rather than the historical three
times.

Rust is a bootstrapped language which means that a previous version of
the compiler is used to build the next version of the compiler. Over
time, however, we change many parts of compiler artifacts such as the
metadata format, symbol names, etc. These changes make artifacts from
one compiler incompatible from another compiler. Consequently if a
compiler wants to be able to use some artifacts then it itself must have
compiled the artifacts.

Historically the rustc build system has achieved this by compiling the
compiler three times:

* An older compiler (stage0) is downloaded to kick off the chain.
* This compiler now compiles a new compiler (stage1)
* The stage1 compiler then compiles another compiler (stage2)
* Finally, the stage2 compiler needs libraries to link against, so it
  compiles all the libraries again.

This entire process amounts in compiling the compiler three times.
Additionally, this process always guarantees that the Rust source tree
can compile itself because the stage2 compiler (created by a freshly
created compiler) would successfully compile itself again. This
property, ensuring Rust can compile itself, is quite important!

In general, though, this third compilation is not required for general
purpose development on the compiler. The third compiler (stage2) can
reuse the libraries that were created during the second compile. In
other words, the second compilation can produce both a compiler and the
libraries that compiler will use. These artifacts *must* be compatible
due to the way plugins work today anyway, and they were created by the
same source code so they *should* be compatible as well.

So given all that, this commit switches the default build process to
only compile the compiler three times, avoiding this third compilation
by copying artifacts from the previous one. Along the way a new entry in
the Travis matrix was also added to ensure that our full bootstrap can
succeed. This entry does not run tests, though, as it should not be
necessary.

To restore the old behavior of a full bootstrap (three compiles) you can
either pass:

    ./configure --enable-full-bootstrap

or if you're using config.toml:

    [build]
    full-bootstrap = true

Overall this will hopefully be an easy 33% win in build times of the
compiler. If we do 33% less work we should be 33% faster! This in turn
should affect cycle times and such on Travis and AppVeyor positively as
well as making it easier to work on the compiler itself.
2016-12-28 14:49:00 -08:00
Brian Anderson
7d428b71de Delete the llvm submodule lockfile when configuring on the bots
This should fix the periodic error that .git/modules/src/llvm/index.lock
exists on the mac slaves.
2016-12-22 22:33:42 +00:00
Alex Crichton
96a5fc76dc rustbuild: Add sccache support
This commit adds support for sccache, a ccache-like compiler which works on MSVC
and stores results into an S3 bucket. This also switches over all Travis and
AppVeyor automation to using sccache to ensure a shared and unified cache over
time which can be shared across builders.

The support for sccache manifests as a new `--enable-sccache` option which
instructs us to configure LLVM differently to use a 'sccache' binary instead of
a 'ccache' binary. All docker images for Travis builds are updated to download
Mozilla's tooltool builds of sccache onto various containers and systems.
Additionally a new `rust-lang-ci-sccache` bucket is configured to hold all of
our ccache goodies.
2016-12-14 15:40:18 -08:00
Felix S. Klock II
25adc4e82e Fix but perhaps not BEST fix for it. 2016-12-08 13:09:20 -10:00
Alex Crichton
0e272de69f mk: Switch rustbuild to the default build system
This commit switches the default build system for Rust from the makefiles to
rustbuild. The rustbuild build system has been in development for almost a year
now and has become quite mature over time. This commit is an implementation of
the proposal on [internals] which slates deletion of the makefiles on
2016-01-02.

[internals]: https://internals.rust-lang.org/t/proposal-for-promoting-rustbuild-to-official-status/4368

This commit also updates various documentation in `README.md`,
`CONTRIBUTING.md`, `src/bootstrap/README.md`, and throughout the source code of
rustbuild itself.

Closes 
2016-12-07 00:30:23 -08:00
Doug Goldstein
f83eb4009e
configure: quote variables
These should probably be quoted.

Signed-off-by: Doug Goldstein <cardoe@cardoe.com>
2016-12-02 11:25:43 -06:00
Doug Goldstein
e39c8d6dc8
configure: only req CMake if we're building LLVM
CMake is only necessary if LLVM is going to be built and not in any
other case.

Signed-off-by: Doug Goldstein <cardoe@cardoe.com>
2016-12-02 11:22:57 -06:00
Brian Anderson
295148862a configure: Fix string equality 2016-11-21 22:11:20 +00:00
bors
b1da18fe9b Auto merge of - cuviper:llvm-link-shared, r=alexcrichton
rustbuild: allow dynamically linking LLVM

The makefiles and `mklldeps.py` called `llvm-config --shared-mode` to
find out if LLVM defaulted to shared or static libraries, and just went
with that.  But under rustbuild, `librustc_llvm/build.rs` was assuming
that LLVM should be static, and even forcing `--link-static` for 3.9+.

Now that build script also uses `--shared-mode` to learn the default,
which should work better for pre-3.9 configured for dynamic linking, as
it wasn't possible back then to choose differently via `llvm-config`.

Further, the configure script now has a new `--enable-llvm-link-shared`
option, which allows one to manually override `--link-shared` on 3.9+
instead of forcing static.

Update: There are now four static/shared scenarios that can happen
for the supported LLVM versions:

- 3.9+: By default use `llvm-config --link-static`
- 3.9+ and `--enable-llvm-link-shared`: Use `--link-shared` instead.
- 3.8: Use `llvm-config --shared-mode` and go with its answer.
- 3.7: Just assume static, maintaining the status quo.
2016-11-19 08:08:26 -08:00