ci: Upload/download from a new S3 bucket
Moving buckets from us-east-1 to us-west-1 because us-west-1 is where
rust-central-station itself runs and in general is where we have all our other
buckets.
ci: Disable rustc debug assertions on OSX
This commit disables debug assertions for OSX in an effort to improve cycle time
on OSX. It looks like #44610 didn't shave off quite as much time as desired so
let's see how much this helps.
This commit disables debug assertions for OSX in an effort to improve cycle time
on OSX. It looks like #44610 didn't shave off quite as much time as desired so
let's see how much this helps.
Make use of Travis's conditional jobs.
Conditional jobs: https://docs.travis-ci.com/user/conditional-builds-stages-jobs/#Conditional-Jobs.
Jobs not matching the condition will not be scheduled at all. This allows us to get rid of `$ALLOW_PR`/`$ALLOW_TRY`/`$SKIP_BUILD` in `.travis.yml`, and perfectly prevent spurious PR failures due to flaky macOS machines.
travis: Move sccache to the us-west-1 region
Most of the other rust-lang buckets are in us-west-1 and I think the original
bucket was just accidentally created in the us-east-1 region. Let's consolidate
by moving it to the same location as the rest of our buckets.
Moving buckets from us-east-1 to us-west-1 because us-west-1 is where
rust-central-station itself runs and in general is where we have all our other
buckets.
Our OSX builders are routinely and significantly over hour 2 hour "soft limit"
for testing PRs. I *think* that a big portion of this time comes from the fact
that LLVM and debug assertions are enabled. In an effort to speed up these
builders and reduce cycle time this commit disables LLVM assertions on OSX for
all builders.
My thinking is that we'll let this bake for a bit after merged to see what the
effect is on timing on Travis. If it doesn't actually help much we can turn them
back on, and if it doesn't help enough we can disable Rust debug assertions as
well.
Most of the other rust-lang buckets are in us-west-1 and I think the original
bucket was just accidentally created in the us-east-1 region. Let's consolidate
by moving it to the same location as the rest of our buckets.
Fix sanitizer tests on buggy kernels
Travis recently pushed an update to the Linux environments, namely the kernels
that we're running on. This in turn caused some of the sanitizer tests we run to
fail. We also apparently weren't the first to hit these failures! Detailed in
google/sanitizers#837 these tests were failing due to a specific commit in the
kernel which has since been backed out, but for now work around the buggy kernel
that's deployed on Travis and eventually we should be able to remove these
flags.
Travis recently pushed an update to the Linux environments, namely the kernels
that we're running on. This in turn caused some of the sanitizer tests we run to
fail. We also apparently weren't the first to hit these failures! Detailed in
google/sanitizers#837 these tests were failing due to a specific commit in the
kernel which has since been backed out, but for now work around the buggy kernel
that's deployed on Travis and eventually we should be able to remove these
flags.
Reorder the job matrix to take advantage of the order how Travis CI starts
them in rust-lang/rust. Plus other refactoring of `.travis.yml`.
1. Move the `$ALLOW_PR` image to the top, so users' PRs will start testing
immediately. Previously the `$ALLOW_PR` image starts 6 minutes after the
build was scheduled.
2. Move the slow macOS images near the top, so they share more time with
the rest of the faster Linux builds, which should shorten total test
time (actually not much, about 7 minutes at most if this change does
work).
3. Merged the `install` section of both Linux and macOS to make the `env:`
section a bit shorter, and enable change 4 below.
4. Do not download or install anything if `$SKIP_BUILD == true`, which
further reduces chance of spurious failure in the PR-CI stage (avoid the
red cross appearing even if CI passed).
Use the `-f` flag to indicate that, for example, a 500 response code is to be
considered a failure, triggering the normal retry logic. Also ignore errors
where we check the date from google.com, as a failure there shouldn't fail the
build.
Add a travis builder for wasm32-unknown-emscripten
This commits add an entry to travis matrix that will execute wasm32-unknown-emscripten tests suites.
- Emscripten for asmjs was updated to sdk-1.37.13-64bit
- The tests are run with node 8.0.0 (it can execute wasm)
- A wrapper script is used to run each test from the directory where it is (workaround for https://github.com/kripken/emscripten/issues/4542)
- Some tests are ignore, see #42629 and #42630
Build instruction profiler runtime as part of compiler-rt
r? @alexcrichton
This is #38608 with some fixes.
Still missing:
- [x] testing with profiler enabled on some builders (on which ones? Should I add the option to some of the already existing configurations, or create a new configuration?);
- [x] enabling distribution (on which builders?);
- [x] documentation.
There is no `minimal` language. Due to travis-ci/travis-ci#4895, it will
fallback to `ruby`, which certainly isn't what we want. `generic` is an
undocumented (travis-ci/docs-travis-ci-com#910) language that serves the
desired purpose.
* Bring back colors on Travis, which was disabled since #39036.
Append --color=always to cargo when running in CI environment.
* Removed `set -x` in the shell scripts. The `retry` function already
prints which command it is running, add `-x` just add noise to the
output.
* Support travis_fold/travis_time. Matching pairs of these allow Travis CI
to collapse the output in between. This greatly cut down the unnecessary
"successful" output one need to scroll through before finding the failed
statement.
Generate XZ-compressed tarballs
Integrate the new `rust-installer` and extend manifests with keys for xz-compressed tarballs.
One of the steps required for https://github.com/rust-lang/rust/issues/21724
Now that the final bug fixes have been merged into sccache we can start
leveraging sccache on the MSVC builders on AppVeyor instead of relying on the
ad-hoc caching strategy of trigger files and whatnot.
ASan and TSan are supported on macOS, and this commit enables their
support.
The sanitizers are always built as *.dylib on Apple platforms, so they
cannot be statically linked into the corresponding `rustc_?san.rlib`. The
dylibs are directly copied to `lib/rustlib/x86_64-apple-darwin/lib/`
instead.
Note, although Xcode also ships with their own copies of ASan/TSan dylibs,
we cannot use them due to version mismatch.
There is a caveat: the sanitizer libraries are linked as @rpath, so the
user needs to additionally pass `-C rpath`:
rustc -Z sanitizer=address -C rpath file.rs
^~~~~~~~
Otherwise there will be a runtime error:
dyld: Library not loaded: @rpath/libclang_rt.asan_osx_dynamic.dylib
Referenced from: /path/to/executable
Reason: image not found
Abort trap: 6
The next commit includes a temporary change in compiler to force the linker
to emit a usable @rpath.
I've tracked down what I believe is the last spurious sccache failure on #40240
to behavior in mio (carllerche/mio#583), and this commit updates the binaries to
a version which has that fix incorporated.
Previously we would use one builder on Travis to produce two sets of host
compilers for two different targets. Unfortunately though we've recently
increased how much we're building for each target so this is starting to take
unnecessarily long (#40804). This commit splits the dist builders in two by
ensuring that we only dist one target on each builder, which should take a much
shorter amount of time. This should also unblock other work such as landing the
RLS (#40584).
travis: Compile OSX releases with Xcode 7
Unfortunately what we're using right now, Xcode 8.2, cannot compile LLVM for OSX
10.7. We've done this historically and Gecko would like to maintain this
compabitiliby. This commit moves our release builders for OSX to using Xcode 7
which can compile LLVM for 10.7.
The builders running tests continue to use Xcode 8.2, however, because the LLDB
version with Xcode 7, 350, is blacklisted in running our LLDB tests. To continue
running LLDB tests we'll stick with Xcode 8.2.
Unfortunately what we're using right now, Xcode 8.2, cannot compile LLVM for OSX
10.7. We've done this historically and Gecko would like to maintain this
compabitiliby. This commit moves our release builders for OSX to using Xcode 7
which can compile LLVM for 10.7.
The builders running tests continue to use Xcode 8.2, however, because the LLDB
version with Xcode 7, 350, is blacklisted in running our LLDB tests. To continue
running LLDB tests we'll stick with Xcode 8.2.
Attempt to cache git modules
Partial resolution of #40772, appveyor remains to be done once travis looks like it's working ok.
The approach in this PR is based on the `--reference` flag to `git-clone`/`git-submodule --update` and is a compromise based on the current limitations of the tools we're using.
The ideal would be:
1. have a cached pristine copy of rust-lang/rust master in `$HOME/rustsrc` with all submodules initialised
2. clone the PR branch with `git clone --recurse-submodules --reference $HOME/rustsrc git@github.com:rust-lang/rust.git`
This would (in the nonexistent ideal world) use the pristine copy as an object cache for the top level repo and all submodules, transferring over the network only the changes on the branch. Unfortunately, a) there is no way to manually control the initial clone with travis and b) even if there was, cloned submodules don't use the submodules of the reference as an object cache. So the steps we end up with are:
1. have a cached pristine copy of rust-lang/rust master in `$HOME/rustsrc` with all submodules initialised
2. have a cloned PR branch
3. extract the path of each submodule, and explicitly `git submodule update --init --reference $HOME/rustsrc/$module $module` (i.e. point directly to the location of the pristine submodule repo) for each one
I've also taken some care to make this forward compatible, both for adding and removing submodules.
r? @alexcrichton
