rust/.travis.yml

368 lines
13 KiB
YAML
Raw Normal View History

language: shell
sudo: required
dist: trusty
2016-06-01 10:14:22 -05:00
services:
- docker
git:
depth: 2
submodules: false
matrix:
2017-01-19 19:18:12 -06:00
fast_finish: true
include:
# Images used in testing PR and try-build should be run first.
- env: IMAGE=x86_64-gnu-llvm-3.9 RUST_BACKTRACE=1
2017-09-16 06:28:08 -05:00
if: type = pull_request OR branch = auto
- env: IMAGE=dist-x86_64-linux DEPLOY=1
if: branch = try OR branch = auto
# "alternate" deployments, these are "nightlies" but have LLVM assertions
# turned on, they're deployed to a different location primarily for
# additional testing.
- env: IMAGE=dist-x86_64-linux DEPLOY_ALT=1 CI_JOB_NAME=dist-x86_64-linux-alt
if: branch = try OR branch = auto
2017-09-16 06:28:08 -05:00
- env: >
RUST_CHECK_TARGET=dist
RUST_CONFIGURE_ARGS="--enable-extended --enable-profiler"
SRC=.
DEPLOY_ALT=1
RUSTC_RETRY_LINKER_ON_SEGFAULT=1
MACOSX_DEPLOYMENT_TARGET=10.7
NO_LLVM_ASSERTIONS=1
NO_DEBUG_ASSERTIONS=1
CI_JOB_NAME=dist-x86_64-apple-alt
os: osx
osx_image: xcode9.3-moar
if: branch = auto
# macOS builders. These are placed near the beginning because they are very
# slow to run.
# OSX builders running tests, these run the full test suite.
#
# Note that the compiler is compiled to target 10.8 here because the Xcode
# version that we're using, 8.2, cannot compile LLVM for OSX 10.7.
- env: >
RUST_CHECK_TARGET=check
RUST_CONFIGURE_ARGS="--build=x86_64-apple-darwin --enable-sanitizers --enable-profiler"
SRC=.
RUSTC_RETRY_LINKER_ON_SEGFAULT=1
2017-03-13 13:36:44 -05:00
MACOSX_DEPLOYMENT_TARGET=10.8
MACOSX_STD_DEPLOYMENT_TARGET=10.7
NO_LLVM_ASSERTIONS=1
NO_DEBUG_ASSERTIONS=1
CI_JOB_NAME=x86_64-apple
os: osx
osx_image: xcode9.3-moar
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: >
RUST_CHECK_TARGET=check
RUST_CONFIGURE_ARGS=--build=i686-apple-darwin
SRC=.
RUSTC_RETRY_LINKER_ON_SEGFAULT=1
2017-03-13 13:36:44 -05:00
MACOSX_DEPLOYMENT_TARGET=10.8
MACOSX_STD_DEPLOYMENT_TARGET=10.7
NO_LLVM_ASSERTIONS=1
NO_DEBUG_ASSERTIONS=1
CI_JOB_NAME=i686-apple
os: osx
osx_image: xcode9.3-moar
2017-09-16 06:28:08 -05:00
if: branch = auto
# OSX builders producing releases. These do not run the full test suite and
# just produce a bunch of artifacts.
#
# Note that these are running in the `xcode7` image instead of the
# `xcode8.2` image as above. That's because we want to build releases for
# OSX 10.7 and `xcode7` is the latest Xcode able to compile LLVM for 10.7.
- env: >
RUST_CHECK_TARGET=dist
rust: Import LLD for linking wasm objects This commit imports the LLD project from LLVM to serve as the default linker for the `wasm32-unknown-unknown` target. The `binaryen` submoule is consequently removed along with "binaryen linker" support in rustc. Moving to LLD brings with it a number of benefits for wasm code: * LLD is itself an actual linker, so there's no need to compile all wasm code with LTO any more. As a result builds should be *much* speedier as LTO is no longer forcibly enabled for all builds of the wasm target. * LLD is quickly becoming an "official solution" for linking wasm code together. This, I believe at least, is intended to be the main supported linker for native code and wasm moving forward. Picking up support early on should help ensure that we can help LLD identify bugs and otherwise prove that it works great for all our use cases! * Improvements to the wasm toolchain are currently primarily focused around LLVM and LLD (from what I can tell at least), so it's in general much better to be on this bandwagon for bugfixes and new features. * Historical "hacks" like `wasm-gc` will soon no longer be necessary, LLD will [natively implement][gc] `--gc-sections` (better than `wasm-gc`!) which means a postprocessor is no longer needed to show off Rust's "small wasm binary size". LLD is added in a pretty standard way to rustc right now. A new rustbuild target was defined for building LLD, and this is executed when a compiler's sysroot is being assembled. LLD is compiled against the LLVM that we've got in tree, which means we're currently on the `release_60` branch, but this may get upgraded in the near future! LLD is placed into rustc's sysroot in a `bin` directory. This is similar to where `gcc.exe` can be found on Windows. This directory is automatically added to `PATH` whenever rustc executes the linker, allowing us to define a `WasmLd` linker which implements the interface that `wasm-ld`, LLD's frontend, expects. Like Emscripten the LLD target is currently only enabled for Tier 1 platforms, notably OSX/Windows/Linux, and will need to be installed manually for compiling to wasm on other platforms. LLD is by default turned off in rustbuild, and requires a `config.toml` option to be enabled to turn it on. Finally the unstable `#![wasm_import_memory]` attribute was also removed as LLD has a native option for controlling this. [gc]: https://reviews.llvm.org/D42511
2017-08-26 20:30:12 -05:00
RUST_CONFIGURE_ARGS="--build=i686-apple-darwin --enable-full-tools --enable-profiler"
SRC=.
DEPLOY=1
RUSTC_RETRY_LINKER_ON_SEGFAULT=1
MACOSX_DEPLOYMENT_TARGET=10.7
NO_LLVM_ASSERTIONS=1
NO_DEBUG_ASSERTIONS=1
CI_JOB_NAME=dist-i686-apple
os: osx
osx_image: xcode9.3-moar
if: branch = auto
2017-09-16 06:28:08 -05:00
- env: >
RUST_CHECK_TARGET=dist
rust: Import LLD for linking wasm objects This commit imports the LLD project from LLVM to serve as the default linker for the `wasm32-unknown-unknown` target. The `binaryen` submoule is consequently removed along with "binaryen linker" support in rustc. Moving to LLD brings with it a number of benefits for wasm code: * LLD is itself an actual linker, so there's no need to compile all wasm code with LTO any more. As a result builds should be *much* speedier as LTO is no longer forcibly enabled for all builds of the wasm target. * LLD is quickly becoming an "official solution" for linking wasm code together. This, I believe at least, is intended to be the main supported linker for native code and wasm moving forward. Picking up support early on should help ensure that we can help LLD identify bugs and otherwise prove that it works great for all our use cases! * Improvements to the wasm toolchain are currently primarily focused around LLVM and LLD (from what I can tell at least), so it's in general much better to be on this bandwagon for bugfixes and new features. * Historical "hacks" like `wasm-gc` will soon no longer be necessary, LLD will [natively implement][gc] `--gc-sections` (better than `wasm-gc`!) which means a postprocessor is no longer needed to show off Rust's "small wasm binary size". LLD is added in a pretty standard way to rustc right now. A new rustbuild target was defined for building LLD, and this is executed when a compiler's sysroot is being assembled. LLD is compiled against the LLVM that we've got in tree, which means we're currently on the `release_60` branch, but this may get upgraded in the near future! LLD is placed into rustc's sysroot in a `bin` directory. This is similar to where `gcc.exe` can be found on Windows. This directory is automatically added to `PATH` whenever rustc executes the linker, allowing us to define a `WasmLd` linker which implements the interface that `wasm-ld`, LLD's frontend, expects. Like Emscripten the LLD target is currently only enabled for Tier 1 platforms, notably OSX/Windows/Linux, and will need to be installed manually for compiling to wasm on other platforms. LLD is by default turned off in rustbuild, and requires a `config.toml` option to be enabled to turn it on. Finally the unstable `#![wasm_import_memory]` attribute was also removed as LLD has a native option for controlling this. [gc]: https://reviews.llvm.org/D42511
2017-08-26 20:30:12 -05:00
RUST_CONFIGURE_ARGS="--target=aarch64-apple-ios,armv7-apple-ios,armv7s-apple-ios,i386-apple-ios,x86_64-apple-ios --enable-full-tools --enable-sanitizers --enable-profiler"
SRC=.
DEPLOY=1
RUSTC_RETRY_LINKER_ON_SEGFAULT=1
MACOSX_DEPLOYMENT_TARGET=10.7
NO_LLVM_ASSERTIONS=1
NO_DEBUG_ASSERTIONS=1
CI_JOB_NAME=dist-x86_64-apple
os: osx
osx_image: xcode9.3-moar
if: branch = auto
# Linux builders, remaining docker images
- env: IMAGE=arm-android
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=armhf-gnu
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=dist-various-1 DEPLOY=1
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=dist-various-2 DEPLOY=1
if: branch = auto
- env: IMAGE=dist-aarch64-linux DEPLOY=1
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=dist-android DEPLOY=1
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=dist-arm-linux DEPLOY=1
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=dist-armhf-linux DEPLOY=1
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=dist-armv7-linux DEPLOY=1
2017-09-16 06:28:08 -05:00
if: branch = auto
2018-01-04 10:55:04 -06:00
- env: IMAGE=dist-i586-gnu-i586-i686-musl DEPLOY=1
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=dist-i686-freebsd DEPLOY=1
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=dist-i686-linux DEPLOY=1
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=dist-mips-linux DEPLOY=1
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=dist-mips64-linux DEPLOY=1
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=dist-mips64el-linux DEPLOY=1
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=dist-mipsel-linux DEPLOY=1
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=dist-powerpc-linux DEPLOY=1
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=dist-powerpc64-linux DEPLOY=1
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=dist-powerpc64le-linux DEPLOY=1
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=dist-s390x-linux DEPLOY=1
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=dist-x86_64-freebsd DEPLOY=1
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=dist-x86_64-musl DEPLOY=1
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=dist-x86_64-netbsd DEPLOY=1
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=asmjs
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=i686-gnu
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=i686-gnu-nopt
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=wasm32-unknown
if: branch = auto
- env: IMAGE=x86_64-gnu
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=x86_64-gnu-full-bootstrap
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=x86_64-gnu-aux
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=x86_64-gnu-tools
if: branch = auto
- env: IMAGE=x86_64-gnu-debug
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=x86_64-gnu-nopt
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=x86_64-gnu-distcheck
2017-09-16 06:28:08 -05:00
if: branch = auto
- env: IMAGE=mingw-check
if: type = pull_request OR branch = auto
- stage: publish toolstate
if: branch = master AND type = push
before_install: []
install: []
sudo: false
script:
MESSAGE_FILE=$(mktemp -t msg.XXXXXX);
. src/ci/docker/x86_64-gnu-tools/repo.sh;
commit_toolstate_change "$MESSAGE_FILE" "$TRAVIS_BUILD_DIR/src/tools/publish_toolstate.py" "$(git rev-parse HEAD)" "$(git log --format=%s -n1 HEAD)" "$MESSAGE_FILE" "$TOOLSTATE_REPO_ACCESS_TOKEN";
env:
global:
- SCCACHE_BUCKET=rust-lang-ci-sccache2
- SCCACHE_REGION=us-west-1
- AWS_ACCESS_KEY_ID=AKIAJAMV3QAMMA6AXHFQ
# AWS_SECRET_ACCESS_KEY=...
- secure: "j96XxTVOSUf4s4r4htIxn/fvIa5DWbMgLqWl7r8z2QfgUwscmkMXAwXuFNc7s7bGTpV/+CgDiMFFM6BAFLGKutytIF6oA02s9b+usQYnM0th7YQ2AIgm9GtMTJCJp4AoyfFmh8F2faUICBZlfVLUJ34udHEe35vOklix+0k4WDo="
# TOOLSTATE_REPO_ACCESS_TOKEN=...
- secure: "ESfcXqv4N2VMhqi2iIyw6da9VrsA78I4iR1asouCaq4hzTTrkB4WNRrfURy6xg72gQ4nMhtRJbB0/2jmc9Cu1+g2CzXtyiL223aJ5CKrXdcvbitopQSDfp07dMWm+UED+hNFEanpErKAeU/6FM3A+J+60PMk8MCF1h9tqNRISJw="
2015-10-18 20:37:14 -05:00
before_install:
ci: Don't use Travis caches for docker images This commit moves away from caching on Travis to our own caching on S3 for caching docker layers between builds. Unfortunately the Travis caches have over time had a few critical pain points: * Caches are only updated for successful builds, meaning that if a build times out or fails in a different location the sucessfully-created docker images isn't always cached. While this makes sense as a general rule of caches it hurts our use cases. * Caches are per-branch and builder which means that we don't have a separate cache on each release channel. All our merges go through the `auto` branch which means that they're all sharing the same cache, even those for merging to master/beta. This means that PRs which switch between master/beta will keep rebuilting and having cache misses. * Caches have historically been invaliated somewhat regularly a little more aggressively than we'd want (I think). * We don't always need to update the contents of the cache if the Docker image didn't change at all, and saving off the docker layers can sometimes be quite expensive. For all these reasons this commit drops the usage of Travis's built-in caching support. Instead our own caching is used by storing blobs to S3. Normally this would be a very risky endeavour but we're basically priming a cache for a cache (docker) so if we get this wrong the failure mode is longer builds, not stale caches. We'll notice that pretty quickly and hopefully fix it! The logic here is inserted directly into the `src/ci/docker/run.sh` script to download an image based on a shasum of the `Dockerfile` and other assorted files. This blob, if found, is loaded into docker and we record what layers were inserted. After docker finishes the build (hopefully quickly with lots of cache hits) we then see the sha of the final image. If it's one of the layers we loaded then there's no need to update the cache. Otherwise we upload our layers to the global cache, possibly overwriting what we previously just downloaded. This is hopefully a step towards mitigating #49278 although it doesn't completely fix it as it means we'll still probably have to retry builds that bust the cache.
2018-03-22 15:39:52 -05:00
# We'll use the AWS cli to download/upload cached docker layers, so install
# that here.
- if [ "$TRAVIS_OS_NAME" = linux ]; then
pip install --user awscli;
export PATH=$PATH:$HOME/.local/bin;
fi
- mkdir -p $HOME/rustsrc
# FIXME(#46924): these two commands are required to enable IPv6,
# they shouldn't exist, please revert once more official solutions appeared.
# see https://github.com/travis-ci/travis-ci/issues/8891#issuecomment-353403729
- if [ "$TRAVIS_OS_NAME" = linux ]; then
echo '{"ipv6":true,"fixed-cidr-v6":"fd9a:8454:6789:13f7::/64"}' | sudo tee /etc/docker/daemon.json;
sudo service docker restart;
fi
install:
2017-09-16 06:28:08 -05:00
- case "$TRAVIS_OS_NAME" in
linux)
travis_retry curl -fo $HOME/stamp https://s3-us-west-1.amazonaws.com/rust-lang-ci2/rust-ci-mirror/2017-03-17-stamp-x86_64-unknown-linux-musl &&
chmod +x $HOME/stamp &&
export PATH=$PATH:$HOME
;;
osx)
if [[ "$RUST_CHECK_TARGET" == dist ]]; then
travis_retry brew update &&
travis_retry brew install xz;
fi &&
travis_retry curl -fo /usr/local/bin/sccache https://s3-us-west-1.amazonaws.com/rust-lang-ci2/rust-ci-mirror/2018-04-02-sccache-x86_64-apple-darwin &&
chmod +x /usr/local/bin/sccache &&
travis_retry curl -fo /usr/local/bin/stamp https://s3-us-west-1.amazonaws.com/rust-lang-ci2/rust-ci-mirror/2017-03-17-stamp-x86_64-apple-darwin &&
chmod +x /usr/local/bin/stamp &&
travis_retry curl -f http://releases.llvm.org/6.0.0/clang+llvm-6.0.0-x86_64-apple-darwin.tar.xz | tar xJf - &&
export CC=`pwd`/clang+llvm-6.0.0-x86_64-apple-darwin/bin/clang &&
export CXX=`pwd`/clang+llvm-6.0.0-x86_64-apple-darwin/bin/clang++ &&
export AR=ar
;;
2017-09-16 06:28:08 -05:00
esac
before_script:
- >
echo "#### Disk usage before running script:";
df -h;
du . | sort -nr | head -n100
- >
RUN_SCRIPT="src/ci/init_repo.sh . $HOME/rustsrc";
if [ "$TRAVIS_OS_NAME" = "osx" ]; then
export RUN_SCRIPT="$RUN_SCRIPT && src/ci/run.sh";
else
export RUN_SCRIPT="$RUN_SCRIPT && src/ci/docker/run.sh $IMAGE";
fi
# Log time information from this machine and an external machine for insight into possible
# clock drift. Timezones don't matter since relative deltas give all the necessary info.
script:
- >
date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
2017-06-28 16:54:05 -05:00
- stamp sh -x -c "$RUN_SCRIPT"
- >
date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
after_success:
- >
echo "#### Build successful; Disk usage after running script:";
df -h;
du . | sort -nr | head -n100
after_failure:
- >
echo "#### Build failed; Disk usage after running script:";
df -h;
du . | sort -nr | head -n100
# Random attempt at debugging currently. Just poking around in here to see if
# anything shows up.
- ls -lat $HOME/Library/Logs/DiagnosticReports/
- find $HOME/Library/Logs/DiagnosticReports
-type f
-name '*.crash'
-not -name '*.stage2-*.crash'
-not -name 'com.apple.CoreSimulator.CoreSimulatorService-*.crash'
-exec printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" {} \;
-exec head -750 {} \;
-exec echo travis_fold":"end:crashlog \; || true
# attempt to debug anything killed by the oom killer on linux, just to see if
# it happened
- dmesg | grep -i kill
notifications:
email: false
before_deploy:
- mkdir -p deploy/$TRAVIS_COMMIT
- >
if [ "$TRAVIS_OS_NAME" == "osx" ]; then
rm -rf build/dist/doc &&
cp -r build/dist/* deploy/$TRAVIS_COMMIT;
else
rm -rf obj/build/dist/doc &&
cp -r obj/build/dist/* deploy/$TRAVIS_COMMIT;
fi
- travis_retry gem update --system
- ls -la deploy/$TRAVIS_COMMIT
deploy:
- provider: s3
bucket: rust-lang-ci2
skip_cleanup: true
local_dir: deploy
upload_dir: rustc-builds
acl: public_read
region: us-west-1
access_key_id: AKIAJVBODR3IA4O72THQ
secret_access_key:
secure: "kUGd3t7JcVWFESgIlzvsM8viZgCA9Encs3creW0xLJaLSeI1iVjlJK4h/2/nO6y224AFrh/GUfsNr4/4AlxPuYb8OU5oC5Lv+Ff2JiRDYtuNpyQSKAQp+bRYytWMtrmhja91h118Mbm90cUfcLPwkdiINgJNTXhPKg5Cqu3VYn0="
on:
branch: auto
condition: $DEPLOY = 1
# this is the same as the above deployment provider except that it uploads to
# a slightly different directory and has a different trigger
- provider: s3
bucket: rust-lang-ci2
skip_cleanup: true
local_dir: deploy
upload_dir: rustc-builds-alt
acl: public_read
region: us-west-1
access_key_id: AKIAJVBODR3IA4O72THQ
secret_access_key:
secure: "kUGd3t7JcVWFESgIlzvsM8viZgCA9Encs3creW0xLJaLSeI1iVjlJK4h/2/nO6y224AFrh/GUfsNr4/4AlxPuYb8OU5oC5Lv+Ff2JiRDYtuNpyQSKAQp+bRYytWMtrmhja91h118Mbm90cUfcLPwkdiINgJNTXhPKg5Cqu3VYn0="
on:
branch: auto
condition: $DEPLOY_ALT = 1
# These two providers are the same as the two above, except deploy on the
# try branch. Travis does not appear to provide a way to use "or" in these
# conditions.
- provider: s3
bucket: rust-lang-ci2
skip_cleanup: true
local_dir: deploy
upload_dir: rustc-builds
acl: public_read
region: us-west-1
access_key_id: AKIAJVBODR3IA4O72THQ
secret_access_key:
secure: "kUGd3t7JcVWFESgIlzvsM8viZgCA9Encs3creW0xLJaLSeI1iVjlJK4h/2/nO6y224AFrh/GUfsNr4/4AlxPuYb8OU5oC5Lv+Ff2JiRDYtuNpyQSKAQp+bRYytWMtrmhja91h118Mbm90cUfcLPwkdiINgJNTXhPKg5Cqu3VYn0="
on:
branch: try
condition: $DEPLOY = 1
- provider: s3
bucket: rust-lang-ci2
skip_cleanup: true
local_dir: deploy
upload_dir: rustc-builds-alt
acl: public_read
region: us-west-1
access_key_id: AKIAJVBODR3IA4O72THQ
secret_access_key:
secure: "kUGd3t7JcVWFESgIlzvsM8viZgCA9Encs3creW0xLJaLSeI1iVjlJK4h/2/nO6y224AFrh/GUfsNr4/4AlxPuYb8OU5oC5Lv+Ff2JiRDYtuNpyQSKAQp+bRYytWMtrmhja91h118Mbm90cUfcLPwkdiINgJNTXhPKg5Cqu3VYn0="
on:
branch: try
condition: $DEPLOY_ALT = 1