Seems like the variable showed by $(ciCheckoutPath) on Azure Pipelines
was wrong, making the toolstate script fail. This commit changes that
function to return the variable previously used by the toolstate script.
Other uses of the function were audited, and there should be no
conflict.
GitHub Actions: preparations, part 2
This PR adds the second batch of commits in preparation for GitHub Actions:
* Removed hardcoded Azure Pipelines variables from `publish_toolstate.sh`
* Fixed a bug in `shared.sh`'s GitHub Actions support
* Fixed binutils missing from MSYS2 on Windows 2019 (GitHub Actions-specific)
* Fixed wrong sysroot in macOS 10.15 onwards (GitHub Actions-specific)
This PR does **not** yet add any builders on GitHub Actions.
r? @alexcrichton
ci: remove 32-bit Apple targets
This PR drops the `i686-apple` and `dist-i686-apple` CI builders, as well as removing the `armv7-apple-ios`, `armv7s-apple-ios` and `i386-apple-ios` targets from the `x86_64-apple` CI builder.
The change was approved in [RFC 2837](https://github.com/rust-lang/rfcs/pull/2837), and it should land in Rust 1.42 stable (so this cycle).
r? @alexcrichton
In their infinite wisdom, Apple decided that (starting from macOS 10.15
onwards) /usr/include is not the location we should all search in for
our beloved C headers. Instead, we should look inside the extremely
intuitive and easily guessable new path:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include
Because why not.
When moving the script out of CI configuration and into a proper script
we lost track of the current directory changing (and as such the
parameters of the script needing to be different now).
Toolstate publication only runs if the channel is "nightly" and
previously the toolstate builders did not know that the channel was
nightly (since they are not dist builders).
A look through bootstrap seems to indicate that nothing should directly
depend on the channel being set to `-dev` on the test builders, though
this may cause some problems with UI tests (if for some reason they're
dumping the channel into stderr), but we cannot find evidence of such so
hopefully this is fine.
I spent a while debugging a strage linker error about an outdated `glibc` version, only to discover that it was caused by a stale `obj` directory. It wasn't obviously to be that using the same obj dir with multiple Docker images (for the same target triple) could be a problem.
This commit adds a note to the README, which should hopefully be helpful to anyone else who runs into this issue.
Update the revision of wasi-libc used in wasm32-wasi
This commit updates the `wasi-libc` repository used to build the
wasm32-wasi target's libstd to ensure that both libstd and libc are
using the same wasi snapshot version.
[CI] fix the `! isCI` check in src/ci/run.sh
Using `if [ ! isCI ] || ...` doesn't run any command, just tests `isCI`
as a string, whereas `if ! isCI || ...` will actually run the `isCI`
command and negate its exit status.
This commit updates the `wasi-libc` repository used to build the
wasm32-wasi target's libstd to ensure that both libstd and libc are
using the same wasi snapshot version.
Using `if [ ! isCI ] || ...` doesn't run any command, just tests `isCI`
as a string, whereas `if ! isCI || ...` will actually run the `isCI`
command and negate its exit status.
LLVM 7 is over a year old, which should be plenty for compatibility. The
last LLVM 6 holdout was llvm-emscripten, which went away in #65501.
I've also included a fix for LLVM 8 lacking `MemorySanitizerOptions`,
which was broken by #66522.
Some environment variables (like DEPLOY or DEPLOY_ALT for dist builders,
or IMAGE on Linux builders) are set on a lot of builders, and whether
they should be present or not can be detected automatically based on the
builder name and the platform.
This commit simplifies the CI configuration by automatically setting
those environment variables.
Misc CI improvements
This PR contains some misc improvements to our CI configuration:
* The environment variables for MinGW builders were greatly simplified, with just `CUSTOM_MINGW=1` to tell the install scripts to install the vendored copy. All the others (`MINGW_URL`, `MINGW_DIR`, `MINGW_ARCHIVE` and `MSYS_BITS`) are detected either from the builder name or the environment.
* Collecting CPU stats and running the build were moved into scripts.
* Toolstate scripts validation was previously a separate step, ran just when `IMAGE=mingw-check`. This moves the validation code inside the actual image.
* Vendored copies are now fetched from https://ci-mirrors.rust-lang.org instead of directly from the bucket.
r? @alexcrichton
CentOS 5 only supports SSLv3 without SNI, and to get newer protocols
working we need to download and compile OpenSSL and cURL from our
mirror. Because of that, we can't use the CDN, as CloudFront requires
TLSv1 with SNI.
This commit changes the dist-x86_64-linux image to bypass the CDN for
OpenSSL and cURL.
This commit replaces the mirrors base URL contained in the MINGW_URL
with a CUSTOM_MINGW=1 environment variable. The mirrors base URL will be
fetched instead through the MIRRORS_BASE environment variable, defined
in src/ci/shared.sh.
Currently the `RUST_CONFIGURE_ARGS` variable apparently has a trailing
newline at the end of it due to the way it's configured in yaml. This
causes issues with MSVC's `install-clang.sh` step where the way the bash
syntax works out means that we drop the arg we're trying to add and it
doesn't actually get added!
The hopeful fix here is to tweak how we specify the yaml syntax to not
have a trailing newline, we'll see what CI says about this...
This updates the libc that the `wasm32-wasi` target links against to the
latest revision, mostly just bringing in minor bug fixes and minor wasm
size improvements.
ci: revert msys2 ca-certificates hack
The hack was added because upstream msys2 broke the ca-certificates package, but since then it has been fixed. This reverts CI to use the upstream package.
Part of #65767
Upload toolstates.json to rust-lang-ci2
This PR does two things:
* Following up with https://github.com/rust-lang/rust/pull/65202, it migrates deploying artifacts to CI in a script. Both uploading release artifacts and CPU stats were merged into the same script, designing it to be easily extended.
* Uploads the toolstate JSON to `rust-lang-ci2` along with the release artifacts, both for Linux and Windows. This is needed because @RalfJung wants to stop shipping MIRI when its tests are failing, and the toolstate repo doesn't have entries for each commit. Having the toolstate data (just for that specific commit) on `rust-lang-ci2` will simplify the code a lot.
r? @alexcrichton
cc @RalfJung
Before this commit toolstates.json was stored in /tmp and it wasn't
mounted outside the build container. That caused uploading the file in
the upload-artifacts task to fail, as the file was missing on the host.
Mounting /tmp/toolstates.json alone is not the best approach: if the
file is missing when the container is started the Docker engine will
create a *directory* named /tmp/toolstates.json.
The Docker issue could be solved by pre-creating an empty file named
/tmp/toolstates.json, but doing that could cause problems if bootstrap
fails to generate the file and the toolstate scripts receive an empty
JSON.
The approach I took in this commit is to instead mount a /tmp/toolstate
directory inside Docker, and create the toolstates.json file in it. That
also required a small bootstrap change to ensure the directory is
created if it's missing.
The hack was added because upstream msys2 broke the ca-certificates
package, but since then it has been fixed. This reverts CI to use the
upstream package.
Uploading the toolstate data for each commit will help our release
tooling understand which components are failing, to possibly skip
shipping broken tools to users.
