This fixes multiple network issues we had when downloading awscli from
PyPI on Azure Pipelines by vendoring awscli itself and its dependencies
in our S3 bucket. Instructions on how to update the cache are present at
the top of src/ci/install-awscli.sh
ci: debug network failures while downloading awscli from PyPI
This adds some random debug code to our CI script while downloading awscli, to *hopefully* pinpoint what's causing the network failures.
r? @Mark-Simulacrum
cc #62967
ci: Install clang on Windows through tarballs
Previously we used the executables built the LLVM project but these
executables are difficult to run in a CI environment, they can
accidentally pollute global state, etc. In testing some of the possible
4-core machine environments for Azure this step would frequently cause
issues.
To assuage these future issues and hopefully make builds slightly more
self-contained, this commit changes to install from a tarball instead.
The tarball isn't provided by LLVM itself, but we use the offical LLVM
installer to extract itself and then we pack up the LLVM installation
directory into the tarball.
docutils 0.15, a dependency of awscli, broke our CI since it's not
compatible with Python 2 due to a bug. This pins all the dependencies of
awscli with docutils 0.14, to make sure this kind of regressions doesn't
happen again.
This commit updates some of our assorted Azure/CI configuration to
prepare for some 4-core machines coming online. We're still in the
process of performance testing them to get final numbers, but some
changes are worth landing ahead of this. The updates here are:
* Use `C:/` instead of `D:/` for submodule checkout since it should have
plenty of space and the 4-core machines won't have `D:/`
* Update `lzma-sys` to 0.1.14 which has support for VS2019, where 0.1.10
doesn't.
* Update `src/ci/docker/run.sh` to work when it itself is running inside
of a docker container (see the comment in the file for more info)
* Print step timings on the `try` branch in addition to the `auto`
branch in. The logs there should be seen by similarly many humans (not
many) and can be useful for performance analysis after a `try` build
runs.
* Install the WIX and InnoSetup tools manually on Windows instead of
relying on pre-installed copies on the VM. This gives us more control
over what's being used on the Azure cloud right now (we control the
version) and in the 4-core machines these won't be pre-installed. Note
that on AppVeyor we actually already were installing InnoSetup, we
just didn't carry that over on Azure!
Previously we used the executables built the LLVM project but these
executables are difficult to run in a CI environment, they can
accidentally pollute global state, etc. In testing some of the possible
4-core machine environments for Azure this step would frequently cause
issues.
To assuage these future issues and hopefully make builds slightly more
self-contained, this commit changes to install from a tarball instead.
The tarball isn't provided by LLVM itself, but we use the offical LLVM
installer to extract itself and then we pack up the LLVM installation
directory into the tarball.
Update to LLVM 9 trunk
Following the preparatory changes in #62474, this updates the LLVM submodule to https://github.com/rust-lang/llvm-project/tree/rustc/9.0-2019-07-12 and:
* Changes the LLVM Rust bindings to account for the new SubtargetSubTypeKV.
* Adjusts a codegen test for the new form of the byval attribute that takes a type.
* Makes a PGO codegen test more liberal with regard to order and linkage.
* Builds InstrProfilingPlatformWindows.c as part of libprofiler_builtins.
* Moves registration of additional passes (in particular sanitizers) to the end of the module pass manager.
* Disables LLDB on builders.
r? @alexcrichton
ci: add a pr builder to test tools when submodules are updated
This PR adds the x86_64-gnu-tools builders to PRs where submodules are updated.
Since it's not possible to *start* the builder only when submodule changes are detected, I opted into adding a "decider" task at the start of the job which sets the `SKIP_JOB` environment variable when submodules are not updated, and I gated the most time-consuming tasks (the actual build and artifacts upload) on the variable not being there. All of this is conditionally included in the `steps/run.yml` only when a template parameter is present, so it should only affect that builder on PRs.
The cost for this should be a dummy builder running for 2/3 minutes for each PR, and we should be able to handle it.
Fixes https://github.com/rust-lang/rust/issues/61837
r? @alexcrichton
During installation of mingw, at least, the git directories change, so
we need to reset the core.autocrlf config to false.
Once we finish checking out submodules, check that the line endings are
\n and not \r\n.
The Azure image enables CRLF conversion on Windows builders, but that
caused regressions both in our test suite (the miri test suite broke)
and in the ecosystem, since we started shipping install scripts with
CRLF endings instead of the old LF. The Godbolt Compiler Explorer is one
such case of breakage.
This adds a step to the build explicitly disabling the conversion before
the repository is checked out.
This commit disables all builders on Travis and almost all builders on
AppVeyor now that they're all running on Azure Pipelines. There is one
remaining builder on AppVeyor which hasn't been migrated yet due to a
test failure on Azure, which we'll be debugging soon. One remaining
builder is left on Travis which is the tools builder whenever a
submodule is changed, but we'll probably turn that off soon since it's
just for PRs.
The other major change in this PR is that the auto builders on Azure are
now configured with "real" prod credentials which should cause them to
publish all artifacts into the official CI buckets.
ci: Sync AppVeyor/Travis with Azure configuration
Manually make sure that we do the same thing across all the services,
uncovering one spot where we needed to pass one more configure flag on
Azure but otherwise we're good to go!
ci: Publish toolstate changes from Azure
This commit moves toolstate publishing from Travis to Azure. We've been
testing on azure for some time now and this works by deleting the Travis
config and updating the credentials used on Azure.
Manually make sure that we do the same thing across all the services,
uncovering one spot where we needed to pass one more configure flag on
Azure but otherwise we're good to go!
This commit moves toolstate publishing from Travis to Azure. We've been
testing on azure for some time now and this works by deleting the Travis
config and updating the credentials used on Azure.
This commit switches the `try` builers to officially happen on Azure
Pipelines instead of Travis where they're currently run. This also cuts
back the number of builders to just the two we run on Travis, leaving
expansion as a possible future extension.
Currently just run it through its paces but don't actually push to
official locations. Instead let's just push to a separate fork (mine) as
well as open issues in a separate fork (mine). Make sure that people
aren't pinged for these issues as well!
This should hopefully ensure that everything is working on Azure and
give us a chance to work through any issues that come up.
ci: Collect CPU usage statistics on Azure
This commit adds a script which we'll execute on Azure Pipelines which
is intended to run in the background and passively collect CPU usage
statistics for our builders. The intention here is that we can use this
information over time to diagnose issues with builders, see where we can
optimize our build, fix parallelism issues, etc. This might not end up
being too useful in the long run but it's data we've wanted to collect
for quite some time now, so here's a stab at it!
Comments about how this is intended to work can be found in the python
script used here to collect CPU usage statistics.
Closes#48828
This commit adds a script which we'll execute on Azure Pipelines which
is intended to run in the background and passively collect CPU usage
statistics for our builders. The intention here is that we can use this
information over time to diagnose issues with builders, see where we can
optimize our build, fix parallelism issues, etc. This might not end up
being too useful in the long run but it's data we've wanted to collect
for quite some time now, so here's a stab at it!
Comments about how this is intended to work can be found in the python
script used here to collect CPU usage statistics.
Closes#48828
Looks like Azure has updated images recently to install Rust by default,
but that can interfere with our own compiler (for example Cargo's test
suite we think) so be sure to uninstall it before proceeding.
Speed up Azure CI installing Windows dependencies
There is known issue where PowerShell is unreasonably slow downloading
files due to an issue with rendering the progress bar, see this [issue](https://github.com/PowerShell/PowerShell/issues/2138)
That issue is fixed in PowerShell Core (available in Azure Pipelines as
pwsh.exe) but it can also be worked around by setting:
$ProgressPreference = 'SilentlyContinue'
I measured downloading LLVM and it took about 220s before, 5s after, so the improvement is significant.
ci: Favor SCRIPT instead of RUST_CHECK_TARGET
Since #61212 we've been timing out on OSX, and this looks to be because
we're building tools like Cargo and the RLS twice instead of once. This
turns out to be a slight bug in our configuration. CI builders using the
`RUST_CHECK_TARGET` directive actually execute `make all` just before
their acual target. In `make all` we're building a stage2 cargo, and
then in `make dist` we're building a stage1 cargo.
Other builders use `SCRIPT` which provides explicit control over what
`x.py` script, for example, is used to execute the build. This moves
almost all targets to using `SCRIPT` to ensure that we're explicitly
specifying what's being built where. Additionally this updates the logic
of `RUST_CHECK_TARGET` to remove the pre-flight tidy as well as the
pre-flight `make all`. The system LLVM builder (run on PRs) now
explicitly runs tidy first and then runs the rest of the test suite.
Since #61212 we've been timing out on OSX, and this looks to be because
we're building tools like Cargo and the RLS twice instead of once. This
turns out to be a slight bug in our configuration. CI builders using the
`RUST_CHECK_TARGET` directive actually execute `make all` just before
their acual target. In `make all` we're building a stage2 cargo, and
then in `make dist` we're building a stage1 cargo.
Other builders use `SCRIPT` which provides explicit control over what
`x.py` script, for example, is used to execute the build. This moves
almost all targets to using `SCRIPT` to ensure that we're explicitly
specifying what's being built where. Additionally this updates the logic
of `RUST_CHECK_TARGET` to remove the pre-flight tidy as well as the
pre-flight `make all`. The system LLVM builder (run on PRs) now
explicitly runs tidy first and then runs the rest of the test suite.
There is known issue where PowerShell is unreasonably slow downloading
files due to an issue with rendering the progress bar, see:
https://github.com/PowerShell/PowerShell/issues/2138
That issue is fixed in PowerShell Core (available in Azure Pipelines as
pwsh.exe) but it can also be worked around by setting:
$ProgressPreference = 'SilentlyContinue'
We were checking out the rustc repo multiple times on auto macOS, and
that was causing an error on the Azure side since multiple checkouts are
apparently not supported. This removes the extra checkout.
* Clean up the `install-windows-build-deps.yml` file and add some more
comments where appropriate.
* Add some comments to `run.yml`
* Don't fast path the `rustfmt` submodule, but we'll take care of that
later if necessary.
