CI: use smaller machines in PR runs
mingw-check job-linux-16c -> job-linux-4c
~job-linux-4c 20 min in auto job
~job-linux-16c 13 min in pr job
with current pr regressed to almost 21 min, it's ok.
mingw-check-tidy job-linux-16c -> job-linux-4c small enough, so reduce to minimal
~ job-linux-16c 3 min
with current pr regressed to almost 5 min, it's ok.
x86_64-gnu-tools job-linux-16c this is top job by time in PR, so don't touch it
~ job-linux-8c 1.30 hour in auto job
~ job-linux-16c 1 hour in pr job (affected by #114613, actual time ~ 30 min)
x86_64-gnu-llvm-15 job-linux-16c don't change too
~ job-linux-8c 1.30 hour in auto job
~ job-linux-16c 30 min in pr job
Noticed while working on https://github.com/rust-lang/rust/pull/114621, so current time affected by always rebuilded docker images (but pr images always rebuilded before too, so nvm)
CI: include workflow name in concurrency group
Currently, this won't change anything, because we only have one relevant workflow (`CI`), but for future proofing we should probably include the workflow name in the concurrency group.
Found by ``@klensy`` [here](https://github.com/rust-lang/rust/pull/113059#discussion_r1247213606).
Currently, this won't change anything, because we only have one relevant workflow (`CI`), but for future proofing we should probably include the workflow name in the concurrency group.
Remove aws cli install.
All runner images have the AWS CLI 2 installed, so there isn't a really strong reason to install our own version anymore.
The version we were installing was 1.27.122. The runner images currently have 2.11.x (the exact version varies by image).
I do not have the means to really test if the new version has any issues. I looked at all the `aws` commands, and none of them seem to be doing anything unusual. The page at https://docs.aws.amazon.com/cli/latest/userguide/cliv2-migration-changes.html contains a list of all the breaking changes, and I didn't see anything that looked important.
Optimize builder sizes
The infra-team is continuously monitoring the efficiency of the CI system in an effort to improve overall build times and resource usage. Some builders have used much less than their allocated resources, so we are testing smaller builder sizes for them.
r? `@pietroalbini`
The infra-team is continuously monitoring the efficiency of the build
system in an effort to improve overall build times and resource usage.
The builders for some of the `x86_64-gnu` targets have used much less
resources than allocated in the past, so we are testing a smaller
builder size for them.
The infra-team is continuously monitoring the efficiency of the build
system in an effort to improve overall build times and resource usage.
The builder for the `mingw-check` target have used much less resources
than allocated in the past, so we are testing a smaller builder size for
it.
The infra-team is continuously monitoring the efficiency of the build
system in an effort to improve overall build times and resource usage.
The builders for the `i686-gnu` targets have used much less resources
than allocated in the past, so we are testing a smaller builder size for
them.
Like #107044, this will let us track compatibility with LLVM 16 going
forward, especially after we eventually upgrade our own to the next.
This also drops `tidy` here and in `x86_64-gnu-llvm-15`, syncing with
that change in #106085.
These builders aren't particularly high on overall average CPU usage and finish in typically around
30 minutes. Cutting their core counts will hopefully not significantly increase wall-time while
cutting costs, allowing us to shift some of the wins into our slower builders.
For now this keeps all the configuration identical (AFAICT) but we'll
likely want to play with the specifics to move some of the slower
builders to larger machines and the faster builders to smaller machines,
likely reducing overall usage and improving CI times.
Previously, it would only run on changes to subtrees, submodules, or select directories.
That made it so that changes to the compiler that broke tools would only be detected on a full bors merge.
This makes it so the tools builder runs by default, making it easier to catch breaking changes to clippy (which was the most effected).
Enable Cargo's sparse protocol in CI
This enables the sparse protocol in CI in order to exercise and dogfood it. This is intended test the production server in a real-world situation.
Closes#107342
Port pgo.sh to Python
This PR ports the `pgo.sh` multi stage build file from bash to Python, to make it easier to add new functionality and gather statistics. Main changes:
1) `pgo.sh` rewritten from Bash to Python. Jump from ~200 Bash LOC to ~650 Python LOC. Bash is, unsurprisingly, more concise for running scripts and binaries.
2) Better logging. Each separate stage is now clearly separated in logs, and the logs can be quickly grepped to find out which stage has completed or failed, and how long it took.
3) Better statistics. At the end of the run, there is now a table that shows the duration of the individual stages, along with a percentual ratio of the total workflow run:
```
2023-01-15T18:13:49.9896916Z stage-build INFO: Timer results
2023-01-15T18:13:49.9902185Z ---------------------------------------------------------
2023-01-15T18:13:49.9902605Z Build rustc (LLVM PGO): 1815.67s (21.47%)
2023-01-15T18:13:49.9902949Z Gather profiles (LLVM PGO): 418.73s ( 4.95%)
2023-01-15T18:13:49.9903269Z Build rustc (rustc PGO): 584.46s ( 6.91%)
2023-01-15T18:13:49.9903835Z Gather profiles (rustc PGO): 806.32s ( 9.53%)
2023-01-15T18:13:49.9904154Z Build rustc (LLVM BOLT): 1662.92s (19.66%)
2023-01-15T18:13:49.9904464Z Gather profiles (LLVM BOLT): 715.18s ( 8.46%)
2023-01-15T18:13:49.9914463Z Final build: 2454.00s (29.02%)
2023-01-15T18:13:49.9914798Z Total duration: 8457.27s
2023-01-15T18:13:49.9915305Z ---------------------------------------------------------
```
A sample run can be seen [here](https://github.com/rust-lang/rust/actions/runs/3923980164/jobs/6707932029).
I tried to keep the code compatible with Python 3.6 and don't use dependencies, which required me to reimplement some small pieces of functionality (like formatting bytes). I suppose that it shouldn't be so hard to upgrade to a newer Python or install dependencies in the CI container, but I'd like to avoid it if it won't be needed.
The code is in a single file `stage-build.py`, so it's a bit cluttered. I can also separate it into multiple files, although having it in a single file has some benefits. The code could definitely be nicer, but I'm a bit wary of introducing a lot of abstraction and similar stuff, as long as the code is stuffed into a single file.
Currently, the Python pipeline should faithfully mirror the bash pipeline one by one. After this PR, I'd like to try to optimize it, e.g. by caching the LLVM builds on S3.
r? `@Mark-Simulacrum`
