Move BOLT from `bootstrap` to `opt-dist`
Currently, we use BOLT to optimize LLVM for x64 Linux. The BOLT instrumentation and optimization step is implemented in `bootstrap`, but it was always quite hacky, because BOLT works quite differently than PGO. Rather than building an instrumented artifact, it takes an already built artifact and instruments it in-place. This is not a good fit for the bootstrap caching mechanism, and it meant that we had to run BOLT "on-the-fly" when packaging LLVM artifacts into the created sysroot.
The BOLT code was also really only used by the PGO script (now called `opt-dist`) and nothing else, so it was quite hardcoded for this one single usage. And even if someone wanted to use the `--llvm-bolt-profile-[use/generate]` bootstrap flags outside of the PGO script, they would also need to implement profile gathering, as this is not performed by bootstrap anyway.
I think that it could be more practical to move the BOLT logic to the `opt-dist` tool instead. This simplifies bootstrap, removes unneeded implementation of BOLT caching (we will now do it exactly once - no need to check if it has been performed already when bootstrap copies `libLLVM.so` around multiple times) and removes two BOLT-specific bootstrap flags, and also one special case for building LLVM (instead I pass the linker flags with `--set llvm.ldflags` from `opt-dist` now).
There are also a few disadvantages to this new approach:
- We have to guess/find the path to the built `libLLVM.so` file (but currently this is quite easy, it's just in `stage2/lib`).
- We have to provide the BOLT profile externally to bootstrap, so that it is packaged into the reproducible artifacts archive. Doesn't seem like a big deal to me.
- We are depending on some inner behavior of boostrap in `opt-dist` (namely, that `libLLVM.so` is hardlinked). But we do that for many other things in the `opt-dist` tool anyway, it's tied quite closely to bootstrap.
I would like to go back to my attempts to also use BOLT for `librustc_driver.so`, and I think that it might be a bit simpler if I also do it from the `opt-dist` tool, so this is the first step towards that.
Anyway, let me know what you think about this. It's just a refactoring in a way, no big deal.
r? bootstrap
fix: Expand eager macros to delimited comma separated expression list
Prior to this, we were just parsing it as an expression which works fine for `()` and `[]` calls as those are tuple and array expressions respectively, but if tails for `{}` calls which with my recent changes reported errors for such eager macro invocations.
Rewrite miri script in rust
This is a sketch of a rewrite of miri script in Rust. It does not include changes made in https://github.com/rust-lang/miri/pull/2908 yet. Environment variables are not properly propagated yet, which is something I plan to address.
This PR is mostly a heads-up about the ongoing effort and it's state.
It's definitely not the cleanest code I've seen in my life, but my first goal was feature/interface parity. I will iterate on it a bit before marking it as ready.
I wonder though how this should be integrated/tested. Are you aware of anyone using `./miri` in their scripts?
I guess we should keep existing `miri` script in place and let it run miri-script package directly?
CI should probably `cargo check` this package as well.
Fixes#2883
Only golden arches
A number of tests in the test suite have applied the somewhat comedic practice of ignoring *every* single target architecture that rustc has ever supported. This is silly, when they are clearly tests built around certain assumptions, primarily of the x86-64 architecture, or in one case when they are only relevant for a handful of 32-bit targets. This has even resulted, in one case, in the same architecture being ignored twice!
Document these better, and use a "revision + only-arch" idiom in the test headers to denote the "golden arches" that actually pass these tests.
This function has some shared code for the thin LTO and fat LTO cases,
but those cases have so little in common that it's actually clearer to
treat them fully separately.
PR #112946 tweaked the naming of LLVM threads, but messed things up
slightly, resulting in threads on Windows having names like `optimize
module {} regex.f10ba03eb5ec7975-cgu.0`.
This commit removes the extraneous `{} `.
The main loop has a *very* complex condition, which includes two
mentions of `codegen_state`. The body of the loop then immediately
switches on the `codegen_state`.
I find it easier to understand if it's a `loop` and we check for exit
conditions after switching on `codegen_state`. We end up with a tiny bit
of code duplication, but it's clear that (a) we never exit in the
`Ongoing` case, (b) we exit in the `Completed` state only if several
things are true (and there's interaction with LTO there), and (c) we
exit in the `Aborted` state if a couple of things are true. Also, the
exit conditions are all simple conjunctions.
This loop condition involves `codegen_state`, `work_items`, and
`running_with_own_token`. But the body of the loop cannot modify
`codegen_state`, so repeatedly checking it is unnecessary.
`CodegenContext` is immutable except for the `worker` field - we clone
`CodegenContext` in multiple places, changing the `worker` field each
time. It's simpler to move the `worker` field out of `CodegenContext`.
Because it's usefulness wasn't clear to me, and I initially wondered if
it could be removed. The text is based on the text in #50972, the PR
that added the flag.
