Make "Assemble stage1 compiler" orders of magnitude faster (take 2)
This used to take upwards of 5 seconds for me locally. I found that the culprit was copying the downloaded LLVM shared object:
```
[22:28:03] Install "/home/jnelson/rust-lang/rust/build/x86_64-unknown-linux-gnu/ci-llvm/lib/libLLVM-14-rust-1.62.0-nightly.so" to "/home/jnelson/rust-lang/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libLLVM-14-rust-1.62.0-nightly.so"
[22:28:09] c Sysroot { compiler: Compiler { stage: 1, host: x86_64-unknown-linux-gnu(x86_64-unknown-linux-gnu) } }
```
It turned out that `install()` used full copies unconditionally. Change it to try using a hard-link before falling back to copying.
- Panic if we generate a symbolic link in a tarball
- Change install to use copy internally, like in my previous PR
- Change copy to dereference symbolic links, which avoids the previous regression in #96803.
I also took the liberty of fixing `x dist llvm-tools` to work even if you don't call `x build` previously.
add comments in `store_dead_field_or_variant`
support multiple log level
add a item ident label
fix ui tests
fix a ui test
fix a rustdoc ui test
use let chain
refactor: remove `store_dead_field_or_variant`
fix a tiny bug
4 commits in 4d92f07f34ba7fb7d7f207564942508f46c225d3..8d42b0e8794ce3787c9f7d6d88b02ae80ebe8d19
2022-06-10 01:11:04 +0000 to 2022-06-17 16:46:26 +0000
- Use specific terminology for sparse HTTP-based registry (rust-lang/cargo#10764)
- chore: Upgrade to clap 3.2 (rust-lang/cargo#10753)
- Improve testing framework for http registries (rust-lang/cargo#10738)
- doc: Improve example of using the links field (rust-lang/cargo#10728)
This adds the typeid and `vcall_visibility` metadata to vtables when the
-Cvirtual-function-elimination flag is set.
The typeid is generated in the same way as for the
`llvm.type.checked.load` intrinsic from the trait_ref.
The offset that is added to the typeid is always 0. This is because LLVM
assumes that vtables are constructed according to the definition in the
Itanium ABI. This includes an "address point" of the vtable. In C++ this
is the offset in the vtable where information for RTTI is placed. Since
there is no RTTI information in Rust's vtables, this "address point" is
always 0. This "address point" in combination with the offset passed to
the `llvm.type.checked.load` intrinsic determines the final function
that should be loaded from the vtable in the
`WholeProgramDevirtualization` pass in LLVM. That's why the
`llvm.type.checked.load` intrinsics are generated with the typeid of the
trait, rather than with that of the function that is called. This
matches what `clang` does for C++.
The vcall_visibility metadata depends on three factors:
1. LTO level: Currently this is always fat LTO, because LLVM only
supports this optimization with fat LTO.
2. Visibility of the trait: If the trait is publicly visible, VFE
can only act on its vtables after linking.
3. Number of CGUs: if there is more than one CGU, also vtables with
restricted visibility could be seen outside of the CGU, so VFE can
only act on them after linking.
To reflect this, there are three visibility levels: Public, LinkageUnit,
and TranslationUnit.
Update minifier version to 0.2.1
This change and these changes come from an idea of `@camelid:` instead of creating a string, we just `write` the type into the file directly.
I don't think it'll have a big impact on perf but it's still a potential small improvement.
r? `@notriddle`
Update cargo
7 commits in 38472bc19f2f76e245eba54a6e97ee6821b3c1db..85e457e158db216a2938d51bc3b617a5a7fe6015
2022-05-31 02:03:24 +0000 to 2022-06-07 21:57:52 +0000
- Make -Z http-registry use index.crates.io when accessing crates-io (rust-lang/cargo#10725)
- Respect submodule update=none strategy in .gitmodules (rust-lang/cargo#10717)
- Expose rust-version through env var (rust-lang/cargo#10713)
- add validation for string "true"/"false" in lto profile (rust-lang/cargo#10676)
- Enhance documentation of testing (rust-lang/cargo#10726)
- Clear disk space on CI. (rust-lang/cargo#10724)
- Enforce to use tar v0.4.38 (rust-lang/cargo#10720)
Add build metrics to rustbuild
This PR adds a new module of rustbuild, `ci_profiler`, whose job is to gather as much information as possible about the CI build as possible and store it in a JSON file uploaded to `ci-artifacts`. Right now for each step it collects:
* Type name and debug representation of the `Step` object.
* Duration of the step (excluding child steps).
* Systemwide CPU stats for the duration of the step (both single core and all cores).
* Which child steps were executed.
This is capable of replacing both the scripts to collect CPU stats and the `[TIMING]` lines in build logs (not yet removed, until we port our tooling to use the CI profiler). The format is also extensible to be able in the future to collect more information.
r? `@Mark-Simulacrum`
library/std: Bump compiler_builtins
Some neat changes include faster float conversions & fixes for AVR 🙂
(note that's it's my first time upgrading `compiler_builtins`, so I'm not 100% sure if bumping `library/std/Cargo.toml` is enough; certainly seems to be so, though.)
Update to rebased rustc-rayon 0.4
In rayon-rs/rayon#938, miri uncovered a race in `rustc-rayon-core` that had already been fixed in the regular `rayon-core`. I have now rebased that fork onto the latest rayon branch, and published as 0.4. I also updated `indexmap` to bump the dependency.
`Cargo.lock` changes:
Updating indexmap v1.8.0 -> v1.8.2
Updating rayon v1.5.1 -> v1.5.3
Updating rayon-core v1.9.1 -> v1.9.3
Updating rustc-rayon v0.3.2 -> v0.4.0
Updating rustc-rayon-core v0.3.2 -> v0.4.1
macros: introduce `fluent_messages` macro
Adds a new `fluent_messages` macro which performs compile-time validation of the compiler's Fluent resources (i.e. that the resources parse and don't multiply define the same messages) and generates constants that make using those messages in diagnostics more ergonomic.
For example, given the following invocation of the macro..
```rust
fluent_messages! {
typeck => "./typeck.ftl",
}
```
..where `typeck.ftl` has the following contents..
```fluent
typeck-field-multiply-specified-in-initializer =
field `{$ident}` specified more than once
.label = used more than once
.label-previous-use = first use of `{$ident}`
```
...then the macro parse the Fluent resource, emitting a diagnostic if it fails to do so...
```text
error: could not parse Fluent resource
--> $DIR/test.rs:35:28
|
LL | missing_message => "./missing-message.ftl",
| ^^^^^^^^^^^^^^^^^^^^^^^
|
= help: see additional errors emitted
error: expected a message field for "missing-message"
--> ./missing-message.ftl:1:1
|
1 | missing-message =
| ^^^^^^^^^^^^^^^^^^
|
```
...or generating the following code if it succeeds:
```rust
pub static DEFAULT_LOCALE_RESOURCES: &'static [&'static str] = &[
include_str!("./typeck.ftl"),
];
mod fluent_generated {
mod typeck {
pub const field_multiply_specified_in_initializer: DiagnosticMessage =
DiagnosticMessage::fluent("typeck-field-multiply-specified-in-initializer");
pub const field_multiply_specified_in_initializer_label_previous_use: DiagnosticMessage =
DiagnosticMessage::fluent_attr(
"typeck-field-multiply-specified-in-initializer",
"previous-use-label"
);
}
}
```
When emitting a diagnostic, the generated constants can be used as follows:
```rust
let mut err = sess.struct_span_err(
span,
fluent::typeck::field_multiply_specified_in_initializer
);
err.span_label(
span,
fluent::typeck::field_multiply_specified_in_initializer_label
);
err.span_label(
previous_use_span,
fluent::typeck::field_multiply_specified_in_initializer_label_previous_use
);
err.emit();
```
I'd like to reduce the verbosity of referring to labels/notes/helps with this scheme (though it wasn't much better before), but I'll leave that for a follow-up.
r? `@oli-obk`
cc `@pvdrz` `@compiler-errors`
Update jemalloc to v5.3
Now that `jemalloc` version 5.3 has been released, this PR updates `tikv-jemalloc-sys` to the corresponding release.
The crates.io publishing issue seems to have been resolved for the `jemalloc-sys` package, and version 5.3.0 is now also available under the historical name (and should become the preferred crate to be used). Therefore, this PR also switches back to using `jemalloc-sys` instead of `tikv-jemalloc-sys`.
Adds a new `fluent_messages` macro which performs compile-time
validation of the compiler's Fluent resources (i.e. that the resources
parse and don't multiply define the same messages) and generates
constants that make using those messages in diagnostics more ergonomic.
For example, given the following invocation of the macro..
```ignore (rust)
fluent_messages! {
typeck => "./typeck.ftl",
}
```
..where `typeck.ftl` has the following contents..
```fluent
typeck-field-multiply-specified-in-initializer =
field `{$ident}` specified more than once
.label = used more than once
.label-previous-use = first use of `{$ident}`
```
...then the macro parse the Fluent resource, emitting a diagnostic if it
fails to do so, and will generate the following code:
```ignore (rust)
pub static DEFAULT_LOCALE_RESOURCES: &'static [&'static str] = &[
include_str!("./typeck.ftl"),
];
mod fluent_generated {
mod typeck {
pub const field_multiply_specified_in_initializer: DiagnosticMessage =
DiagnosticMessage::fluent("typeck-field-multiply-specified-in-initializer");
pub const field_multiply_specified_in_initializer_label_previous_use: DiagnosticMessage =
DiagnosticMessage::fluent_attr(
"typeck-field-multiply-specified-in-initializer",
"previous-use-label"
);
}
}
```
When emitting a diagnostic, the generated constants can be used as
follows:
```ignore (rust)
let mut err = sess.struct_span_err(
span,
fluent::typeck::field_multiply_specified_in_initializer
);
err.span_default_label(span);
err.span_label(
previous_use_span,
fluent::typeck::field_multiply_specified_in_initializer_label_previous_use
);
err.emit();
```
Signed-off-by: David Wood <david.wood@huawei.com>
In #95604 the compiler started generating a temporary symbols.o which is added
to the linker invocation. This object file has an `e_flags` which may be invalid
for 32-bit MIPS targets. Even though symbols.o doesn't contain code, linking
with [lld fails](https://github.com/llvm/llvm-project/blob/main/lld/ELF/Arch/MipsArchTree.cpp#L79) with
```
rust-lld: error: foo-cgu.0.rcgu.o: ABI 'o32' is incompatible with target ABI 'n64'
```
because it omits the ABI bits (EF_MIPS_ABI_O32) so lld assumes it's using the
N64 ABI. This breaks linking on nightly for the out-of-tree [psx
target](https://github.com/ayrtonm/psx-sdk-rs/issues/9), the builtin
mipsel-sony-psp target (cc @overdrivenpotato) and any other 32-bit MIPS
target using lld.
This PR sets the ABI in `e_flags` to O32 since that's the only ABI for 32-bit
MIPS that LLVM supports. It also sets other `e_flags` bits based on the target.
I had to bump the object crate version since some of these constants were [added
recently](https://github.com/gimli-rs/object/pull/433). I'm not sure if this
PR needs a test, but I can confirm that it fixes the linking issue on both
targets I mentioned.
