Stop keeping metadata in memory before writing it to disk
Fixes#96358
I created this PR according with the instruction given in the issue except for the following points:
- While the issue says "Write metadata into the temporary file in `encode_and_write_metadata` even if `!need_metadata_file`", I could not do that. That is because though I tried to do that and run `x.py test`, I got a lot of test failures as follows.
<details>
<summary>List of failed tests</summary>
<pre>
<code>
failures:
[ui] src/test/ui/json-multiple.rs
[ui] src/test/ui/json-options.rs
[ui] src/test/ui/rmeta/rmeta-rpass.rs
[ui] src/test/ui/save-analysis/emit-notifications.rs
[ui] src/test/ui/svh/changing-crates.rs
[ui] src/test/ui/svh/svh-change-lit.rs
[ui] src/test/ui/svh/svh-change-significant-cfg.rs
[ui] src/test/ui/svh/svh-change-trait-bound.rs
[ui] src/test/ui/svh/svh-change-type-arg.rs
[ui] src/test/ui/svh/svh-change-type-ret.rs
[ui] src/test/ui/svh/svh-change-type-static.rs
[ui] src/test/ui/svh/svh-use-trait.rs
test result: FAILED. 12915 passed; 12 failed; 100 ignored; 0 measured; 0 filtered out; finished in 71.41s
Some tests failed in compiletest suite=ui mode=ui host=x86_64-unknown-linux-gnu target=x86_64-unknown-linux-gnu
Build completed unsuccessfully in 0:01:58
</code>
</pre>
</details>
- I could not resolve the extra tasks about `create_rmeta_file` and `create_compressed_metadata_file` for my lack of ability.
macros: `LintDiagnostic` derive
- Move `LintDiagnosticBuilder` into `rustc_errors` so that a diagnostic derive can refer to it.
- Introduce a `DecorateLint` trait, which is equivalent to `SessionDiagnostic` or `AddToDiagnostic` but for lints. Necessary without making more changes to the lint infrastructure as `DecorateLint` takes a `LintDiagnosticBuilder` and re-uses all of the existing logic for determining what type of diagnostic a lint should be emitted as (e.g. error/warning).
- Various refactorings of the diagnostic derive machinery (extracting `build_field_mapping` helper and moving `sess` field out of the `DiagnosticDeriveBuilder`).
- Introduce a `LintDiagnostic` derive macro that works almost exactly like the `SessionDiagnostic` derive macro except that it derives a `DecorateLint` implementation instead. A new derive is necessary for this because `SessionDiagnostic` is intended for when the generated code creates the diagnostic. `AddToDiagnostic` could have been used but it would have required more changes to the lint machinery.
~~At time of opening this pull request, ignore all of the commits from #98624, it's just the last few commits that are new.~~
r? `@oli-obk`
`SessionDiagnostic` isn't suitable for use on lints as whether or not it
creates an error or a warning is decided at compile-time by the macro,
whereas lints decide this at runtime based on the location of the lint
being reported (as it will depend on the user's `allow`/`deny`
attributes, etc). Re-using most of the machinery for
`SessionDiagnostic`, this macro introduces a `LintDiagnostic` derive
which implements a `DecorateLint` trait, taking a
`LintDiagnosticBuilder` and adding to the lint according to the
diagnostic struct.
Bump RLS to latest master on rust-lang/rls
Of primary interest, this merges
rust-lang/rls@ece09b88c0 into rust-lang/rust,
which brings in the changes that fix RLS tests broken by #97853. #97853 already
introduced that commit's changes (under
rust-lang/rls@27f4044df0) but without putting those changes on
rust-lang/rls as a branch, so we ended up with an orphan commit that caused
trouble when updating submodules in rust-lang/rust.
This commit, once merged into rust-lang/rust, should continue to let RLS tests
to pass on rust-lang/rust's side and move us back into a healthy state where tip
of the submodule points to a valid master commit in the rust-lang/rls
repository.
cc https://github.com/rust-lang/rust/issues/98451, but not marking as fixed as I believe we need to add verification to prevent future oversights.
Fully remove submodule handling from bootstrap.py
These submodules were previously updated in python because Cargo gives a hard error if toml files
are missing from the workspace:
```
error: failed to load manifest for workspace member `/home/jnelson/rust-lang/rust/src/tools/rls`
Caused by:
failed to read `/home/jnelson/rust-lang/rust/src/tools/rls/Cargo.toml`
Caused by:
No such file or directory (os error 2)
failed to run: /home/jnelson/rust-lang/rust/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /home/jnelson/rust-lang/rust/src/bootstrap/Cargo.toml
```
However, bootstrap doesn't actually need to be part of the workspace.
Remove it so we can move submodule handling fully to Rust, avoiding duplicate code between Rust and Python.
Note that this does break `cargo run`; it has to be `cd src/bootstrap && cargo run` now.
Given that we're planning to make the main entrypoint a shell script (or rust binary),
I think this is a good tradeoff for reduced complexity in bootstrap.py.
To get this working, I also had to remove support for vendoring when using the git sources, because `cargo vendor` requires all submodules to be checked out. I think this is ok; people who care about this are likely already using the pre-vendored `rustc-src` tarball.
Fixes https://github.com/rust-lang/rust/issues/90764. Helps with #94829
Of primary interest, this merges
rust-lang/rls@ece09b88c0 into rust-lang/rust,
which brings in the changes that fix RLS tests broken by #97853. #97853 already
introduced that commit's changes (under
27f4044df03d15c7c38a483c3e4635cf4f51807d) but without putting those changes on
rust-lang/rls as a branch, so we ended up with an orphan commit that caused
trouble when updating submodules in rust-lang/rust.
This commit, once merged into rust-lang/rust, should continue to let RLS tests
to pass on rust-lang/rust's side and move us back into a healthy state where tip
of the submodule points to a valid master commit in the rust-lang/rls
repository.
Migrate two diagnostics from the `rustc_builtin_macros` crate
Migrate two diagnostics to use the struct derive and be translatable.
r? ```@davidtwco```
These submodules were previously updated in python because Cargo gives a hard error if toml files
are missing from the workspace:
```
error: failed to load manifest for workspace member `/home/jnelson/rust-lang/rust/src/tools/rls`
Caused by:
failed to read `/home/jnelson/rust-lang/rust/src/tools/rls/Cargo.toml`
Caused by:
No such file or directory (os error 2)
failed to run: /home/jnelson/rust-lang/rust/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /home/jnelson/rust-lang/rust/src/bootstrap/Cargo.toml
```
However, bootstrap doesn't actually need to be part of the workspace.
Remove it so we can move submodule handling fully to Rust, avoiding duplicate code between Rust and Python.
Note that this does break `cargo run`; it has to be `cd src/bootstrap && cargo run` now.
Given that we're planning to make the main entrypoint a shell script (or rust binary),
I think this is a good tradeoff for reduced complexity in bootstrap.py.
Make "Assemble stage1 compiler" orders of magnitude faster (take 2)
This used to take upwards of 5 seconds for me locally. I found that the culprit was copying the downloaded LLVM shared object:
```
[22:28:03] Install "/home/jnelson/rust-lang/rust/build/x86_64-unknown-linux-gnu/ci-llvm/lib/libLLVM-14-rust-1.62.0-nightly.so" to "/home/jnelson/rust-lang/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libLLVM-14-rust-1.62.0-nightly.so"
[22:28:09] c Sysroot { compiler: Compiler { stage: 1, host: x86_64-unknown-linux-gnu(x86_64-unknown-linux-gnu) } }
```
It turned out that `install()` used full copies unconditionally. Change it to try using a hard-link before falling back to copying.
- Panic if we generate a symbolic link in a tarball
- Change install to use copy internally, like in my previous PR
- Change copy to dereference symbolic links, which avoids the previous regression in #96803.
I also took the liberty of fixing `x dist llvm-tools` to work even if you don't call `x build` previously.
add comments in `store_dead_field_or_variant`
support multiple log level
add a item ident label
fix ui tests
fix a ui test
fix a rustdoc ui test
use let chain
refactor: remove `store_dead_field_or_variant`
fix a tiny bug
4 commits in 4d92f07f34ba7fb7d7f207564942508f46c225d3..8d42b0e8794ce3787c9f7d6d88b02ae80ebe8d19
2022-06-10 01:11:04 +0000 to 2022-06-17 16:46:26 +0000
- Use specific terminology for sparse HTTP-based registry (rust-lang/cargo#10764)
- chore: Upgrade to clap 3.2 (rust-lang/cargo#10753)
- Improve testing framework for http registries (rust-lang/cargo#10738)
- doc: Improve example of using the links field (rust-lang/cargo#10728)
This adds the typeid and `vcall_visibility` metadata to vtables when the
-Cvirtual-function-elimination flag is set.
The typeid is generated in the same way as for the
`llvm.type.checked.load` intrinsic from the trait_ref.
The offset that is added to the typeid is always 0. This is because LLVM
assumes that vtables are constructed according to the definition in the
Itanium ABI. This includes an "address point" of the vtable. In C++ this
is the offset in the vtable where information for RTTI is placed. Since
there is no RTTI information in Rust's vtables, this "address point" is
always 0. This "address point" in combination with the offset passed to
the `llvm.type.checked.load` intrinsic determines the final function
that should be loaded from the vtable in the
`WholeProgramDevirtualization` pass in LLVM. That's why the
`llvm.type.checked.load` intrinsics are generated with the typeid of the
trait, rather than with that of the function that is called. This
matches what `clang` does for C++.
The vcall_visibility metadata depends on three factors:
1. LTO level: Currently this is always fat LTO, because LLVM only
supports this optimization with fat LTO.
2. Visibility of the trait: If the trait is publicly visible, VFE
can only act on its vtables after linking.
3. Number of CGUs: if there is more than one CGU, also vtables with
restricted visibility could be seen outside of the CGU, so VFE can
only act on them after linking.
To reflect this, there are three visibility levels: Public, LinkageUnit,
and TranslationUnit.
Update minifier version to 0.2.1
This change and these changes come from an idea of `@camelid:` instead of creating a string, we just `write` the type into the file directly.
I don't think it'll have a big impact on perf but it's still a potential small improvement.
r? `@notriddle`
Update cargo
7 commits in 38472bc19f2f76e245eba54a6e97ee6821b3c1db..85e457e158db216a2938d51bc3b617a5a7fe6015
2022-05-31 02:03:24 +0000 to 2022-06-07 21:57:52 +0000
- Make -Z http-registry use index.crates.io when accessing crates-io (rust-lang/cargo#10725)
- Respect submodule update=none strategy in .gitmodules (rust-lang/cargo#10717)
- Expose rust-version through env var (rust-lang/cargo#10713)
- add validation for string "true"/"false" in lto profile (rust-lang/cargo#10676)
- Enhance documentation of testing (rust-lang/cargo#10726)
- Clear disk space on CI. (rust-lang/cargo#10724)
- Enforce to use tar v0.4.38 (rust-lang/cargo#10720)
Add build metrics to rustbuild
This PR adds a new module of rustbuild, `ci_profiler`, whose job is to gather as much information as possible about the CI build as possible and store it in a JSON file uploaded to `ci-artifacts`. Right now for each step it collects:
* Type name and debug representation of the `Step` object.
* Duration of the step (excluding child steps).
* Systemwide CPU stats for the duration of the step (both single core and all cores).
* Which child steps were executed.
This is capable of replacing both the scripts to collect CPU stats and the `[TIMING]` lines in build logs (not yet removed, until we port our tooling to use the CI profiler). The format is also extensible to be able in the future to collect more information.
r? `@Mark-Simulacrum`
library/std: Bump compiler_builtins
Some neat changes include faster float conversions & fixes for AVR 🙂
(note that's it's my first time upgrading `compiler_builtins`, so I'm not 100% sure if bumping `library/std/Cargo.toml` is enough; certainly seems to be so, though.)
Update to rebased rustc-rayon 0.4
In rayon-rs/rayon#938, miri uncovered a race in `rustc-rayon-core` that had already been fixed in the regular `rayon-core`. I have now rebased that fork onto the latest rayon branch, and published as 0.4. I also updated `indexmap` to bump the dependency.
`Cargo.lock` changes:
Updating indexmap v1.8.0 -> v1.8.2
Updating rayon v1.5.1 -> v1.5.3
Updating rayon-core v1.9.1 -> v1.9.3
Updating rustc-rayon v0.3.2 -> v0.4.0
Updating rustc-rayon-core v0.3.2 -> v0.4.1
macros: introduce `fluent_messages` macro
Adds a new `fluent_messages` macro which performs compile-time validation of the compiler's Fluent resources (i.e. that the resources parse and don't multiply define the same messages) and generates constants that make using those messages in diagnostics more ergonomic.
For example, given the following invocation of the macro..
```rust
fluent_messages! {
typeck => "./typeck.ftl",
}
```
..where `typeck.ftl` has the following contents..
```fluent
typeck-field-multiply-specified-in-initializer =
field `{$ident}` specified more than once
.label = used more than once
.label-previous-use = first use of `{$ident}`
```
...then the macro parse the Fluent resource, emitting a diagnostic if it fails to do so...
```text
error: could not parse Fluent resource
--> $DIR/test.rs:35:28
|
LL | missing_message => "./missing-message.ftl",
| ^^^^^^^^^^^^^^^^^^^^^^^
|
= help: see additional errors emitted
error: expected a message field for "missing-message"
--> ./missing-message.ftl:1:1
|
1 | missing-message =
| ^^^^^^^^^^^^^^^^^^
|
```
...or generating the following code if it succeeds:
```rust
pub static DEFAULT_LOCALE_RESOURCES: &'static [&'static str] = &[
include_str!("./typeck.ftl"),
];
mod fluent_generated {
mod typeck {
pub const field_multiply_specified_in_initializer: DiagnosticMessage =
DiagnosticMessage::fluent("typeck-field-multiply-specified-in-initializer");
pub const field_multiply_specified_in_initializer_label_previous_use: DiagnosticMessage =
DiagnosticMessage::fluent_attr(
"typeck-field-multiply-specified-in-initializer",
"previous-use-label"
);
}
}
```
When emitting a diagnostic, the generated constants can be used as follows:
```rust
let mut err = sess.struct_span_err(
span,
fluent::typeck::field_multiply_specified_in_initializer
);
err.span_label(
span,
fluent::typeck::field_multiply_specified_in_initializer_label
);
err.span_label(
previous_use_span,
fluent::typeck::field_multiply_specified_in_initializer_label_previous_use
);
err.emit();
```
I'd like to reduce the verbosity of referring to labels/notes/helps with this scheme (though it wasn't much better before), but I'll leave that for a follow-up.
r? `@oli-obk`
cc `@pvdrz` `@compiler-errors`
Update jemalloc to v5.3
Now that `jemalloc` version 5.3 has been released, this PR updates `tikv-jemalloc-sys` to the corresponding release.
The crates.io publishing issue seems to have been resolved for the `jemalloc-sys` package, and version 5.3.0 is now also available under the historical name (and should become the preferred crate to be used). Therefore, this PR also switches back to using `jemalloc-sys` instead of `tikv-jemalloc-sys`.
Adds a new `fluent_messages` macro which performs compile-time
validation of the compiler's Fluent resources (i.e. that the resources
parse and don't multiply define the same messages) and generates
constants that make using those messages in diagnostics more ergonomic.
For example, given the following invocation of the macro..
```ignore (rust)
fluent_messages! {
typeck => "./typeck.ftl",
}
```
..where `typeck.ftl` has the following contents..
```fluent
typeck-field-multiply-specified-in-initializer =
field `{$ident}` specified more than once
.label = used more than once
.label-previous-use = first use of `{$ident}`
```
...then the macro parse the Fluent resource, emitting a diagnostic if it
fails to do so, and will generate the following code:
```ignore (rust)
pub static DEFAULT_LOCALE_RESOURCES: &'static [&'static str] = &[
include_str!("./typeck.ftl"),
];
mod fluent_generated {
mod typeck {
pub const field_multiply_specified_in_initializer: DiagnosticMessage =
DiagnosticMessage::fluent("typeck-field-multiply-specified-in-initializer");
pub const field_multiply_specified_in_initializer_label_previous_use: DiagnosticMessage =
DiagnosticMessage::fluent_attr(
"typeck-field-multiply-specified-in-initializer",
"previous-use-label"
);
}
}
```
When emitting a diagnostic, the generated constants can be used as
follows:
```ignore (rust)
let mut err = sess.struct_span_err(
span,
fluent::typeck::field_multiply_specified_in_initializer
);
err.span_default_label(span);
err.span_label(
previous_use_span,
fluent::typeck::field_multiply_specified_in_initializer_label_previous_use
);
err.emit();
```
Signed-off-by: David Wood <david.wood@huawei.com>
In #95604 the compiler started generating a temporary symbols.o which is added
to the linker invocation. This object file has an `e_flags` which may be invalid
for 32-bit MIPS targets. Even though symbols.o doesn't contain code, linking
with [lld fails](https://github.com/llvm/llvm-project/blob/main/lld/ELF/Arch/MipsArchTree.cpp#L79) with
```
rust-lld: error: foo-cgu.0.rcgu.o: ABI 'o32' is incompatible with target ABI 'n64'
```
because it omits the ABI bits (EF_MIPS_ABI_O32) so lld assumes it's using the
N64 ABI. This breaks linking on nightly for the out-of-tree [psx
target](https://github.com/ayrtonm/psx-sdk-rs/issues/9), the builtin
mipsel-sony-psp target (cc @overdrivenpotato) and any other 32-bit MIPS
target using lld.
This PR sets the ABI in `e_flags` to O32 since that's the only ABI for 32-bit
MIPS that LLVM supports. It also sets other `e_flags` bits based on the target.
I had to bump the object crate version since some of these constants were [added
recently](https://github.com/gimli-rs/object/pull/433). I'm not sure if this
PR needs a test, but I can confirm that it fixes the linking issue on both
targets I mentioned.
Use futex-based locks and thread parker on {Free, Open, DragonFly}BSD.
This switches *BSD to our futex-based locks and thread parker.
Tracking issue: https://github.com/rust-lang/rust/issues/93740
This is a draft, because this still needs a new version of the `libc` crate to be published that includes https://github.com/rust-lang/libc/pull/2770.
r? `@Amanieu`
Add a new Rust attribute to support embedding debugger visualizers
Implemented [this RFC](https://github.com/rust-lang/rfcs/pull/3191) to add support for embedding debugger visualizers into a PDB.
Added a new attribute `#[debugger_visualizer]` and updated the `CrateMetadata` to store debugger visualizers for crate dependencies.
RFC: https://github.com/rust-lang/rfcs/pull/3191
Avoid using `rand::thread_rng` in the stdlib benchmarks.
This is kind of an anti-pattern because it introduces extra nondeterminism for no real reason. In thread_rng's case this comes both from the random seed and also from the reseeding operations it does, which occasionally does syscalls (which adds additional nondeterminism). The impact of this would be pretty small in most cases, but it's a good practice to avoid (particularly because avoiding it was not hard).
Anyway, several of our benchmarks already did the right thing here anyway, so the change was pretty easy and mostly just applying it more universally. That said, the stdlib benchmarks aren't particularly stable (nor is our benchmark framework particularly great), so arguably this doesn't matter that much in practice.
~~Anyway, this also bumps the `rand` dev-dependency to 0.8, since it had fallen somewhat out of date.~~ Nevermind, too much of a headache.
Enable tracing for all queries
This allows you to log everything within a specific query, e.g.
```
env RUSTC_LOG=[mir_borrowck]
```
dumping all borrowck queries may be a bit verbose, so you can also restrict it to just an item of your choice:
```
env RUSTC_LOG=[mir_borrowck{key=\.\*name_of_item\.\*}]
```
the regex `.*` in the key name are because the key is a debug printed DefId, so you'd get all kinds of things like hashes in there. The tracing logs will show you the key, so you can restrict it further if you want.
Cleanup `DebuggerVisualizerFile` type and other minor cleanup of queries.
Merge the queries for debugger visualizers into a single query.
Revert move of `resolve_path` to `rustc_builtin_macros`. Update dependencies in Cargo.toml for `rustc_passes`.
Respond to PR comments. Load visualizer files into opaque bytes `Vec<u8>`. Debugger visualizers for dynamically linked crates should not be embedded in the current crate.
Update the unstable book with the new feature. Add the tracking issue for the debugger_visualizer feature.
Respond to PR comments and minor cleanups.
This tool will generate a JSON file with statistics about each
individual step to disk. It will be used in rust-lang/rust's CI to
replace the mix of scripts and log scraping we currently have to gather
this data.
This attempts to keep the logic as close to the original python as possible.
`probably_large` has been removed, since it was always `True`, and UTF-8 paths are no longer supported when patching files for NixOS.
I can readd UTF-8 support if desired.
Note that this required making `llvm_link_shared` computed on-demand,
since we don't know whether it will be static or dynamic until we download LLVM from CI.
Update cargo
7 commits in dba5baf4345858c591517b24801902a062c399f8..edffc4ada3d77799e5a04eeafd9b2f843d29fc23
2022-04-13 21:58:27 +0000 to 2022-04-19 17:38:29 +0000
- Document cargo-add (rust-lang/cargo#10578)
- feat: Support '-F' as an alias for '--features' (rust-lang/cargo#10576)
- Completion support for `cargo-add` (rust-lang/cargo#10577)
- Add a link to the document in the timings report (rust-lang/cargo#10492)
- feat: Import cargo-add into cargo (rust-lang/cargo#10472)
- Part 8 of RFC2906 - Keep `InheritableFields` in a `LazyCell` inside `… (rust-lang/cargo#10568)
- Part 7 of RFC2906 - Add support for inheriting `exclude` and `include` (rust-lang/cargo#10565)
Remove `--extern-location` and all associated code
`--extern-location` was an experiment to investigate the best way to
generate useful diagnostics for unused dependency warnings by enabling a
build system to identify the corresponding build config.
While I did successfully use this, I've since been convinced the
alternative `--json unused-externs` mechanism is the way to go, and
there's no point in having two mechanisms with basically the same
functionality.
This effectively reverts https://github.com/rust-lang/rust/pull/72603
Add support for LibreSSL 3.4.x
This updates the `openssl` and `openssl-sys` crates to support building
the toolchain with system libraries up to LibreSSL 3.4.x.
LibreSSL 3.4.0 has been supported since `openssl-sys` version 0.9.67,
LibreSSL 3.4.x since `openssl-sys` 0.9.72.
This updates the `openssl` and `openssl-sys` crates to support building
the toolchain with system libraries up to LibreSSL 3.4.x.
LibreSSL 3.4.0 has been supported since `openssl-sys` version 0.9.67,
LibreSSL 3.4.x since `openssl-sys` 0.9.72.
`--extern-location` was an experiment to investigate the best way to
generate useful diagnostics for unused dependency warnings by enabling a
build system to identify the corresponding build config.
While I did successfully use this, I've since been convinced the
alternative `--json unused-externs` mechanism is the way to go, and
there's no point in having two mechanisms with basically the same
functionality.
This effectively reverts https://github.com/rust-lang/rust/pull/72603
With the updated libc, UNIX stack overflow handling in libstd can now
use the common `si_addr` accessor function, rather than attempting to
use a field from that name in `siginfo_t`. This simplifies the
collection of the fault address, particularly on platforms where that
data resides within a union in `siginfo_t`.
Conditional on the parallel compiler being enabled, use a different
`IntlLangMemoizer` which supports being sent between threads in
`FluentBundle`.
Signed-off-by: David Wood <david.wood@huawei.com>
Extend loading of Fluent bundles so that bundles can be loaded from the
sysroot based on the language requested by the user, or using a nightly
flag.
Sysroot bundles are loaded from `$sysroot/share/locale/$locale/*.ftl`.
Signed-off-by: David Wood <david.wood@huawei.com>
This commit updates the signatures of all diagnostic functions to accept
types that can be converted into a `DiagnosticMessage`. This enables
existing diagnostic calls to continue to work as before and Fluent
identifiers to be provided. The `SessionDiagnostic` derive just
generates normal diagnostic calls, so these APIs had to be modified to
accept Fluent identifiers.
In addition, loading of the "fallback" Fluent bundle, which contains the
built-in English messages, has been implemented.
Each diagnostic now has "arguments" which correspond to variables in the
Fluent messages (necessary to render a Fluent message) but no API for
adding arguments has been added yet. Therefore, diagnostics (that do not
require interpolation) can be converted to use Fluent identifiers and
will be output as before.
`MultiSpan` contains labels, which are more complicated with the
introduction of diagnostic translation and will use types from
`rustc_errors` - however, `rustc_errors` depends on `rustc_span` so
`rustc_span` cannot use types like `DiagnosticMessage` without
dependency cycles. Introduce a new `rustc_error_messages` crate that can
contain `DiagnosticMessage` and `MultiSpan`.
Signed-off-by: David Wood <david.wood@huawei.com>
The majority of the code is only used by either rustbuild or
rustc_llvm's build script. Rust_build is compiled once for rustbuild and
once for every stage. This means that the majority of the code in this
crate is needlessly compiled multiple times. By moving only the code
actually used by the respective crates to rustbuild and rustc_llvm's
build script, this needless duplicate compilation is avoided.
Bump autocfg to 1.1.0
autocfg 1.1.0 makes it so that rustflags from the build are correctly
passed to the compiler probes, which in turn means those probes more
accurately reflect the outer build conditions. This is particularly
important if rustflags includes _required_ `-Clink-arg=` flags without
which builds will fail, as older versions of `autocfg` will then fail
the probe and erroneously report the probed feature as unavailable.
See also
https://github.com/rust-lang/rust/issues/94007#issuecomment-1040668261
Remove num_cpus dependency from bootstrap, build-manifest and rustc_s…
…ession
`std::threads::available_parallelism` was stabilized in rust 1.59.
r? ```````````````````````````@Mark-Simulacrum```````````````````````````
autocfg 1.1.0 makes it so that rustflags from the build are correctly
passed to the compiler probes, which in turn means those probes more
accurately reflect the outer build conditions. This is particularly
important if rustflags includes _required_ `-Clink-arg=` flags without
which builds will fail, as older versions of `autocfg` will then fail
the probe and erroneously report the probed feature as unavailable.
See also
https://github.com/rust-lang/rust/issues/94007#issuecomment-1040668261
debuginfo: Simplify TypeMap used during LLVM debuginfo generation.
This PR simplifies the TypeMap that is used in `rustc_codegen_llvm::debuginfo::metadata`. It was unnecessarily complicated because it was originally implemented when types were not yet normalized before codegen. So it did it's own normalization and kept track of multiple unnormalized types being mapped to a single unique id.
This PR is based on https://github.com/rust-lang/rust/pull/93503, which is not merged yet.
The PR also removes the arena used for allocating string ids and instead uses `InlinableString` from the [inlinable_string](https://crates.io/crates/inlinable_string) crate. That might not be the best choice, since that crate does not seem to be very actively maintained. The [flexible-string](https://crates.io/crates/flexible-string) crate would be an alternative.
r? `@ghost`
The previous implementation was written before types were properly
normalized for code generation and had to assume a more complicated
relationship between types and their debuginfo -- generating separate
identifiers for debuginfo nodes that were based on normalized types.
Since types are now already normalized, we can use them as identifiers
for debuginfo nodes.
removing architecture requirements for RustyHermit
RustHermit and HermitCore is able to run on aarch64 and x86_64. In the future these operating systems will also support RISC-V. Consequently, the dependency to a specific target should be removed.
The build process of `hermit-abi` fails if the architecture isn't supported.
RustHermit and HermitCore is able to run on aarch64 and x86_64.
In the future these operating systems will also support RISC-V.
Consequently, the dependency to a specific target should be removed.
Building hermit-abi fails if the architecture isn't supported.
Drop time dependency from bootstrap
This was only used for the inclusion of 'current' dates into our manpages, but
it is not clear that this is practically necessary. The manpage is essentially
never updated, and so we can likely afford to keep a manual date in these files.
It also seems possible to just omit it, but that may cause other tools trouble,
so avoid doing that for now.
This is largely done to reduce bootstrap complexity; the time crate is not particularly
small and in #92480 would have started pulling in num-threads, which does runtime
thread count detection. I would prefer to avoid that, so filing this to just drop the nearly
unused dependency entirely.
r? `@pietroalbini`
This was only used for the inclusion of 'current' dates into our manpages, but
it is not clear that this is practically necessary. The manpage is essentially
never updated, and so we can likely afford to keep a manual date in these files.
It also seems possible to just omit it, but that may cause other tools trouble,
so avoid doing that for now.
remove unused `jemallocator` crate
When it was noticed that the rustc binary wasn't actually using jemalloc via `#[global_allocator]` and that was removed, the dependency remained.
Tests pass locally with a `jemalloc = true` build, but I'll trigger a try build to ensure I haven't missed an edge-case somewhere.
r? ```@ghost``` until that completes
Bump libc and fix remove_dir_all on Fuchsia after CVE fix
With the previous `is_dir` impl, we would attempt to unlink
a directory in the None branch, but Fuchsia supports returning
ENOTEMPTY from unlinkat() without the AT_REMOVEDIR flag because
we don't currently differentiate unlinking files and directories
by default.
On the Fuchsia side I've opened https://fxbug.dev/92273 to discuss
whether this is the correct behavior, but it doesn't seem like
addressing the error code is necessary to make our tests happy.
Depends on https://github.com/rust-lang/libc/pull/2654 since we
apparently haven't needed to reference DT_UNKNOWN before this.
With the previous `is_dir` impl, we would attempt to unlink
a directory in the None branch, but Fuchsia supports returning
ENOTEMPTY from unlinkat() without the AT_REMOVEDIR flag because
we don't currently differentiate unlinking files and directories
by default.
On the Fuchsia side I've opened https://fxbug.dev/92273 to discuss
whether this is the correct behavior, but it doesn't seem like
addressing the error code is necessary to make our tests happy.
Updates std's libc crate to include DT_UNKNOWN for Fuchsia.
Update some rustc dependencies to deduplicate them
This PR updates `rand` and `itertools` in rustc (not the whole workspace) in order to deduplicate them (and hopefully slightly improve compile times).
~~Currently, `object` is still duplicated, but https://github.com/rust-lang/thorin/pull/15 and updating `thorin` in the future will remove the use of version 0.27.~~ Update: Thorin 0.2 has now been released, and this PR updates `rustc_codegen_ssa` to use it and deduplicate the `object` crate.
There's a final tiny rustc dependency, `cfg-if`, which will be left: as both versions 0.1.x and 1.0 looked to be heavily depended on, they will require a few cascading updates to be removed.
Introduce drop range tracking to generator interior analysis
This PR addresses cases such as this one from #57478:
```rust
struct Foo;
impl !Send for Foo {}
let _: impl Send = || {
let guard = Foo;
drop(guard);
yield;
};
```
Previously, the `generator_interior` pass would unnecessarily include the type `Foo` in the generator because it was not aware of the behavior of `drop`. We fix this issue by introducing a drop range analysis that finds portions of the code where a value is guaranteed to be dropped. If a value is dropped at all suspend points, then it is no longer included in the generator type. Note that we are using "dropped" in a generic sense to include any case in which a value has been moved. That is, we do not only look at calls to the `drop` function.
There are several phases to the drop tracking algorithm, and we'll go into more detail below.
1. Use `ExprUseVisitor` to find values that are consumed and borrowed.
2. `DropRangeVisitor` uses consume and borrow information to gather drop and reinitialization events, as well as build a control flow graph.
3. We then propagate drop and reinitialization information through the CFG until we reach a fix point (see `DropRanges::propagate_to_fixpoint`).
4. When recording a type (see `InteriorVisitor::record`), we check the computed drop ranges to see if that value is definitely dropped at the suspend point. If so, we skip including it in the type.
## 1. Use `ExprUseVisitor` to find values that are consumed and borrowed.
We use `ExprUseVisitor` to identify the places where values are consumed. We track both the `hir_id` of the value, and the `hir_id` of the expression that consumes it. For example, in the expression `[Foo]`, the `Foo` is consumed by the array expression, so after the array expression we can consider the `Foo` temporary to be dropped.
In this process, we also collect values that are borrowed. The reason is that the MIR transform for generators conservatively assumes anything borrowed is live across a suspend point (see `rustc_mir_transform::generator::locals_live_across_suspend_points`). We match this behavior here as well.
## 2. Gather drop events, reinitialization events, and control flow graph
After finding the values of interest, we perform a post-order traversal over the HIR tree to find the points where these values are dropped or reinitialized. We use the post-order index of each event because this is how the existing generator interior analysis refers to the position of suspend points and the scopes of variables.
During this traversal, we also record branching and merging information to handle control flow constructs such as `if`, `match`, and `loop`. This is necessary because values may be dropped along some control flow paths but not others.
## 3. Iterate to fixed point
The previous pass found the interesting events and locations, but now we need to find the actual ranges where things are dropped. Upon entry, we have a list of nodes ordered by their position in the post-order traversal. Each node has a set of successors. For each node we additionally keep a bitfield with one bit per potentially consumed value. The bit is set if we the value is dropped along all paths entering this node.
To compute the drop information, we first reverse the successor edges to find each node's predecessors. Then we iterate through each node, and for each node we set its dropped value bitfield to the intersection of all incoming dropped value bitfields.
If any bitfield for any node changes, we re-run the propagation loop again.
## 4. Ignore dropped values across suspend points
At this point we have a data structure where we can ask whether a value is guaranteed to be dropped at any post order index for the HIR tree. We use this information in `InteriorVisitor` to check whether a value in question is dropped at a particular suspend point. If it is, we do not include that value's type in the generator type.
Note that we had to augment the region scope tree to include all yields in scope, rather than just the last one as we did before.
r? `@nikomatsakis`
All tests pass now! The issue was that we weren't handling all edges
correctly, but now they are handled consistently.
This includes code to dump a graphviz file for the CFG we built for drop
tracking.
Also removes old DropRanges tests.
This adds support for branching and merging control flow and uses this
to correctly handle the case where a value is dropped in one branch of
an if expression but not another.
There are other cases we need to handle, which will come in follow up
patches.
Issue #57478
ProjectionPredicate should be able to handle both associated types and consts so this adds the
first step of that. It mainly just pipes types all the way down, not entirely sure how to handle
consts, but hopefully that'll come with time.
rustdoc: remove hand-rolled isatty
This PR replaces bindings to the platform-specific isatty APIs with the `isatty` crate, as done elsewhere in the repository.
Update rayon and rustc-rayon
This updates rayon for various tools and rustc-rayon for the compiler's parallel mode.
- rayon v1.3.1 -> v1.5.1
- rayon-core v1.7.1 -> v1.9.1
- rustc-rayon v0.3.1 -> v0.3.2
- rustc-rayon-core v0.3.1 -> v0.3.2
... and indirectly, this updates all of crossbeam-* to their latest versions.
Fixes#92677 by removing crossbeam-queue, but there's still a lingering question about how tidy discovers "runtime" dependencies. None of this is truly in the standard library's dependency tree at all.
