rustc_data_structures has a dependency on crossbeam-utils but never uses
it. It appears to have originally had this dependency in order to set
the "nightly" feature; however, its other dependencies use a different
version of crossbeam-utils, so this doesn't actually affect anything.
Furthermore, in current crossbeam-utils, the "nightly" feature has
become a no-op.
This updates the `openssl` and `openssl-sys` crates to support building
the toolchain with system libraries up to OpenSSL 3.0.0. This does not
affect the static version used via `openssl-src` in CI builds.
ref: https://github.com/sfackler/rust-openssl/pull/1264
Rustfix update
This updates to rustfix 0.6.0. One of the key changes here is https://github.com/rust-lang/rustfix/pull/195 which changes rustfix to apply multi-part suggestions. One of the tests needs to updated because there are some overlapping suggestions which rustfix cannot handle. The solution is to only apply the machine-applicable suggestions to avoid the overlapping issue.
This also includes a minor change to compiletest to provide better error messages with rustfix.
Use Tera templates for rustdoc.
Replaces a format!() call in layout::render with a template
expansion. Introduces a `templates` field in SharedContext so parts
of rustdoc can share pre-rendered templates.
This currently builds in a copy of the single template available, like
with static files. However, future work can make this live-loadable with
a perma-unstable flag, to make rustdoc developers' work easier.
Part of #84419.
Demo at https://hoffman-andrews.com/rust/tera/std/string/struct.String.html.
Replaces a format!() call in layout::render with a template
expansion. Introduces a `templates` field in SharedContext so parts
of rustdoc can share pre-rendered templates.
This currently builds in a copy of the single template available, like
with static files. However, future work can make this live-loadable with
a perma-unstable flag, to make rustdoc developers' work easier.
Lazify is_really_default condition in the RustdocGUI bootstrap step
The `RustdocGUI::should_run` condition spawns `npm list` several times which adds up to seconds of wall-time.
Evaluate the condition lazily to to keep `./x.py test tidy` and similar short-running tasks fast.
Fixes#86147
The `RustdocGUI::should_run` condition spawns `npm list` several times
which adds up to seconds of wall-time.
Evaluate the condition lazily to to keep `./x.py test tidy` and similar
short-running tasks fast.
rustc: Store metadata-in-rlibs in object files
This commit updates how rustc compiler metadata is stored in rlibs.
Previously metadata was stored as a raw file that has the same format as
`--emit metadata`. After this commit, however, the metadata is encoded
into a small object file which has one section which is the contents of
the metadata.
The motivation for this commit is to fix a common case where #83730
arises. The problem is that when rustc crates a `dylib` crate type it
needs to include entire rlib files into the dylib, so it passes
`--whole-archive` (or the equivalent) to the linker. The problem with
this, though, is that the linker will attempt to read all files in the
archive. If the metadata file were left as-is (today) then the linker
would generate an error saying it can't read the file. The previous
solution was to alter the rlib just before linking, creating a new
archive in a temporary directory which has the metadata file removed.
This problem from before this commit is now removed if the metadata file
is stored in an object file that the linker can read. The only caveat we
have to take care of is to ensure that the linker never actually
includes the contents of the object file into the final output. We apply
similar tricks as the `.llvmbc` bytecode sections to do this.
This involved changing the metadata loading code a bit, namely updating
some of the LLVM C APIs used to use non-deprecated ones and fiddling
with the lifetimes a bit to get everything to work out. Otherwise though
this isn't intended to be a functional change really, only that metadata
is stored differently in archives now.
This should end up fixing #83730 because by default dylibs will no
longer have their rlib dependencies "altered" meaning that
split-debuginfo will continue to have valid paths pointing at the
original rlibs. (note that we still "alter" rlibs if LTO is enabled to
remove Rust object files and we also "alter" for the #[link(cfg)]
feature, but that's rarely used).
Closes#83730
This commit updates how rustc compiler metadata is stored in rlibs.
Previously metadata was stored as a raw file that has the same format as
`--emit metadata`. After this commit, however, the metadata is encoded
into a small object file which has one section which is the contents of
the metadata.
The motivation for this commit is to fix a common case where #83730
arises. The problem is that when rustc crates a `dylib` crate type it
needs to include entire rlib files into the dylib, so it passes
`--whole-archive` (or the equivalent) to the linker. The problem with
this, though, is that the linker will attempt to read all files in the
archive. If the metadata file were left as-is (today) then the linker
would generate an error saying it can't read the file. The previous
solution was to alter the rlib just before linking, creating a new
archive in a temporary directory which has the metadata file removed.
This problem from before this commit is now removed if the metadata file
is stored in an object file that the linker can read. The only caveat we
have to take care of is to ensure that the linker never actually
includes the contents of the object file into the final output. We apply
similar tricks as the `.llvmbc` bytecode sections to do this.
This involved changing the metadata loading code a bit, namely updating
some of the LLVM C APIs used to use non-deprecated ones and fiddling
with the lifetimes a bit to get everything to work out. Otherwise though
this isn't intended to be a functional change really, only that metadata
is stored differently in archives now.
This should end up fixing #83730 because by default dylibs will no
longer have their rlib dependencies "altered" meaning that
split-debuginfo will continue to have valid paths pointing at the
original rlibs. (note that we still "alter" rlibs if LTO is enabled to
remove Rust object files and we also "alter" for the #[link(cfg)]
feature, but that's rarely used).
Closes#83730
Support Android ndk versions `r23-beta3` and up
Since android ndk version `r23-beta3`, `libgcc` has been replaced with `libunwind`. This moves the linking of `libgcc`/`libunwind` into the `unwind` crate where we check if the system compiler can find `libunwind` and fall back to `libgcc` if needed.
Rollup of 8 pull requests
Successful merges:
- #85717 (Document `From` impls for cow.rs)
- #85850 (Remove unused feature gates)
- #85888 (Fix typo in internal documentation for `TrustedRandomAccess`)
- #85889 (Restoring the `num_def_ids` function in the CStore API )
- #85899 (jsondocck small cleanup)
- #85937 (Fix bad suggestions for code from proc_macro)
- #85963 (Show `::{{constructor}}` in std::any::type_name().)
- #85977 (Fix linkcheck script from getting out of sync.)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
jsondocck small cleanup
updated `shlex` (there was some fix 6db4704fca)
replaced `lazy_static` with `once_cell`
removed `serde` direct dependency (`serde_json` will pull it)
Reduce the amount of untracked state in TyCtxt
Access to untracked global state may generate instances of #84970.
The GlobalCtxt contains the lowered HIR, the resolver outputs and interners.
By wrapping the resolver inside a query, we make sure those accesses are properly tracked.
As a no_hash query, all dependent queries essentially become `eval_always`,
what they should have been from the beginning.
Don't panic when failing to initialize incremental directory.
This removes a panic when rustc fails to initialize the incremental directory. This can commonly happen on various filesystems that don't support locking (often various network filesystems). Panics can be confusing and scary, and there are already plenty of issues reporting this.
This has been panicking since 1.22 due to I think #44502 which was a major rework of how things work. Previously, things were simpler and the [`load_dep_graph`](https://github.com/rust-lang/rust/blob/1.21.0/src/librustc_incremental/persist/load.rs#L43-L65) function would emit an error and then continue on without panicking. With 1.22, [`load_dep_graph`](https://github.com/rust-lang/rust/blob/1.22.0/src/librustc_incremental/persist/load.rs#L44) was changed so that it assumes it can load the data without errors. Today, the problem is that it calls [`prepare_session_directory`](fbf1b1a719/compiler/rustc_interface/src/passes.rs (L175-L179)) and then immediately calls `garbage_collect_session_directories` which will panic since the session is `IncrCompSession::NotInitialized`.
The solution here is to have `prepare_session_directory` return an error that must be handled so that compilation stops if it fails.
Some other options:
* Ignore directory lock failures.
* Print a warning on directory lock failure, but otherwise continue with incremental enabled.
* Print a warning on directory lock failure, and disable incremental.
* Provide a different locking mechanism.
Cargo ignores lock errors if locking is not supported, so that would be a precedent for the first option. These options would require quite a bit more changes, but I'm happy to entertain any of them, as I think they all have valid justifications.
There is more discussion on the many issues where this is reported: #49773, #59224, #66513, #76251. I'm not sure if this can be considered closing any of those, though, since I think there is some value in discussing if there is a way to avoid the error altogether. But I think it would make sense to at least close all but one to consolidate them.
Update cc
Recent commits have improved `cc`'s finding of MSVC tools on Windows. In particular it should help to address these issues: #83043 and #43468
This avoids the following warning:
```
warning: trailing semicolon in macro used in expression position
--> /home/joshua/.local/lib/cargo/registry/src/github.com-1ecc6299db9ec823/log-0.4.11/src/macros.rs:152:45
|
147 | / macro_rules! debug {
148 | | (target: $target:expr, $($arg:tt)+) => (
149 | | log!(target: $target, $crate::Level::Debug, $($arg)+);
150 | | );
151 | | ($($arg:tt)+) => (
152 | | log!($crate::Level::Debug, $($arg)+);
| | ^
153 | | )
154 | | }
| |_- in this expansion of `debug!`
|
::: src/tools/rustfmt/src/modules/visitor.rs:36:23
|
36 | Err(e) => debug!("{}", e),
| --------------- in this macro invocation
|
= note: requested on the command line with `-W semicolon-in-expressions-from-macros`
= warning: this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!
= note: for more information, see issue #79813 <https://github.com/rust-lang/rust/issues/79813>
```
Use the object crate for metadata reading
This allows sharing the metadata reader between cg_llvm, cg_clif and other codegen backends.
This is not currently useful for rlib reading with cg_spirv ([rust-gpu](https://github.com/EmbarkStudios/rust-gpu/)) as it uses tar rather than ar as .rlib format, but it is useful for dylib reading required for loading proc macros. (cc `@eddyb)`
The object crate is already trusted as dependency of libstd through backtrace. As far as I know it supports reading all object file formats used by targets for which we support rust dylibs with crate metadata, but I am not certain. If this happens to not be the case, I could keep using LLVM for reading dylib metadata.
Marked as WIP for a perf run and as it is based on #83637.
Implement `x.py test src/tools/clippy --bless`
- Add clippy_dev to the rust workspace
Before, it would give an error that it wasn't either included or
excluded from the workspace:
```
error: current package believes it's in a workspace when it's not:
current: /home/joshua/rustc/src/tools/clippy/clippy_dev/Cargo.toml
workspace: /home/joshua/rustc/Cargo.toml
this may be fixable by adding `src/tools/clippy/clippy_dev` to the `workspace.members` array of the manifest located at: /home/joshua/rustc/Cargo.toml
Alternatively, to keep it out of the workspace, add the package to the `workspace.exclude` array, or add an empty `[workspace]` table to the package's manifest.
```
- Change clippy's copy of compiletest not to special-case
rust-lang/rust. Using OUT_DIR confused `clippy_dev` and it couldn't find
the test outputs. This is one of the reasons why `cargo dev bless` used
to silently do nothing (the others were that `CARGO_TARGET_DIR` and
`PROFILE` weren't set appropriately).
- Run clippy_dev on test failure
I tested this by removing a couple lines from a stderr file, and they
were correctly replaced.
- Fix clippy_dev warnings
Update grab bag
This PR slides a bunch of crate versions forward until suddenly a bunch of deps fall out of the tree!
In doing so this mostly picks up a version bump in the `redox_users` crate which makes most of the features default to optional.
crossbeam-utils 0.7 => 0.8.3 (where applicable)
https://github.com/crossbeam-rs/crossbeam/blob/master/crossbeam-utils/CHANGELOG.md
directories 3.0.1 => 3.0.2
ignore 0.4.16 => 0.4.17
tempfile 3.0.5 => tempfile 3.2
Removes constant_time_eq from deps exceptions
Removes arrayref from deps exceptions
And also removes:
- blake2b_simd
- const_fn (the package, not the feature)
- constant_time_eq
- redox_users 0.3.4
- rust-argon2
- Add clippy_dev to the rust workspace
Before, it would give an error that it wasn't either included or
excluded from the workspace:
```
error: current package believes it's in a workspace when it's not:
current: /home/joshua/rustc/src/tools/clippy/clippy_dev/Cargo.toml
workspace: /home/joshua/rustc/Cargo.toml
this may be fixable by adding `src/tools/clippy/clippy_dev` to the `workspace.members` array of the manifest located at: /home/joshua/rustc/Cargo.toml
Alternatively, to keep it out of the workspace, add the package to the `workspace.exclude` array, or add an empty `[workspace]` table to the package's manifest.
```
- Change clippy's copy of compiletest not to special-case
rust-lang/rust. Using OUT_DIR confused `clippy_dev` and it couldn't find
the test outputs. This is one of the reasons why `cargo dev bless` used
to silently do nothing (the others were that `CARGO_TARGET_DIR` and
`PROFILE` weren't set appropriately).
- Run clippy_dev on test failure
I tested this by removing a couple lines from a stderr file, and they
were correctly replaced.
- Fix clippy_dev warnings
Use arrayvec 0.7, drop smallvec 0.6
With the arrival of min const generics, many alt-vec libraries have
updated to use it in some way and arrayvec is no exception. Use the
latest with minor refactoring.
Also, rustc_workspace_hack is the only user of smallvec 0.6 in the
entire tree, so drop it.
With the arrival of min const generics, many alt-vec libraries have
updated to use it in some way and arrayvec is no exception. Use the
latest with minor refactoring.
Also, rustc_workspace_hack is the only user of smallvec 0.6 in the
entire tree, so drop it.
Update stdarch submodule (to before it switched to const generics)
https://github.com/rust-lang/rust/pull/83278#issuecomment-812389823: This unblocks #82539.
Major changes:
- More AVX-512 intrinsics.
- More ARM & AArch64 NEON intrinsics.
- Updated unstable WASM intrinsics to latest draft standards.
- std_detect is now a separate crate instead of a submodule of std.
I double-checked and the first use of const generics looks like 8d5017861e, which isn't included in this PR.
r? `@Amanieu`
This also includes a cherry-pick of
ec1461905b
and https://github.com/rust-lang/stdarch/pull/1108 to fix a build
failure.
It also adds a re-export of various macros to the crate root of libstd -
previously they would show up automatically because std_detect was defined
in the same crate.
Bump libc dependency of std to 0.2.93
Update `libc` dependency of `std` to the latest version. That allows to consume the https://github.com/rust-lang/libc/pull/2131 fix and fix build for the `mipsel-unknown-linux-uclibc` target.
r? `@JohnTitor`
Add an Mmap wrapper to rustc_data_structures
This wrapper implements StableAddress and falls back to directly reading the file on wasm32.
Taken from #83640, which I will close due to the perf regression.
Remove/replace some outdated crates from the dependency tree
- Remove `cloudabi` by updating `parking_lot` to 0.11.1.
- Replace `packed_simd` with `packed_simd2` by updating `bytecount` to 0.6.2.
Find more invalid doc attributes
- Lint on `#[doc(123)]`, `#[doc("hello")]`, etc.
- Lint every attribute; e.g., will now report two warnings for `#[doc(foo, bar)]`
- Add hyphen to "crate level"
- Display paths like `#[doc(foo::bar)]` correctly instead of as an empty string
Update to rustc-rayon 0.3.1
This pulls in rust-lang/rustc-rayon#8 to fix#81425. (h/t `@ammaraskar)`
That revealed weak constraints on `rustc_arena::DropArena`, because its
`DropType` was holding type-erased raw pointers to generic `T`. We can
implement `Send` for `DropType` (under `cfg(parallel_compiler)`) by
requiring all `T: Send` before they're type-erased.
This pulls in rust-lang/rustc-rayon#8 to fix#81425. (h/t @ammaraskar)
That revealed weak constraints on `rustc_arena::DropArena`, because its
`DropType` was holding type-erased raw pointers to generic `T`. We can
implement `Send` for `DropType` (under `cfg(parallel_compiler)`) by
requiring all `T: Send` before they're type-erased.
Update Cargo
Output of `git log --oneline c68432f1e..970bc67c3`:
970bc67c3 (HEAD, origin/master, origin/auto-cargo, origin/HEAD) Auto merge of #9243 - wickerwaka:configurable-env-doc, r=ehuss
4d7a29b75 Document the configurable-env usntable option
f7a7a3f91 Auto merge of #9229 - alexcrichton:fix-borrow-mut, r=ehuss
3f2ece7a9 Fix a `BorrowMut` error when stdout is closed
7441e8c23 Auto merge of #8825 - Aaron1011:feature/report-future-incompat, r=ehuss
139ed73f5 Add future-incompat tracking issue number.
9ea350368 Fix some minor formatting issues.
f03d47ce4 Address review comments
6177c6584 Implement future incompatibility report support
c69409658 Auto merge of #9022 - nagisa:nagisa/manifest_path, r=alexcrichton
548300b20 Add the path to the manifest in json output
99e714c05 Auto merge of #9230 - kornelski:nobinaries, r=alexcrichton
61a31bc5f Auto merge of #9236 - kornelski:track-assert, r=Eh2406
3f7f0942c track_caller on custom assert functions
6977dee10 Explain `cargo install` is not for libraries
e4aebf0a0 Auto merge of #9231 - joshtriplett:clear-to-eol-if-color, r=alexcrichton
b219f0eb7 Auto merge of #9181 - jyn514:computer-says-no, r=ehuss
0b1816578 Remove unhelpful link to Cargo book
ea46f5ce3 Use ANSI clear-to-EOL if color is force-enabled
a6394bcc1 Remove unnecessary `config` argument to `Features::add`
3a86ecf2d Fix TODO about nightly features
09677c83c Be less unix-centric in error messages
ecfdced0d Fix test that assumed tests always were run on the stable channel
eba541994 Update comment in build_script_env
a5720117f Make `nightly_features_allowed` a field instead of a function
169b09ce7 Compute `enable_nightly_features` once instead of on each call
8fc86e155 Remove unused thread_locals
4b096beae Fix `masquerade_as_nightly_cargo` in work threads
e56417c8c Suggest RUSTC_BOOTSTRAP=crate instead of RUSTC_BOOTSTRAP=1
418129dae Downgrade error to a warning when `RUSTC_BOOTSTRAP` is set or this is the nightly channel
6c422a2c0 Restrict RUSTC_BOOTSTRAP in build.rs
Bump libc dependency of std to 0.2.88.
This PR bumps the `libc` dependency of `std` to 0.2.88. This will fix `TcpListener::accept` for Android on x86 platforms (31a2777d8f).
This will really finally fix https://github.com/rust-lang/rust/issues/82400 for the main branch :)
r? ``@JohnTitor``
Update minifier dependency version
Very small PR simply upgrading the minifier-rs version we use in rustdoc.
Some details might be useful: there were a few bug fixes and a lot of cleanup/code improvements.
r? `@camelid`
Output of `git log --oneline c68432f1e..970bc67c3`:
970bc67c3 (HEAD, origin/master, origin/auto-cargo, origin/HEAD) Auto merge of #9243 - wickerwaka:configurable-env-doc, r=ehuss
4d7a29b75 Document the configurable-env usntable option
f7a7a3f91 Auto merge of #9229 - alexcrichton:fix-borrow-mut, r=ehuss
3f2ece7a9 Fix a `BorrowMut` error when stdout is closed
7441e8c23 Auto merge of #8825 - Aaron1011:feature/report-future-incompat, r=ehuss
139ed73f5 Add future-incompat tracking issue number.
9ea350368 Fix some minor formatting issues.
f03d47ce4 Address review comments
6177c6584 Implement future incompatibility report support
c69409658 Auto merge of #9022 - nagisa:nagisa/manifest_path, r=alexcrichton
548300b20 Add the path to the manifest in json output
99e714c05 Auto merge of #9230 - kornelski:nobinaries, r=alexcrichton
61a31bc5f Auto merge of #9236 - kornelski:track-assert, r=Eh2406
3f7f0942c track_caller on custom assert functions
6977dee10 Explain `cargo install` is not for libraries
e4aebf0a0 Auto merge of #9231 - joshtriplett:clear-to-eol-if-color, r=alexcrichton
b219f0eb7 Auto merge of #9181 - jyn514:computer-says-no, r=ehuss
0b1816578 Remove unhelpful link to Cargo book
ea46f5ce3 Use ANSI clear-to-EOL if color is force-enabled
a6394bcc1 Remove unnecessary `config` argument to `Features::add`
3a86ecf2d Fix TODO about nightly features
09677c83c Be less unix-centric in error messages
ecfdced0d Fix test that assumed tests always were run on the stable channel
eba541994 Update comment in build_script_env
a5720117f Make `nightly_features_allowed` a field instead of a function
169b09ce7 Compute `enable_nightly_features` once instead of on each call
8fc86e155 Remove unused thread_locals
4b096beae Fix `masquerade_as_nightly_cargo` in work threads
e56417c8c Suggest RUSTC_BOOTSTRAP=crate instead of RUSTC_BOOTSTRAP=1
418129dae Downgrade error to a warning when `RUSTC_BOOTSTRAP` is set or this is the nightly channel
6c422a2c0 Restrict RUSTC_BOOTSTRAP in build.rs
Update tracing to 0.1.25
* Update tracing from 0.1.18 to 0.1.25
* Update tracing-subscriber from 0.2.13 to 0.2.16
* Update tracing-tree from 0.1.6 to 0.1.8
* Add pin-project-lite to the list of allowed dependencies (it is now a direct dependency of tracing).
Always compile rustdoc with debug logging enabled when `download-rustc` is set
Previously, logging at DEBUG or below would always be silenced, because
rustc compiles tracing with the `static_max_level_info` feature. That
makes sense for release artifacts, but not for developing rustdoc.
Instead, this compiles two different versions of tracing: one in the
release artifacts, distributed in the sysroot, and a new version
compiled by rustdoc. Since `rustc_driver` is always linked to the
version of sysroot, this copy/pastes `init_env_logging` into rustdoc.
To avoid compiling an unnecessary version of tracing when
`download-rustc` isn't set, this adds a new `using-ci-artifacts`
feature for rustdoc and passes that feature in bootstrap.
Addresses https://github.com/rust-lang/rust/issues/81930. This builds on https://github.com/rust-lang/rust/pull/79540.
r? `@Mark-Simulacrum`
Previously, logging at DEBUG or below would always be silenced, because
rustc compiles tracing with the `static_max_level_info` feature. That
makes sense for release artifacts, but not for developing rustdoc.
Instead, this compiles two different versions of tracing: one in the
release artifacts, distributed in the sysroot, and a new version
compiled by rustdoc. Since `rustc_driver` is always linked to the
version of sysroot, this copy/pastes `init_env_logging` into rustdoc.
The builds the second version of tracing unconditionally; see the code
for details on why.
Update measureme dependency to the latest version
This version adds the ability to use `rdpmc` hardware-based performance
counters instead of wall-clock time for measuring duration. This also
introduces a dependency on the `perf-event-open-sys` crate on Linux
which is used when using hardware counters.
r? ```@oli-obk```
Replace const_cstr with cstr crate
This PR replaces the `const_cstr` macro inside `rustc_data_structures` with `cstr` macro from [cstr](https://crates.io/crates/cstr) crate.
The two macros basically serve the same purpose, which is to generate `&'static CStr` from a string literal. `cstr` is better because it validates the literal at compile time, while the existing `const_cstr` does it at runtime when `debug_assertions` is enabled. In addition, the value `cstr` generates can be used in constant context (which is seemingly not needed anywhere currently, though).
This version adds the ability to use `rdpmc` hardware-based performance
counters instead of wall-clock time for measuring duration. This also
introduces a dependency on the `perf-event-open-sys` crate on Linux
which is used when using hardware counters.
Make the `Query` enum a simple struct.
A lot of code in `rustc_query_system` is generic over it, only to encode an exceptional error case: query cycles.
The delayed computations are now done at cycle detection.
rustc_codegen_ssa: tune codegen according to available concurrency
This change tunes ahead-of-time codegening according to the amount of
concurrency available, rather than according to the number of CPUs on
the system. This can lower memory usage by reducing the number of
compiled LLVM modules in memory at once, particularly across several
rustc instances.
Previously, each rustc instance would assume that it should codegen
ahead of time to meet the demand of number-of-CPUs workers. But often, a
rustc instance doesn't have nearly that much concurrency available to
it, because the concurrency availability is split, via the jobserver,
across all active rustc instances spawned by the driving cargo process,
and is further limited by the `-j` flag argument. Therefore, each rustc
might have had several times the number of LLVM modules in memory than
it really needed to meet demand. If the modules were large, the effect
on memory usage would be noticeable.
With this change, the required amount of ahead-of-time codegen scales up
with the actual number of workers running within a rustc instance. Note
that the number of workers running can be less than the actual
concurrency available to a rustc instance. However, if more concurrency
is actually available, workers are spun up quickly as job tokens are
acquired, and the ahead-of-time codegen scales up quickly as well.
This change tunes ahead-of-time codegening according to the amount of
concurrency available, rather than according to the number of CPUs on
the system. This can lower memory usage by reducing the number of
compiled LLVM modules in memory at once, particularly across several
rustc instances.
Previously, each rustc instance would assume that it should codegen
ahead of time to meet the demand of number-of-CPUs workers. But often, a
rustc instance doesn't have nearly that much concurrency available to
it, because the concurrency availability is split, via the jobserver,
across all active rustc instances spawned by the driving cargo process,
and is further limited by the `-j` flag argument. Therefore, each rustc
might have had several times the number of LLVM modules in memory than
it really needed to meet demand. If the modules were large, the effect
on memory usage would be noticeable.
With this change, the required amount of ahead-of-time codegen scales up
with the actual number of workers running within a rustc instance. Note
that the number of workers running can be less than the actual
concurrency available to a rustc instance. However, if more concurrency
is actually available, workers are spun up quickly as job tokens are
acquired, and the ahead-of-time codegen scales up quickly as well.
Borrowck: refactor visited map to a bitset
This PR refactors `Borrows` and the `precompute_borrows_out_of_scope` function so that this initial phase has a much reduced memory pressure. This is achieved by reducing what is stored on the heap, and also reusing heap memory as much as possible.
This allows a build system to indicate a location in its own dependency
specification files (eg Cargo's `Cargo.toml`) which can be reported
along side any unused crate dependency.
This supports several types of location:
- 'json' - provide some json-structured data, which is included in the json diagnostics
in a `tool_metadata` field
- 'raw' - emit the provided string into the output. This also appears as a json string in
`tool_metadata`.
If no `--extern-location` is explicitly provided then a default json entry of the form
`"tool_metadata":{"name":<cratename>,"path":<cratepath>}` is emitted.
For better throughput during parallel processing by LLVM, we used to sort
CGUs largest to smallest. This would lead to better thread utilization
by, for example, preventing a large CGU from being processed last and
having only one LLVM thread working while the rest remained idle.
However, this strategy would lead to high memory usage, as it meant the
LLVM-IR for all of the largest CGUs would be resident in memory at once.
Instead, we can compromise by ordering CGUs such that the largest and
smallest are first, second largest and smallest are next, etc. If there
are large size variations, this can reduce memory usage significantly.
Add AArch64 big-endian and ILP32 targets
This PR adds 3 new AArch64 targets:
- `aarch64_be-unknown-linux-gnu`
- `aarch64-unknown-linux-gnu_ilp32`
- `aarch64_be-unknown-linux-gnu_ilp32`
It also fixes some ABI issues on big-endian ARM and AArch64.
Upgrade Chalk
~~Blocked on rust-lang/chalk#670~~
~~Now blocked on rust-lang/chalk#680 and release~~
In addition to the straight upgrade, I also tried to fix some tests by properly returning variables and max universes in the solution. Unfortunately, this actually triggers the same perf problem that rustc traits code runs into in `canonicalizer`. Not sure what the root cause of this problem is, or why it's supposed to be solved in chalk.
r? ```@nikomatsakis```
