Remove num_cpus dependency from bootstrap, build-manifest and rustc_s…
…ession
`std::threads::available_parallelism` was stabilized in rust 1.59.
r? ```````````````````````````@Mark-Simulacrum```````````````````````````
autocfg 1.1.0 makes it so that rustflags from the build are correctly
passed to the compiler probes, which in turn means those probes more
accurately reflect the outer build conditions. This is particularly
important if rustflags includes _required_ `-Clink-arg=` flags without
which builds will fail, as older versions of `autocfg` will then fail
the probe and erroneously report the probed feature as unavailable.
See also
https://github.com/rust-lang/rust/issues/94007#issuecomment-1040668261
debuginfo: Simplify TypeMap used during LLVM debuginfo generation.
This PR simplifies the TypeMap that is used in `rustc_codegen_llvm::debuginfo::metadata`. It was unnecessarily complicated because it was originally implemented when types were not yet normalized before codegen. So it did it's own normalization and kept track of multiple unnormalized types being mapped to a single unique id.
This PR is based on https://github.com/rust-lang/rust/pull/93503, which is not merged yet.
The PR also removes the arena used for allocating string ids and instead uses `InlinableString` from the [inlinable_string](https://crates.io/crates/inlinable_string) crate. That might not be the best choice, since that crate does not seem to be very actively maintained. The [flexible-string](https://crates.io/crates/flexible-string) crate would be an alternative.
r? `@ghost`
The previous implementation was written before types were properly
normalized for code generation and had to assume a more complicated
relationship between types and their debuginfo -- generating separate
identifiers for debuginfo nodes that were based on normalized types.
Since types are now already normalized, we can use them as identifiers
for debuginfo nodes.
removing architecture requirements for RustyHermit
RustHermit and HermitCore is able to run on aarch64 and x86_64. In the future these operating systems will also support RISC-V. Consequently, the dependency to a specific target should be removed.
The build process of `hermit-abi` fails if the architecture isn't supported.
RustHermit and HermitCore is able to run on aarch64 and x86_64.
In the future these operating systems will also support RISC-V.
Consequently, the dependency to a specific target should be removed.
Building hermit-abi fails if the architecture isn't supported.
Drop time dependency from bootstrap
This was only used for the inclusion of 'current' dates into our manpages, but
it is not clear that this is practically necessary. The manpage is essentially
never updated, and so we can likely afford to keep a manual date in these files.
It also seems possible to just omit it, but that may cause other tools trouble,
so avoid doing that for now.
This is largely done to reduce bootstrap complexity; the time crate is not particularly
small and in #92480 would have started pulling in num-threads, which does runtime
thread count detection. I would prefer to avoid that, so filing this to just drop the nearly
unused dependency entirely.
r? `@pietroalbini`
This was only used for the inclusion of 'current' dates into our manpages, but
it is not clear that this is practically necessary. The manpage is essentially
never updated, and so we can likely afford to keep a manual date in these files.
It also seems possible to just omit it, but that may cause other tools trouble,
so avoid doing that for now.
remove unused `jemallocator` crate
When it was noticed that the rustc binary wasn't actually using jemalloc via `#[global_allocator]` and that was removed, the dependency remained.
Tests pass locally with a `jemalloc = true` build, but I'll trigger a try build to ensure I haven't missed an edge-case somewhere.
r? ```@ghost``` until that completes
Bump libc and fix remove_dir_all on Fuchsia after CVE fix
With the previous `is_dir` impl, we would attempt to unlink
a directory in the None branch, but Fuchsia supports returning
ENOTEMPTY from unlinkat() without the AT_REMOVEDIR flag because
we don't currently differentiate unlinking files and directories
by default.
On the Fuchsia side I've opened https://fxbug.dev/92273 to discuss
whether this is the correct behavior, but it doesn't seem like
addressing the error code is necessary to make our tests happy.
Depends on https://github.com/rust-lang/libc/pull/2654 since we
apparently haven't needed to reference DT_UNKNOWN before this.
With the previous `is_dir` impl, we would attempt to unlink
a directory in the None branch, but Fuchsia supports returning
ENOTEMPTY from unlinkat() without the AT_REMOVEDIR flag because
we don't currently differentiate unlinking files and directories
by default.
On the Fuchsia side I've opened https://fxbug.dev/92273 to discuss
whether this is the correct behavior, but it doesn't seem like
addressing the error code is necessary to make our tests happy.
Updates std's libc crate to include DT_UNKNOWN for Fuchsia.
Update some rustc dependencies to deduplicate them
This PR updates `rand` and `itertools` in rustc (not the whole workspace) in order to deduplicate them (and hopefully slightly improve compile times).
~~Currently, `object` is still duplicated, but https://github.com/rust-lang/thorin/pull/15 and updating `thorin` in the future will remove the use of version 0.27.~~ Update: Thorin 0.2 has now been released, and this PR updates `rustc_codegen_ssa` to use it and deduplicate the `object` crate.
There's a final tiny rustc dependency, `cfg-if`, which will be left: as both versions 0.1.x and 1.0 looked to be heavily depended on, they will require a few cascading updates to be removed.
Introduce drop range tracking to generator interior analysis
This PR addresses cases such as this one from #57478:
```rust
struct Foo;
impl !Send for Foo {}
let _: impl Send = || {
let guard = Foo;
drop(guard);
yield;
};
```
Previously, the `generator_interior` pass would unnecessarily include the type `Foo` in the generator because it was not aware of the behavior of `drop`. We fix this issue by introducing a drop range analysis that finds portions of the code where a value is guaranteed to be dropped. If a value is dropped at all suspend points, then it is no longer included in the generator type. Note that we are using "dropped" in a generic sense to include any case in which a value has been moved. That is, we do not only look at calls to the `drop` function.
There are several phases to the drop tracking algorithm, and we'll go into more detail below.
1. Use `ExprUseVisitor` to find values that are consumed and borrowed.
2. `DropRangeVisitor` uses consume and borrow information to gather drop and reinitialization events, as well as build a control flow graph.
3. We then propagate drop and reinitialization information through the CFG until we reach a fix point (see `DropRanges::propagate_to_fixpoint`).
4. When recording a type (see `InteriorVisitor::record`), we check the computed drop ranges to see if that value is definitely dropped at the suspend point. If so, we skip including it in the type.
## 1. Use `ExprUseVisitor` to find values that are consumed and borrowed.
We use `ExprUseVisitor` to identify the places where values are consumed. We track both the `hir_id` of the value, and the `hir_id` of the expression that consumes it. For example, in the expression `[Foo]`, the `Foo` is consumed by the array expression, so after the array expression we can consider the `Foo` temporary to be dropped.
In this process, we also collect values that are borrowed. The reason is that the MIR transform for generators conservatively assumes anything borrowed is live across a suspend point (see `rustc_mir_transform::generator::locals_live_across_suspend_points`). We match this behavior here as well.
## 2. Gather drop events, reinitialization events, and control flow graph
After finding the values of interest, we perform a post-order traversal over the HIR tree to find the points where these values are dropped or reinitialized. We use the post-order index of each event because this is how the existing generator interior analysis refers to the position of suspend points and the scopes of variables.
During this traversal, we also record branching and merging information to handle control flow constructs such as `if`, `match`, and `loop`. This is necessary because values may be dropped along some control flow paths but not others.
## 3. Iterate to fixed point
The previous pass found the interesting events and locations, but now we need to find the actual ranges where things are dropped. Upon entry, we have a list of nodes ordered by their position in the post-order traversal. Each node has a set of successors. For each node we additionally keep a bitfield with one bit per potentially consumed value. The bit is set if we the value is dropped along all paths entering this node.
To compute the drop information, we first reverse the successor edges to find each node's predecessors. Then we iterate through each node, and for each node we set its dropped value bitfield to the intersection of all incoming dropped value bitfields.
If any bitfield for any node changes, we re-run the propagation loop again.
## 4. Ignore dropped values across suspend points
At this point we have a data structure where we can ask whether a value is guaranteed to be dropped at any post order index for the HIR tree. We use this information in `InteriorVisitor` to check whether a value in question is dropped at a particular suspend point. If it is, we do not include that value's type in the generator type.
Note that we had to augment the region scope tree to include all yields in scope, rather than just the last one as we did before.
r? `@nikomatsakis`
All tests pass now! The issue was that we weren't handling all edges
correctly, but now they are handled consistently.
This includes code to dump a graphviz file for the CFG we built for drop
tracking.
Also removes old DropRanges tests.
This adds support for branching and merging control flow and uses this
to correctly handle the case where a value is dropped in one branch of
an if expression but not another.
There are other cases we need to handle, which will come in follow up
patches.
Issue #57478
ProjectionPredicate should be able to handle both associated types and consts so this adds the
first step of that. It mainly just pipes types all the way down, not entirely sure how to handle
consts, but hopefully that'll come with time.
rustdoc: remove hand-rolled isatty
This PR replaces bindings to the platform-specific isatty APIs with the `isatty` crate, as done elsewhere in the repository.
Update rayon and rustc-rayon
This updates rayon for various tools and rustc-rayon for the compiler's parallel mode.
- rayon v1.3.1 -> v1.5.1
- rayon-core v1.7.1 -> v1.9.1
- rustc-rayon v0.3.1 -> v0.3.2
- rustc-rayon-core v0.3.1 -> v0.3.2
... and indirectly, this updates all of crossbeam-* to their latest versions.
Fixes#92677 by removing crossbeam-queue, but there's still a lingering question about how tidy discovers "runtime" dependencies. None of this is truly in the standard library's dependency tree at all.
Update cargo
6 commits in 358e79fe56fe374649275ca7aebaafd57ade0e8d..06b9d31743210b788b130c8a484c2838afa6fc27
2022-01-04 18:39:45 +0000 to 2022-01-11 23:47:29 +0000
- Port cargo to clap3 (rust-lang/cargo#10265)
- feat: support rustflags per profile (rust-lang/cargo#10217)
- Make bors ignore the PR template so it doesn't end up in merge messages (rust-lang/cargo#10267)
- Be resilient to most IO error and filesystem loop while walking dirs (rust-lang/cargo#10214)
- Remove the option to disable pipelining (rust-lang/cargo#10258)
- Always ask rustc for messages about artifacts, and always process them (rust-lang/cargo#10255)
Make rlib metadata strip works with MIPSr6 architecture
Because MIPSr6 has many differences with previous MIPSr2 arch, the previous rlib metadata stripping code in `rustc_codegen_ssa` is only for MIPSr2/r3/r5 (which share the same elf e_flags).
This commit fixed this problem. It makes `rustc_codegen_ssa` happy when compiling rustc for MIPSr6 target or hosts.
e_flags REF: e356027016/llvm/include/llvm/BinaryFormat/ELF.h (L562)
`thorin` is a Rust implementation of a DWARF packaging utility that
supports reading DWARF objects from archive files (i.e. rlibs) and
therefore is better suited for integration into rustc.
Signed-off-by: David Wood <david.wood@huawei.com>
Extract init_env_logger to crate
I've been doing some work on rustc_ast_pretty using an out-of-tree main.rs and Cargo.toml with the following:
```toml
[dependencies]
rustc_ast = { path = "../rust/compiler/rustc_ast" }
rustc_ast_pretty = { path = "../rust/compiler/rustc_ast_pretty" }
rustc_span = { path = "../rust/compiler/rustc_span" }
```
Rustc_ast_pretty helpfully uses `tracing::debug!` but I found that in order to enable the debug output, my test crate must depend on rustc_driver which is an enormously bigger dependency than what I have been using so far, and slows down iteration time because an enormous dependency tree between rustc_ast and rustc_driver must now be rebuilt after every ast change.
I pulled out the tracing initialization to a new minimal rustc_log crate so that projects depending on the other rustc crates, like rustc_ast_pretty, can access the `debug!` messages in them without building all the rest of rustc.
The task of the macro is simple enough that a decl macro is almost ten
times shorter than the original proc macro. The proc macro is 159 lines
while the decl macro is just 18 lines.
This reduces the amount of dependencies of rustbuild from 45 to 37. It
also slight reduces compilation time from 47s to 44s for debug builds.
Store liveness in interval sets for region inference
On the 100,000 line test case from https://github.com/rust-lang/rust/issues/90445, this reduces memory usage from 35 GB to 444 MB at peak (based on DHAT results, though with regular malloc), and yields a 9.4x speedup, with wall time going from 14.5 seconds to 1.5s. Performance results show that for the majority of real-world code this has little to no impact, but it's expected to generally scale better for auto-generated functions and other cases which stress this area of the compiler, as results on #90445 illustrate.
There may also be further room for improvement in future PRs making use of this data structures benefits over raw bitsets (which, at some level, are a less perfect fit for representing liveness, which is almost always composed of contiguous ranges, not point locations).
Fixes#90445.
This is a compact, fast storage for variable-sized sets, typically consisting of
larger ranges. It is less efficient than a bitset if ranges are both small and
the domain size is small, but will still perform acceptably. With enormous
domain sizes and large ranges, the interval set performs much better, as it can
be much more densely packed in memory than the uncompressed bit set alternative.
Update chalk to 0.75.0
- Compute flags in `intern_ty`
- Remove `tracing-serde` from `PERMITTED_DEPENDENCIES`
- Bump `tracing-tree` to 0.2.0
- Bump `tracing-subscriber` to 0.3.3
Stabilize asm! and global_asm!
Tracking issue: #72016
It's been almost 2 years since the original [RFC](https://github.com/rust-lang/rfcs/pull/2850) was posted and we're finally ready to stabilize this feature!
The main changes in this PR are:
- Removing `asm!` and `global_asm!` from the prelude as per the decision in #87228.
- Stabilizing the `asm` and `global_asm` features.
- Removing the unstable book pages for `asm` and `global_asm`. The contents are moved to the [reference](https://github.com/rust-lang/reference/pull/1105) and [rust by example](https://github.com/rust-lang/rust-by-example/pull/1483).
- All links to these pages have been removed to satisfy the link checker. In a later PR these will be replaced with links to the reference or rust by example.
- Removing the automatic suggestion for using `llvm_asm!` instead of `asm!` if you're still using the old syntax, since it doesn't work anymore with `asm!` no longer being in the prelude. This only affects code that predates the old LLVM-style `asm!` being renamed to `llvm_asm!`.
- Updating `stdarch` and `compiler-builtins`.
- Updating all the tests.
r? `@joshtriplett`
replace dynamic library module with libloading
This PR deletes the `rustc_metadata::dynamic_lib` module in favor of the popular and better tested [`libloading` crate](https://github.com/nagisa/rust_libloading/).
We don't benefit from `libloading`'s symbol lifetimes since we end up leaking the loaded library in all cases, but the call-sites look much nicer by improving error handling and abstracting away some transmutes. We also can remove `rustc_metadata`'s direct dependencies on `libc` and `winapi`.
This PR also adds an exception for `libloading` (and its license) to tidy, so this will need sign-off from the compiler team.
They are also removed from the prelude as per the decision in
https://github.com/rust-lang/rust/issues/87228.
stdarch and compiler-builtins are updated to work with the new, stable
asm! and global_asm! macros.
