Add more *-unwind ABI variants
The following *-unwind ABIs are now supported:
- "C-unwind"
- "cdecl-unwind"
- "stdcall-unwind"
- "fastcall-unwind"
- "vectorcall-unwind"
- "thiscall-unwind"
- "aapcs-unwind"
- "win64-unwind"
- "sysv64-unwind"
- "system-unwind"
cc `@rust-lang/wg-ffi-unwind`
Currently, it can be `None` if the conversion from `OsString` fails, in
which case all searches will skip over the `SearchPathFile`.
The commit changes things so that the `SearchPathFile` just doesn't get
created in the first place. Same behaviour, but slightly simpler code.
rustdoc: Pre-calculate traits that are in scope for doc links
This eliminates one more late use of resolver (part of #83761).
At early doc link resolution time we go through parent modules of items from the current crate, reexports of items from other crates, trait items, and impl items collected by `collect-intra-doc-links` pass, determine traits that are in scope in each such module, and put those traits into a map used by later rustdoc passes.
r? `@jyn514`
Store a `Symbol` instead of an `Ident` in `AssocItem`
This is the same idea as #92533, but for `AssocItem` instead
of `VariantDef`/`FieldDef`.
With this change, we no longer have any uses of
`#[stable_hasher(project(...))]`
`Decoder` has two impls:
- opaque: this impl is already partly infallible, i.e. in some places it
currently panics on failure (e.g. if the input is too short, or on a
bad `Result` discriminant), and in some places it returns an error
(e.g. on a bad `Option` discriminant). The number of places where
either happens is surprisingly small, just because the binary
representation has very little redundancy and a lot of input reading
can occur even on malformed data.
- json: this impl is fully fallible, but it's only used (a) for the
`.rlink` file production, and there's a `FIXME` comment suggesting it
should change to a binary format, and (b) in a few tests in
non-fundamental ways. Indeed #85993 is open to remove it entirely.
And the top-level places in the compiler that call into decoding just
abort on error anyway. So the fallibility is providing little value, and
getting rid of it leads to some non-trivial performance improvements.
Much of this commit is pretty boring and mechanical. Some notes about
a few interesting parts:
- The commit removes `Decoder::{Error,error}`.
- `InternIteratorElement::intern_with`: the impl for `T` now has the same
optimization for small counts that the impl for `Result<T, E>` has,
because it's now much hotter.
- Decodable impls for SmallVec, LinkedList, VecDeque now all use
`collect`, which is nice; the one for `Vec` uses unsafe code, because
that gave better perf on some benchmarks.
This is the same idea as #92533, but for `AssocItem` instead
of `VariantDef`/`FieldDef`.
With this change, we no longer have any uses of
`#[stable_hasher(project(...))]`
Implement raw-dylib support for windows-gnu
Add support for `#[link(kind = "raw-dylib")]` on windows-gnu targets. Work around binutils's linker's inability to read import libraries produced by LLVM by calling out to the binutils `dlltool` utility to create an import library from a temporary .DEF file; this approach is effectively a slightly refined version of `@mati865's` earlier attempt at this strategy in PR #88801. (In particular, this attempt at this strategy adds support for `#[link_ordinal(...)]` as well.)
In support of #58713.
Implement `#[rustc_must_implement_one_of]` attribute
This PR adds a new attribute — `#[rustc_must_implement_one_of]` that allows changing the "minimal complete definition" of a trait. It's similar to GHC's minimal `{-# MINIMAL #-}` pragma, though `#[rustc_must_implement_one_of]` is weaker atm.
Such attribute was long wanted. It can be, for example, used in `Read` trait to make transitions to recently added `read_buf` easier:
```rust
#[rustc_must_implement_one_of(read, read_buf)]
pub trait Read {
fn read(&mut self, buf: &mut [u8]) -> Result<usize> {
let mut buf = ReadBuf::new(buf);
self.read_buf(&mut buf)?;
Ok(buf.filled_len())
}
fn read_buf(&mut self, buf: &mut ReadBuf<'_>) -> Result<()> {
default_read_buf(|b| self.read(b), buf)
}
}
impl Read for Ty0 {}
//^ This will fail to compile even though all `Read` methods have default implementations
// Both of these will compile just fine
impl Read for Ty1 {
fn read(&mut self, buf: &mut [u8]) -> Result<usize> { /* ... */ }
}
impl Read for Ty2 {
fn read_buf(&mut self, buf: &mut ReadBuf<'_>) -> Result<()> { /* ... */ }
}
```
For now, this is implemented as an internal attribute to start experimenting on the design of this feature. In the future we may want to extend it:
- Allow arbitrary requirements like `a | (b & c)`
- Allow multiple requirements like
- ```rust
#[rustc_must_implement_one_of(a, b)]
#[rustc_must_implement_one_of(c, d)]
```
- Make it appear in rustdoc documentation
- Change the syntax?
- Etc
Eventually, we should make an RFC and make this (or rather similar) attribute public.
---
I'm fairly new to compiler development and not at all sure if the implementation makes sense, but at least it passes tests :)
rustc_metadata: Stop passing `CrateMetadataRef` by reference (step 1)
It's already a (fat) reference.
Double referencing it creates lifetime issues for its methods that want to return iterators.
---
Extracted from https://github.com/rust-lang/rust/pull/92245 for a perf run.
The PR changes a lot of symbol names due to function signature changes, so it's hard to do differential profiling, let's spend some machine time instead.
The field is also renamed from `ident` to `name. In most cases,
we don't actually need the `Span`. A new `ident` method is added
to `VariantDef` and `FieldDef`, which constructs the full `Ident`
using `tcx.def_ident_span()`. This method is used in the cases
where we actually need an `Ident`.
This makes incremental compilation properly track changes
to the `Span`, without all of the invalidations caused by storing
a `Span` directly via an `Ident`.
rustdoc: Introduce a resolver cache for sharing data between early doc link resolution and later passes
The refactoring parts of https://github.com/rust-lang/rust/pull/88679, shouldn't cause any slowdowns.
r? `@jyn514`
rustc_metadata: Optimize and document module children decoding
The first commit limits the item in the `item_children`/`each_child_of_item` query to modules (in name resolution sense) and adds a corresponding assertion.
The `associated_item_def_ids` query collecting children of traits and impls specifically now uses a simplified implementation not decoding unnecessary data instead of `each_child_of_item`, this gives a nice performance improvement.
The second commit does some renaming that clarifies the terminology used for all items in a module vs `use` items only.
By avoiding formatting and allocations in the no-ident case, and by making the span mandatory if the ident exists.
Use the optimized `opt_item_ident` to cleanup `fn each_child_of_item`
Stabilize -Z symbol-mangling-version=v0 as -C symbol-mangling-version=v0
This allows selecting `v0` symbol-mangling without an unstable option. Selecting `legacy` still requires -Z unstable-options.
This does not change the default symbol-mangling-version. See https://github.com/rust-lang/rust/pull/89917 for a pull request changing the default. Rationale, from #89917:
Rust's current mangling scheme depends on compiler internals; loses information about generic parameters (and other things) which makes for a worse experience when using external tools that need to interact with Rust symbol names; is inconsistent; and can contain . characters which aren't universally supported. Therefore, Rust has defined its own symbol mangling scheme which is defined in terms of the Rust language, not the compiler implementation; encodes information about generic parameters in a reversible way; has a consistent definition; and generates symbols that only use the characters A-Z, a-z, 0-9, and _.
Support for the new Rust symbol mangling scheme has been added to upstream tools that will need to interact with Rust symbols (e.g. debuggers).
This pull request allows enabling the new v0 symbol-mangling-version.
See #89917 for references to the implementation of v0, and for references to the tool changes to decode Rust symbols.
Remove effect of `#[no_link]` attribute on name resolution
Previously it hid all non-macro names from other crates.
This has no relation to linking and can change name resolution behavior in some cases (e.g. glob conflicts), in addition to just producing the "unresolved name" errors.
I can kind of understand the possible reasoning behind the current behavior - if you can use names from a `no_link` crates then you can use, for example, functions too, but whether it will actually work or produce link-time errors will depend on random factors like inliner behavior.
(^^^ This is not the actual reason why the current behavior exist, I've looked through git history and it's mostly accidental.)
I think this risk is ok for such an obscure attribute, and we don't need to specifically prevent use of non-macro items from such crates.
(I'm not actually sure why would anyone use `#[no_link]` on a crate, even if it's macro only, if you aware of any use cases, please share. IIRC, at some point it was used for crates implementing custom derives - the now removed legacy ones, not the current proc macros.)
Extracted from https://github.com/rust-lang/rust/pull/91795.
This allows selecting `v0` symbol-mangling without an unstable option.
Selecting `legacy` still requires -Z unstable-options.
Continue supporting -Z symbol-mangling-version for compatibility for
now, but show a deprecation warning for it.
Import `SourceFile`s from crate before decoding foreign `Span`
Fixes#92163Fixes#92014
When writing to the incremental cache, we encode all `Span`s
we encounter, regardless of whether or not their `SourceFile`
comes from the local crate, or from a foreign crate.
When we decode a `Span`, we use the `StableSourceFileId` we encoded
to locate the matching `SourceFile` in the current session. If this
id corresponds to a `SourceFile` from another crate, then we need to
have already imported that `SourceFile` into our current session.
This usually happens automatically during resolution / macro expansion,
when we try to resolve definitions from other crates. In certain cases,
however, we may try to load a `Span` from a transitive dependency
without having ever imported the `SourceFile`s from that crate, leading
to an ICE.
This PR fixes the issue by enconding the `SourceFile`'s `CrateNum`
when we encode a `Span`. During decoding, we call `imported_source_files()`
when we encounter a foreign `CrateNum`, which ensure that all
`SourceFile`s from that crate are imported into the current session.
rustc_metadata: Encode list of all crate's traits into metadata
While working on https://github.com/rust-lang/rust/pull/88679 I noticed that rustdoc is casually doing something quite expensive, something that is used only for error reporting in rustc - collecting all traits from all crates in the dependency tree.
This PR trades some minor extra time spent by metadata encoder in rustc for major gains for rustdoc (and for rustc runs with errors, which execute the `all_traits` query for better diagnostics).
rustc_metadata: Switch crate data iteration from a callback to iterator
The iteration looks more conventional this way, and some allocations are avoided.
rustc_metadata: Merge `get_ctor_def_id` and `get_ctor_kind`
Also avoid decoding the whole `ty::AssocItem` to get a `has_self` flag.
A small optimization and cleanup extracted from https://github.com/rust-lang/rust/pull/89059.
Fixes#92163Fixes#92014
When writing to the incremental cache, we encode all `Span`s
we encounter, regardless of whether or not their `SourceFile`
comes from the local crate, or from a foreign crate.
When we decode a `Span`, we use the `StableSourceFileId` we encoded
to locate the matching `SourceFile` in the current session. If this
id corresponds to a `SourceFile` from another crate, then we need to
have already imported that `SourceFile` into our current session.
This usually happens automatically during resolution / macro expansion,
when we try to resolve definitions from other crates. In certain cases,
however, we may try to load a `Span` from a transitive dependency
without having ever imported the `SourceFile`s from that crate, leading
to an ICE.
This PR fixes the issue by calling `imported_source_files()`
when we encounter a `SourceFile` with a foreign `CrateNum`.
This ensures that all `SourceFile`s from that crate are imported
into the current session.
Remove `SymbolStr`
This was originally proposed in https://github.com/rust-lang/rust/pull/74554#discussion_r466203544. As well as removing the icky `SymbolStr` type, it allows the removal of a lot of `&` and `*` occurrences.
Best reviewed one commit at a time.
r? `@oli-obk`
Previously it hid all non-macro names from other crates.
This has no relation to linking and can change name resolution behavior in some cases (e.g. glob conflicts), in addition to just producing the "unresolved name" errors
By changing `as_str()` to take `&self` instead of `self`, we can just
return `&str`. We're still lying about lifetimes, but it's a smaller lie
than before, where `SymbolStr` contained a (fake) `&'static str`!
replace dynamic library module with libloading
This PR deletes the `rustc_metadata::dynamic_lib` module in favor of the popular and better tested [`libloading` crate](https://github.com/nagisa/rust_libloading/).
We don't benefit from `libloading`'s symbol lifetimes since we end up leaking the loaded library in all cases, but the call-sites look much nicer by improving error handling and abstracting away some transmutes. We also can remove `rustc_metadata`'s direct dependencies on `libc` and `winapi`.
This PR also adds an exception for `libloading` (and its license) to tidy, so this will need sign-off from the compiler team.
Bump rmeta version to fix rustc_serialize ICE
#91407 changed the serialization format which leads to ICEs for nightly users such as #91663 and linked issues. The issue can be solved by running `cargo clean`. But bumping the metadata version should lead to the cached files being discarded, avoiding the issue entirely.
#91407 changed the serialization format which leads to ICEs for nightly users such as #91663 and linked issue.
Bumping the metadata version should lead to the cached files being discarded instead.
Record more artifact sizes during self-profiling.
This PR adds artifact size recording for
- "linked artifacts" (executables, RLIBs, dylibs, static libs)
- object files
- dwo files
- assembly files
- crate metadata
- LLVM bitcode files
- LLVM IR files
- codegen unit size estimates
Currently the identifiers emitted for these are hard-coded as string literals. Is it worth adding constants to https://github.com/rust-lang/measureme/blob/master/measureme/src/rustc.rs instead? We don't do that for query names and the like -- but artifact kinds might be more stable than query names.
Type inference for inline consts
Fixes#78132Fixes#78174Fixes#81857Fixes#89964
Perform type checking/inference of inline consts in the same context as the outer def, similar to what is currently done to closure.
Doing so would require `closure_base_def_id` of the inline const to return the outer def, and since `closure_base_def_id` can be called on non-local crate (and thus have no HIR available), a new `DefKind` is created for inline consts.
The type of the generated anon const can capture lifetime of outer def, so we couldn't just use the typeck result as the type of the inline const's def. Closure has a similar issue, and it uses extra type params `CK, CS, U` to capture closure kind, input/output signature and upvars. I use a similar approach for inline consts, letting it have an extra type param `R`, and then `typeof(InlineConst<[paremt generics], R>)` would just be `R`. In borrowck region requirements are also propagated to the outer MIR body just like it's currently done for closure.
With this PR, inline consts in expression position are quitely usable now; however the usage in pattern position is still incomplete -- since those does not remain in the MIR borrowck couldn't verify the lifetime there. I have left an ignored test as a FIXME.
Some disucssions can be found on [this Zulip thread](https://rust-lang.zulipchat.com/#narrow/stream/260443-project-const-generics/topic/inline.20consts.20typeck).
cc `````@spastorino````` `````@lcnr`````
r? `````@nikomatsakis`````
`````@rustbot````` label A-inference F-inline_const T-compiler
Improve error when an .rlib can't be parsed
This usually describes either an error in the compiler itself or some
sort of IO error. Either way, we should report it to the user rather
than just saying "crate not found".
This only gives an error if the crate couldn't be loaded at all - if the
compiler finds another .rlib or .rmeta file which was valid, it will
continue to compile the crate.
Example output:
```
error[E0785]: found invalid metadata files for crate `foo`
--> bar.rs:3:24
|
3 | println!("{}", foo::FOO_11_49[0]);
| ^^^
|
= warning: failed to parse rlib '/home/joshua/test-rustdoc/libfoo.rlib': Invalid archive extended name offset
```
cc `@ehuss`
This usually describes either an error in the compiler itself or some
sort of IO error. Either way, we should report it to the user rather
than just saying "crate not found".
This only gives an error if the crate couldn't be loaded at all - if the
compiler finds another .rlib or .rmeta file which was valid, it will
continue to compile the crate.
Example output:
```
error[E0785]: found invalid metadata files for crate `foo`
--> bar.rs:3:24
|
3 | println!("{}", foo::FOO_11_49[0]);
| ^^^
|
= warning: failed to parse rlib '/home/joshua/test-rustdoc/libfoo.rlib': Invalid archive extended name offset
```
TraitKind -> Trait
TyAliasKind -> TyAlias
ImplKind -> Impl
FnKind -> Fn
All `*Kind`s in AST are supposed to be enums.
Tuple structs are converted to braced structs for the types above, and fields are reordered in syntactic order.
Also, mutable AST visitor now correctly visit spans in defaultness, unsafety, impl polarity and constness.
Improve and test cross-crate hygiene
- Decode the parent expansion for traits and enums in `rustc_resolve`, this was already being used for resolution in typeck
- Avoid suggesting importing names with def-site hygiene, since it's often not useful
- Add more tests
r? `@petrochenkov`
Adopt let_else across the compiler
This performs a substitution of code following the pattern:
```
let <id> = if let <pat> = ... { identity } else { ... : ! };
```
To simplify it to:
```
let <pat> = ... { identity } else { ... : ! };
```
By adopting the `let_else` feature (cc #87335).
The PR also updates the syn crate because the currently used version of the crate doesn't support `let_else` syntax yet.
Note: Generally I'm the person who *removes* usages of unstable features from the compiler, not adds more usages of them, but in this instance I think it hopefully helps the feature get stabilized sooner and in a better state. I have written a [comment](https://github.com/rust-lang/rust/issues/87335#issuecomment-944846205) on the tracking issue about my experience and what I feel could be improved before stabilization of `let_else`.
polymorphization: shims and predicates
Supersedes #75737 and #75414. This pull request includes up some changes to polymorphization which hadn't landed previously and gets stage2 bootstrapping and the test suite passing when polymorphization is enabled. There are still issues with `type_id` and polymorphization to investigate but this should get polymorphization in a reasonable state to work on.
- #75737 and #75414 both worked but were blocked on having the rest of the test suite pass (with polymorphization enabled) with and without the PRs. It makes more sense to just land these so that the changes are in.
- #75737's changes remove the restriction of `InstanceDef::Item` on polymorphization, so that shims can now be polymorphized. This won't have much of an effect until polymorphization's analysis is more advanced, but it doesn't hurt.
- #75414's changes remove all logic which marks parameters as used based on their presence in predicates - given #75675, this will enable more polymorphization and avoid the symbol clashes that predicate logic previously sidestepped.
- Polymorphization now explicitly checks (and skips) foreign items, this is necessary for stage2 bootstrapping to work when polymorphization is enabled.
- The conditional determining the emission of a note adding context to a post-monomorphization error has been modified. Polymorphization results in `optimized_mir` running for shims during collection where that wouldn't happen previously, some errors are emitted during `optimized_mir` and these were considered post-monomorphization errors with the existing logic (more errors and shims have a `DefId` coming from the std crate, not the local crate), adding a note that resulted in tests failing. It isn't particularly feasible to change where polymorphization runs or prevent it from using `optimized_mir`, so it seemed more reasonable to not change the conditional.
- `characteristic_def_id_of_type` was being invoked during partitioning for self types of impl blocks which had projections that depended on the value of unused generic parameters of a function - this caused a ICE in a debuginfo test. If partitioning is enabled and the instance needs substitution then this is skipped. That test still fails for me locally, but not with an ICE, but it fails in a fresh checkout too, so 🤷♂️.
r? `@lcnr`
This performs a substitution of code following the pattern:
let <id> = if let <pat> = ... { identity } else { ... : ! };
To simplify it to:
let <pat> = ... { identity } else { ... : ! };
By adopting the let_else feature.
Only dylib and rlib candidates were included in the error. I think the
reason is that at the time this error was originally implemented, rmeta
crate sources were represented different from dylib and rlib sources.
I wrote up more detailed analysis in [this comment][1].
The new version of the code is also a bit easier to read and should be
more robust to future changes since it uses `CrateSources::paths()`.
[1]: https://github.com/rust-lang/rust/pull/88675#issuecomment-935282436
Correct decoding of foreign expansions during incr. comp.
Fixes https://github.com/rust-lang/rust/issues/74946
The original issue was due to a wrong assertion in `expn_hash_to_expn_id`.
The secondary issue was due to a mismatch between the encoding and decoding paths for expansions that are created after the TyCtxt is created.
Implement `#[link_ordinal(n)]`
Allows the use of `#[link_ordinal(n)]` with `#[link(kind = "raw-dylib")]`, allowing Rust to link against DLLs that export symbols by ordinal rather than by name. As long as the ordinal matches, the name of the function in Rust is not required to match the name of the corresponding function in the exporting DLL.
Part of #58713.
Make an initial guess for metadata size to reduce buffer resizes
When reading metadata, the compiler starts with a `Vec::new()`, which will need to grow repeatedly as the metadata gets decompressed into it. Reduce the number of resizes by starting out at the size of the compressed data.
Rework HIR API to make invocations of the hir_crate query harder.
`hir_crate` forces the recomputation of queries that depend on it.
This PR aims at avoiding useless invocations of `hir_crate` by making dependent code go through `tcx.hir()`.
This commit removes the restriction of `InstanceDef::Item` on
polymorphization, so that shims can now be polymorphized.
Signed-off-by: David Wood <david.wood@huawei.com>
fix non_blanket_impls iteration order
We sometimes iterate over all `non_blanket_impls`, not sure if this is observable outside
of error messages (i.e. as incremental bugs). This should fix the underlying issue of #86986.
second attempt of #88718
r? `@nikomatsakis`
Migrate in-tree crates to 2021
This replaces #89075 (cherry picking some of the commits from there), and closes#88637 and fixes#89074.
It excludes a migration of the library crates for now (see tidy diff) because we have some pending bugs around macro spans to fix there.
I instrumented bootstrap during the migration to make sure all crates moved from 2018 to 2021 had the compatibility warnings applied first.
Originally, the intent was to support cargo fix --edition within bootstrap, but this proved fairly difficult to pull off. We'd need to architect the check functionality to support running cargo check and cargo fix within the same x.py invocation, and only resetting sysroots on check. Further, it was found that cargo fix doesn't behave too well with "not quite workspaces", such as Clippy which has several crates. Bootstrap runs with --manifest-path ... for all the tools, and this makes cargo fix only attempt migration for that crate. We can't use e.g. --workspace due to needing to maintain sysroots for different phases of compilation appropriately.
It is recommended to skip the mass migration of Cargo.toml's to 2021 for review purposes; you can also use `git diff d6cd2c6c87 -I'^edition = .20...$'` to ignore the edition = 2018/21 lines in the diff.
Lower only one HIR owner at a time
Based on https://github.com/rust-lang/rust/pull/83723
Additional diff is here: https://github.com/cjgillot/rust/compare/ownernode...lower-mono
Lowering is very tangled and has a tendency to intertwine the transformation of different items. This PR aims at simplifying the logic by:
- moving global analyses to the resolver (item_generics_num_lifetimes, proc_macros, trait_impls);
- removing a few special cases (non-exported macros and use statements);
- restricting the amount of available information at any one time;
- avoiding back-and-forth between different owners: an item must now be lowered all at once, and its parent cannot refer to its nodes.
I also removed the sorting of bodies by span. The diagnostic ordering changes marginally, since definitions are pretty much sorted already according to the AST. This uncovered a subtlety in thir-unsafeck.
(While these items could logically be in different PRs, the dependency between commits and the amount of conflicts force a monolithic PR.)
This also adjusts the lint docs generation to accept (and ignore) an allow
attribute, rather than expecting the documentation to be immediately followed by
the lint name.
This encoding allows for random access without an expensive upfront decoding
state which in turn allows simplifying the DefPathIndex lookup logic without
regressing performance.
Add -Z panic-in-drop={unwind,abort} command-line option
This PR changes `Drop` to abort if an unwinding panic attempts to escape it, making the process abort instead. This has several benefits:
- The current behavior when unwinding out of `Drop` is very unintuitive and easy to miss: unwinding continues, but the remaining drops in scope are simply leaked.
- A lot of unsafe code doesn't expect drops to unwind, which can lead to unsoundness:
- https://github.com/servo/rust-smallvec/issues/14
- https://github.com/bluss/arrayvec/issues/3
- There is a code size and compilation time cost to this: LLVM needs to generate extra landing pads out of all calls in a drop implementation. This can compound when functions are inlined since unwinding will then continue on to process drops in the callee, which can itself unwind, etc.
- Initial measurements show a 3% size reduction and up to 10% compilation time reduction on some crates (`syn`).
One thing to note about `-Z panic-in-drop=abort` is that *all* crates must be built with this option for it to be sound since it makes the compiler assume that dropping `Box<dyn Any>` will never unwind.
cc https://github.com/rust-lang/lang-team/issues/97
generic_const_exprs: use thir for abstract consts instead of mir
Changes `AbstractConst` building to use `thir` instead of `mir` so that there's less chance of consts unifying when they shouldn't because lowering to mir dropped information (see `abstract-consts-as-cast-5.rs` test)
r? `@lcnr`
Encode spans relative to the enclosing item
The aim of this PR is to avoid recomputing queries when code is moved without modification.
MCP at https://github.com/rust-lang/compiler-team/issues/443
This is achieved by :
1. storing the HIR owner LocalDefId information inside the span;
2. encoding and decoding spans relative to the enclosing item in the incremental on-disk cache;
3. marking a dependency to the `source_span(LocalDefId)` query when we translate a span from the short (`Span`) representation to its explicit (`SpanData`) representation.
Since all client code uses `Span`, step 3 ensures that all manipulations
of span byte positions actually create the dependency edge between
the caller and the `source_span(LocalDefId)`.
This query return the actual absolute span of the parent item.
As a consequence, any source code motion that changes the absolute byte position of a node will either:
- modify the distance to the parent's beginning, so change the relative span's hash;
- dirty `source_span`, and trigger the incremental recomputation of all code that
depends on the span's absolute byte position.
With this scheme, I believe the dependency tracking to be accurate.
For the moment, the spans are marked during lowering.
I'd rather do this during def-collection,
but the AST MutVisitor is not practical enough just yet.
The only difference is that we attach macro-expanded spans
to their expansion point instead of the macro itself.
CrateLocator refactorings
This makes the `CrateLocator` a lot cleaner IMHO and much more self-contained. The last commit removes `extra_filename` from the crate metadata. This is an **insta-stable** change as it allows a crate like `libfoo-abc.rlib` to be used as dependency and then be renamed as `libfoo-bcd.rlib` while still being found as indirect dependency. This may reduce performance when there are a lot of versions of the same crate available as the extra filename won't be used to do an early rejection of crates before trying to load metadata, but it makes the logic to find the right filename a lot cleaner.
Correctly handle remapping from path containing the current directory with trailing paths
If we have a `auxiliary/lib.rs`, and we generate the metadata with `--remap-path-prefix $PWD/auxiliary=xyz`, the path to `$PWD/auxiliary/lib.rs` won't be correctly remapped in the metadata. This is because internally, path to the working directory itself and relative paths to files under the working directory are remapped separately (hence neither are affected since neither has `$PWD/auxiliary` as prefix), but the concatenation between the working directory and the relative path is not remapped. This PR fixes that.
Remove `Session.used_attrs` and move logic to `CheckAttrVisitor`
Instead of updating global state to mark attributes as used,
we now explicitly emit a warning when an attribute is used in
an unsupported position. As a side effect, we are to emit more
detailed warning messages (instead of just a generic "unused" message).
`Session.check_name` is removed, since its only purpose was to mark
the attribute as used. All of the callers are modified to use
`Attribute.has_name`
Additionally, `AttributeType::AssumedUsed` is removed - an 'assumed
used' attribute is implemented by simply not performing any checks
in `CheckAttrVisitor` for a particular attribute.
We no longer emit unused attribute warnings for the `#[rustc_dummy]`
attribute - it's an internal attribute used for tests, so it doesn't
mark sense to treat it as 'unused'.
With this commit, a large source of global untracked state is removed.
Instead of updating global state to mark attributes as used,
we now explicitly emit a warning when an attribute is used in
an unsupported position. As a side effect, we are to emit more
detailed warning messages (instead of just a generic "unused" message).
`Session.check_name` is removed, since its only purpose was to mark
the attribute as used. All of the callers are modified to use
`Attribute.has_name`
Additionally, `AttributeType::AssumedUsed` is removed - an 'assumed
used' attribute is implemented by simply not performing any checks
in `CheckAttrVisitor` for a particular attribute.
We no longer emit unused attribute warnings for the `#[rustc_dummy]`
attribute - it's an internal attribute used for tests, so it doesn't
mark sense to treat it as 'unused'.
With this commit, a large source of global untracked state is removed.
Fixes#85019
A `SourceFile` created during compilation may have a relative
path (e.g. if rustc itself is invoked with a relative path).
When we write out crate metadata, we convert all relative paths
to absolute paths using the current working direction.
However, the working directory is not included in the crate hash.
This means that the crate metadata can change while the crate
hash remains the same. Among other problems, this can cause a
fingerprint mismatch ICE, since incremental compilation uses
the crate metadata hash to determine if a foreign query is green.
This commit moves the field holding the working directory from
`Session` to `Options`, including it as part of the crate hash.
Plugin interface cleanup
The first commit performs two uncontroversial cleanups. The second commit removes `#[plugin_registrar]` and instead requires you to export a `__rustc_plugin_registrar` function, this will require a change to servo's script_plugins (cc `@jdm)`
Fix feature gate checking of static-nobundle and native_link_modifiers
Feature native_link_modifiers_bundle don't need feature static-nobundle
to work.
Also check the feature gates when using native_link_modifiers from command line options. Current nighly compiler don't check those feature gate.
```
> touch lib.rs
> rustc +nightly lib.rs -L /usr/lib -l static:+bundle=dl --crate-type=rlib
> rustc +nightly lib.rs -L /usr/lib -l dylib:+as-needed=dl --crate-type=dylib -Ctarget-feature=-crt-static
> rustc +nightly lib.rs -L /usr/lib -l static:-bundle=dl --crate-type=rlib
error[E0658]: kind="static-nobundle" is unstable
|
= note: see issue #37403 <https://github.com/rust-lang/rust/issues/37403> for more information
= help: add `#![feature(static_nobundle)]` to the crate attributes to enable
error: aborting due to previous error
For more information about this error, try `rustc --explain E0658`.
```
First found this in https://github.com/rust-lang/rust/pull/85600#discussion_r676612655
This was removed by #85284 in favor of -Zprofiler-runtime=<name>.
However the suggested -Zprofiler-runtime=None doesn't work because
"None" is treated as a crate name.
Since RFC 3052 soft deprecated the authors field anyway, hiding it from
crates.io, docs.rs, and making Cargo not add it by default, and it is
not generally up to date/useful information, we should remove it from
crates in this repo.
Store all HIR owners in the same container
This replaces the previous storage in a BTreeMap for each of Item/ImplItem/TraitItem/ForeignItem.
This should allow for a more compact storage.
Based on https://github.com/rust-lang/rust/pull/83114
Previously, this would change the test output when RUSTUP_HOME was set:
```
---- [ui] ui/issues/issue-49851/compiler-builtins-error.rs stdout ----
diff of stderr:
1 error[E0463]: can't find crate for `core`
2 |
3 = note: the `thumbv7em-none-eabihf` target may not be installed
+ = help: consider downloading the target with `rustup target add thumbv7em-none-eabihf`
4
5 error: aborting due to previous error
6
```
Originally, I fixed it by explicitly unsetting RUSTUP_HOME in
compiletest. Then I realized that almost no one has RUSTUP_HOME set,
since rustup doesn't set it itself; although it does set RUST_RECURSION_COUNT
whenever it launches a proxy. Then it was pointed out that this runtime
check doesn't really make sense and it's fine to make it unconditional.
Remove nondeterminism in multiple-definitions test
Compare all fields in `DllImport` when sorting to avoid nondeterminism in the error for multiple inconsistent definitions of an extern function. Restore the multiple-definitions test.
Resolves#87084.
Reuse CrateNum for proc-macro crates even when cross-compiling
Proc-macros are always compiled for the host, so this should be the same
in every way as recompiling the crate.
I am not sure why the previous code special-cased the target, since the
compiler properly gives an error when trying to load a crate for a
different host:
```
error[E0461]: couldn't find crate `dependency` with expected target triple x86_64-unknown-linux-gnu
--> /home/joshua/rustc4/src/test/ui/cfg-dependent.rs:8:2
|
LL | dependency::is_64();
| ^^^^^^^^^^
|
= note: the following crate versions were found:
crate `dependency`, target triple i686-unknown-linux-gnu: /home/joshua/rustc4/build/x86_64-unknown-linux-gnu/test/ui/cfg-dependent/auxiliary/libdependency.so
```
I think another possible fix is to remove the check altogether. But I'm
not sure, and this fix works, so I'm not making the larger change here.
Fixes https://github.com/rust-lang/rust/issues/56935.
r? `@petrochenkov` cc `@alexcrichton`
Proc-macros are always compiled for the host, so this should be the same
in every way as recompiling the crate.
I am not sure why the previous code special-cased the target, since the
compiler properly gives an error when trying to load a crate for a
different host:
```
error[E0461]: couldn't find crate `dependency` with expected target triple x86_64-unknown-linux-gnu
--> /home/joshua/rustc4/src/test/ui/cfg-dependent.rs:8:2
|
LL | dependency::is_64();
| ^^^^^^^^^^
|
= note: the following crate versions were found:
crate `dependency`, target triple i686-unknown-linux-gnu: /home/joshua/rustc4/build/x86_64-unknown-linux-gnu/test/ui/cfg-dependent/auxiliary/libdependency.so
```
I think another possible fix is to remove the check altogether. But I'm
not sure, and this fix works, so I'm not making the larger change here.
Remove unused dependencies from compiler crates
Various compiler crates have dependencies that they don't appear to use. I used some scripting to detect such dependencies, filtered them based on some manual review, and removed those that do indeed appear to be entirely unused.
Due to the std/alloc split, it is not possible to make
`alloc::collections::TryReserveError::AllocError` non-exhaustive without
having an unstable, doc-hidden method to construct (which negates the
benefits from `#[non_exhaustive]`.
Provide option for specifying the profiler runtime
Currently, if `-Zinstrument-coverage` is enabled, the target is linked
against the `library/profiler_builtins` crate (which pulls in LLVM's
compiler-rt runtime).
This option enables backends to specify an alternative runtime crate for
handling injected instrumentation calls.
Don't use a generator for BoxedResolver
The generator is non-trivial and requires unsafe code anyway. Using regular unsafe code without a generator is much easier to follow.
Based on #85810 as it touches rustc_interface too.
Partial support for raw-dylib linkage
First cut of functionality for issue #58713: add support for `#[link(kind = "raw-dylib")]` on `extern` blocks in lib crates compiled to .rlib files. Does not yet support `#[link_name]` attributes on functions, or the `#[link_ordinal]` attribute, or `#[link(kind = "raw-dylib")]` on `extern` blocks in bin crates; I intend to publish subsequent PRs to fill those gaps. It's also not yet clear whether this works for functions in `extern "stdcall"` blocks; I also intend to investigate that shortly and make any necessary changes as a follow-on PR.
This implementation calls out to an LLVM function to construct the actual `.idata` sections as temporary `.lib` files on disk and then links those into the generated .rlib.
This does not yet support #[link_name] attributes on functions, the #[link_ordinal]
attribute, #[link(kind = "raw-dylib")] on extern blocks in bin crates, or
stdcall functions on 32-bit x86.
Restoring the `num_def_ids` function in the CStore API
## The context
I am the maintainer of https://github.com/hacspec/hacspec, an embedded Rust DSL aimed at cryptographic specifications. As it is normal for an embedded DSL, Hacspec's compiler relies on being plugged to the internal API of the Rust compiler, which is unstable and subject to changes.
## The problem
The Hacspec compiler features its own typechecker, that performs an additional, more restrictive typechecking pass over the Rust code of a crate. To complete this typechecking, the Hacspec compiler needs to retrieve the signature of functions defined in non-local imported crates. Rather than retrieving these signatures on-demand, the Hacspec compiler pre-populates its typechecking context with all the Hacspec-compatible symbols defined in non-local crates first. This requires having a way to iterate over all the definitions in a non-local crate.
I used to do this with `CrateMetadata::all_def_path_hashes_and_def_ids`, but this function was deleted in 908bf5a310. Then, I fellback on `CStore::num_def_ids`, exploiting the fact that all the `DefIds` for a crate have the same `krate_num` and range from `0` to `num_def_ids(krate_num)`. But `num_def_ids` was deleted in b6120bfb35.
I looked to the `Cstore::item_children_untracked` function to replicate the feature of traversing through all the `DefId` for a crate, using `CRATE_DEF_INDEX` as the root, but this does not work as recursive `Cstore::item_children_untracked` calls do not reach all the symbols I was able to reach using the two previous methods.
## Description of this PR
This PR simply restores in the public API of `CStore` the `num_def_ids` function, giving the size of the definition table for a given crate.
Remove unused feature gates
The first commit removes a usage of a feature gate, but I don't expect it to be controversial as the feature gate was only used to workaround a limitation of rust in the past. (closures never being `Clone`)
The second commit uses `#[allow_internal_unstable]` to avoid leaking the `trusted_step` feature gate usage from inside the index newtype macro. It didn't work for the `min_specialization` feature gate though.
The third commit removes (almost) all feature gates from the compiler that weren't used anyway.
Restored underlying num_def_ids_method
Update compiler/rustc_metadata/src/rmeta/decoder/cstore_impl.rs
Changed name to fit with naming convention
Co-authored-by: bjorn3 <bjorn3@users.noreply.github.com>
Update compiler/rustc_metadata/src/rmeta/decoder/cstore_impl.rs
Replace regular doc with Rustdoc comment
Co-authored-by: Joshua Nelson <jyn514@gmail.com>
Clarifies third-party use of num_def_ids_untracked
Currently, if `-Zinstrument-coverage` is enabled, the target is linked
against the `library/profiler_builtins` crate (which pulls in LLVM's
compiler-rt runtime).
This option enables backends to specify an alternative runtime crate for
handling injected instrumentation calls.
Remove CrateNum parameter for queries that only work on local crate
The pervasive `CrateNum` parameter is a remnant of the multi-crate rustc idea.
Using `()` as query key in those cases avoids having to worry about the validity of the query key.
Preserve `SyntaxContext` for invalid/dummy spans in crate metadata
Fixes#85197
We already preserved the `SyntaxContext` for invalid/dummy spans in the
incremental cache, but we weren't doing the same for crate metadata.
If an invalid (lo/hi from different files) span is written to the
incremental cache, we will decode it with a 'dummy' location, but keep
the original `SyntaxContext`. Since the crate metadata encoder was only
checking for `DUMMY_SP` (dummy location + root `SyntaxContext`),
the metadata encoder would treat it as a normal span, encoding the
`SyntaxContext`. As a result, the final span encoded to the metadata
would change across sessions, even if the crate itself was unchanged.
This could lead to an 'unstable fingerprint' ICE under the following conditions:
1. We compile a crate with an invalid span using incremental compilation. The metadata encoder discards the `SyntaxContext` since the span is invalid, while the incremental cache encoder preserves the `SyntaxContext`
2. From another crate, we execute a foreign query, decoding the invalid span from the metadata as `DUMMY_SP` (e.g. with `SyntaxContext::root()`). This span gets hashed into the query fingerprint. So far, this has always happened through the `optimized_mir` query.
3. We recompile the first crate using our populated incremental cache, without changing anything. We load the (previously) invalid span from our incremental cache - it gets converted to a span with a dummy (but valid) location, along with the original `SyntaxContext`. This span gets written out to the crate metadata - since it now has a valid location, we preserve its `SyntaxContext`.
4. We recompile the second crate, again using a populated incremental cache. We now re-run the foreign query `optimized_mir` - the foreign crate hash is unchanged, but we end up decoding a different span (it now ha a non-root `SyntaxContext`). This results in the fingerprint changing, resulting in an ICE.
This PR updates our encoding of spans in the crate metadata to mirror
the encoding of spans into the incremental cache. We now always encode a
`SyntaxContext`, and encode location information for spans with a
non-dummy location.
Fix diagnostic for cross crate private tuple struct constructors
Fixes#78708.
There was already some limited support for certain cross-crate scenarios but that didn't handle a tuple struct rexported from an inner module for example (e.g. the NonZero* types as seen in #85049).
```Rust
➜ cat bug.rs
fn main() {
let _x = std::num::NonZeroU32(12);
let n = std::num::NonZeroU32::new(1).unwrap();
match n {
std::num::NonZeroU32(i) => {},
}
}
```
**Before:**
<details>
```Rust
➜ rustc +nightly bug.rs
error[E0423]: expected function, tuple struct or tuple variant, found struct `std::num::NonZeroU32`
--> bug.rs:2:14
|
2 | let _x = std::num::NonZeroU32(12);
| ^^^^^^^^^^^^^^^^^^^^^^^^ help: use struct literal syntax instead: `std::num::NonZeroU32 { 0: val }`
|
::: /home/luqman/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/num/nonzero.rs:148:1
[snip]
error[E0532]: expected tuple struct or tuple variant, found struct `std::num::NonZeroU32`
--> bug.rs:5:9
|
5 | std::num::NonZeroU32(i) => {},
| ^^^^^^^^^^^^^^^^^^^^^^^ help: use struct pattern syntax instead: `std::num::NonZeroU32 { 0 }`
|
::: /home/luqman/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/num/nonzero.rs:148:1
[snip]
error: aborting due to 2 previous errors
Some errors have detailed explanations: E0423, E0532.
For more information about an error, try `rustc --explain E0423`.
```
</details>
**After:**
<details>
```Rust
➜ /rust/build/x86_64-unknown-linux-gnu/stage1/bin/rustc bug.rs
error[E0423]: cannot initialize a tuple struct which contains private fields
--> bug.rs:2:14
|
2 | let _x = std::num::NonZeroU32(12);
| ^^^^^^^^^^^^^^^^^^^^
|
note: constructor is not visible here due to private fields
--> /rust/library/core/src/num/nonzero.rs:148:1
[snip]
error[E0532]: cannot match against a tuple struct which contains private fields
--> bug.rs:5:9
|
5 | std::num::NonZeroU32(i) => {},
| ^^^^^^^^^^^^^^^^^^^^
|
note: constructor is not visible here due to private fields
--> bug.rs:5:30
|
5 | std::num::NonZeroU32(i) => {},
| ^ private field
error: aborting due to 2 previous errors
Some errors have detailed explanations: E0423, E0532.
For more information about an error, try `rustc --explain E0423`.
```
</details>
One question is if we should only collect the needed info for the cross-crate case after encountering an error instead of always doing it. Perf run perhaps to gauge the impact.
Fixes#85197
We already preserved the `SyntaxContext` for invalid/dummy spans in the
incremental cache, but we weren't doing the same for crate metadata.
If an invalid (lo/hi from different files) span is written to the
incremental cache, we will decode it with a 'dummy' location, but keep
the original `SyntaxContext`. Since the crate metadata encoder was only
checking for `DUMMY_SP` (dummy location + root `SyntaxContext`),
the metadata encoder would treat it as a normal span, encoding the
`SyntaxContext`. As a result, the final span encoded to the metadata
would change across sessions, even if the crate itself was unchanged.
This PR updates our encoding of spans in the crate metadata to mirror
the encoding of spans into the incremental cache. We now always encode a
`SyntaxContext`, and encode location information for spans with a
non-dummy location.
Fix `--remap-path-prefix` not correctly remapping `rust-src` component paths and unify handling of path mapping with virtualized paths
This PR fixes#73167 ("Binaries end up containing path to the rust-src component despite `--remap-path-prefix`") by preventing real local filesystem paths from reaching compilation output if the path is supposed to be remapped.
`RealFileName::Named` introduced in #72767 is now renamed as `LocalPath`, because this variant wraps a (most likely) valid local filesystem path.
`RealFileName::Devirtualized` is renamed as `Remapped` to be used for remapped path from a real path via `--remap-path-prefix` argument, as well as real path inferred from a virtualized (during compiler bootstrapping) `/rustc/...` path. The `local_path` field is now an `Option<PathBuf>`, as it will be set to `None` before serialisation, so it never reaches any build output. Attempting to serialise a non-`None` `local_path` will cause an assertion faliure.
When a path is remapped, a `RealFileName::Remapped` variant is created. The original path is preserved in `local_path` field and the remapped path is saved in `virtual_name` field. Previously, the `local_path` is directly modified which goes against its purpose of "suitable for reading from the file system on the local host".
`rustc_span::SourceFile`'s fields `unmapped_path` (introduced by #44940) and `name_was_remapped` (introduced by #41508 when `--remap-path-prefix` feature originally added) are removed, as these two pieces of information can be inferred from the `name` field: if it's anything other than a `FileName::Real(_)`, or if it is a `FileName::Real(RealFileName::LocalPath(_))`, then clearly `name_was_remapped` would've been false and `unmapped_path` would've been `None`. If it is a `FileName::Real(RealFileName::Remapped{local_path, virtual_name})`, then `name_was_remapped` would've been true and `unmapped_path` would've been `Some(local_path)`.
cc `@eddyb` who implemented `/rustc/...` path devirtualisation
This PR implements span quoting, allowing proc-macros to produce spans
pointing *into their own crate*. This is used by the unstable
`proc_macro::quote!` macro, allowing us to get error messages like this:
```
error[E0412]: cannot find type `MissingType` in this scope
--> $DIR/auxiliary/span-from-proc-macro.rs:37:20
|
LL | pub fn error_from_attribute(_args: TokenStream, _input: TokenStream) -> TokenStream {
| ----------------------------------------------------------------------------------- in this expansion of procedural macro `#[error_from_attribute]`
...
LL | field: MissingType
| ^^^^^^^^^^^ not found in this scope
|
::: $DIR/span-from-proc-macro.rs:8:1
|
LL | #[error_from_attribute]
| ----------------------- in this macro invocation
```
Here, `MissingType` occurs inside the implementation of the proc-macro
`#[error_from_attribute]`. Previosuly, this would always result in a
span pointing at `#[error_from_attribute]`
This will make many proc-macro-related error message much more useful -
when a proc-macro generates code containing an error, users will get an
error message pointing directly at that code (within the macro
definition), instead of always getting a span pointing at the macro
invocation site.
This is implemented as follows:
* When a proc-macro crate is being *compiled*, it causes the `quote!`
macro to get run. This saves all of the sapns in the input to `quote!`
into the metadata of *the proc-macro-crate* (which we are currently
compiling). The `quote!` macro then expands to a call to
`proc_macro::Span::recover_proc_macro_span(id)`, where `id` is an
opaque identifier for the span in the crate metadata.
* When the same proc-macro crate is *run* (e.g. it is loaded from disk
and invoked by some consumer crate), the call to
`proc_macro::Span::recover_proc_macro_span` causes us to load the span
from the proc-macro crate's metadata. The proc-macro then produces a
`TokenStream` containing a `Span` pointing into the proc-macro crate
itself.
The recursive nature of 'quote!' can be difficult to understand at
first. The file `src/test/ui/proc-macro/quote-debug.stdout` shows
the output of the `quote!` macro, which should make this eaier to
understand.
This PR also supports custom quoting spans in custom quote macros (e.g.
the `quote` crate). All span quoting goes through the
`proc_macro::quote_span` method, which can be called by a custom quote
macro to perform span quoting. An example of this usage is provided in
`src/test/ui/proc-macro/auxiliary/custom-quote.rs`
Custom quoting currently has a few limitations:
In order to quote a span, we need to generate a call to
`proc_macro::Span::recover_proc_macro_span`. However, proc-macros
support renaming the `proc_macro` crate, so we can't simply hardcode
this path. Previously, the `quote_span` method used the path
`crate::Span` - however, this only works when it is called by the
builtin `quote!` macro in the same crate. To support being called from
arbitrary crates, we need access to the name of the `proc_macro` crate
to generate a path. This PR adds an additional argument to `quote_span`
to specify the name of the `proc_macro` crate. Howver, this feels kind
of hacky, and we may want to change this before stabilizing anything
quote-related.
Additionally, using `quote_span` currently requires enabling the
`proc_macro_internals` feature. The builtin `quote!` macro
has an `#[allow_internal_unstable]` attribute, but this won't work for
custom quote implementations. This will likely require some additional
tricks to apply `allow_internal_unstable` to the span of
`proc_macro::Span::recover_proc_macro_span`.
This commit implements both the native linking modifiers infrastructure
as well as an initial attempt at the individual modifiers from the RFC.
It also introduces a feature flag for the general syntax along with
individual feature flags for each modifier.
Give a better error when `std` or `core` are missing
- Suggest using `rustup target add` if `RUSTUP_HOME` is set. I don't know if there's any precedent for doing this, but it seems harmless enough and it will be a big help.
- On nightly, suggest using `cargo build -Z build-std` if `CARGO` is set
- Add a note about `#![no_std]` if `std` is missing but not core
- Add a note that std may be unsupported if `std` is missing but not core
Fixes https://github.com/rust-lang/rust/issues/84418.
r? `@petrochenkov`
- Suggest using `rustup target add` if `RUSTUP_HOME` is set. I don't know if there's any precedent for doing this, but it seems harmless enough and it will be a big help.
- Add a note about `#![no_std]` if `std` is missing but not core
- On nightly, suggest using `cargo build -Z build-std` if `CARGO` is set
- Add a note that std may be unsupported if `std` is missing but not core
- Don't suggest `#![no_std]` when the load isn't injected by the
compiler
Add an unstable --json=unused-externs flag to print unused externs
This adds an unstable flag to print a list of the extern names not used by cargo.
This PR will enable cargo to collect unused dependencies from all units and provide warnings.
The companion PR to cargo is: https://github.com/rust-lang/cargo/pull/8437
The goal is eventual stabilization of this flag in rustc as well as in cargo.
Discussion of this feature is mostly contained inside these threads: #57274#72342#72603
The feature builds upon the internal datastructures added by #72342
Externs are uniquely identified by name and the information is sufficient for cargo.
If the mode is enabled, rustc will print json messages like:
```
{"unused_extern_names":["byteorder","openssl","webpki"]}
```
For a crate that got passed byteorder, openssl and webpki dependencies but needed none of them.
### Q: Why not pass -Wunused-crate-dependencies?
A: See [ehuss's comment here](https://github.com/rust-lang/rust/issues/57274#issuecomment-624839355)
TLDR: it's cleaner. Rust's warning system wasn't built to be filtered or edited by cargo.
Even a basic implementation of the feature would have to change the "n warnings emitted" line that rustc prints at the end.
Cargo ideally wants to synthesize its own warnings anyways. For example, it would be hard for rustc to emit warnings like
"dependency foo is only used by dev targets", suggesting to make it a dev-dependency instead.
### Q: Make rustc emit used or unused externs?
A: Emitting used externs has the advantage that it simplifies cargo's collection job.
However, emitting unused externs creates less data to be communicated between rustc and cargo.
Often you want to paste a cargo command obtained from `cargo build -vv` for doing something
completely unrelated. The message is emitted always, even if no warning or error is emitted.
At that point, even this tiny difference in "noise" matters. That's why I went with emitting unused externs.
### Q: One json msg per extern or a collective json msg?
A: Same as above, the data format should be concise. Having 30 lines for the 30 crates a crate uses would be disturbing to readers.
Also it helps the cargo implementation to know that there aren't more unused deps coming.
### Q: Why use names of externs instead of e.g. paths?
A: Names are both sufficient as well as neccessary to uniquely identify a passed `--extern` arg.
Names are sufficient because you *must* pass a name when passing an `--extern` arg.
Passing a path is optional on the other hand so rustc might also figure out a crate's location from the file system.
You can also put multiple paths for the same extern name, via e.g. `--extern hello=/usr/lib/hello.rmeta --extern hello=/usr/local/lib/hello.rmeta`,
but rustc will only ever use one of those paths.
Also, paths don't identify a dependency uniquely as it is possible to have multiple different extern names point to the same path.
So paths are ill-suited for identification.
### Q: What about 2015 edition crates?
A: They are fully supported.
Even on the 2015 edition, an explicit `--extern` flag is is required to enable `extern crate foo;` to work (outside of sysroot crates, which this flag doesn't warn about anyways).
So the lint would still fire on 2015 edition crates if you haven't included a dependency specified in Cargo.toml using `extern crate foo;` or similar.
The lint won't fire if your sole use in the crate is through a `extern crate foo;` statement, but that's not its job.
For detecting unused `extern crate foo` statements, there is the `unused_extern_crates` lint
which can be enabled by `#![warn(unused_extern_crates)]` or similar.
cc ```@jsgf``` ```@ehuss``` ```@petrochenkov``` ```@estebank```
Add an Mmap wrapper to rustc_data_structures
This wrapper implements StableAddress and falls back to directly reading the file on wasm32.
Taken from #83640, which I will close due to the perf regression.
coverage bug fixes and optimization support
Adjusted LLVM codegen for code compiled with `-Zinstrument-coverage` to
address multiple, somewhat related issues.
Fixed a significant flaw in prior coverage solution: Every counter
generated a new counter variable, but there should have only been one
counter variable per function. This appears to have bloated .profraw
files significantly. (For a small program, it increased the size by
about 40%. I have not tested large programs, but there is anecdotal
evidence that profraw files were way too large. This is a good fix,
regardless, but hopefully it also addresses related issues.
Fixes: #82144
Invalid LLVM coverage data produced when compiled with -C opt-level=1
Existing tests now work up to at least `opt-level=3`. This required a
detailed analysis of the LLVM IR, comparisons with Clang C++ LLVM IR
when compiled with coverage, and a lot of trial and error with codegen
adjustments.
The biggest hurdle was figuring out how to continue to support coverage
results for unused functions and generics. Rust's coverage results have
three advantages over Clang's coverage results:
1. Rust's coverage map does not include any overlapping code regions,
making coverage counting unambiguous.
2. Rust generates coverage results (showing zero counts) for all unused
functions, including generics. (Clang does not generate coverage for
uninstantiated template functions.)
3. Rust's unused functions produce minimal stubbed functions in LLVM IR,
sufficient for including in the coverage results; while Clang must
generate the complete LLVM IR for each unused function, even though
it will never be called.
This PR removes the previous hack of attempting to inject coverage into
some other existing function instance, and generates dedicated instances
for each unused function. This change, and a few other adjustments
(similar to what is required for `-C link-dead-code`, but with lower
impact), makes it possible to support LLVM optimizations.
Fixes: #79651
Coverage report: "Unexecuted instantiation:..." for a generic function
from multiple crates
Fixed by removing the aforementioned hack. Some "Unexecuted
instantiation" notices are unavoidable, as explained in the
`used_crate.rs` test, but `-Zinstrument-coverage` has new options to
back off support for either unused generics, or all unused functions,
which avoids the notice, at the cost of less coverage of unused
functions.
Fixes: #82875
Invalid LLVM coverage data produced with crate brotli_decompressor
Fixed by disabling the LLVM function attribute that forces inlining, if
`-Z instrument-coverage` is enabled. This attribute is applied to
Rust functions with `#[inline(always)], and in some cases, the forced
inlining breaks coverage instrumentation and reports.
FYI: `@wesleywiser`
r? `@tmandry`
Adjusted LLVM codegen for code compiled with `-Zinstrument-coverage` to
address multiple, somewhat related issues.
Fixed a significant flaw in prior coverage solution: Every counter
generated a new counter variable, but there should have only been one
counter variable per function. This appears to have bloated .profraw
files significantly. (For a small program, it increased the size by
about 40%. I have not tested large programs, but there is anecdotal
evidence that profraw files were way too large. This is a good fix,
regardless, but hopefully it also addresses related issues.
Fixes: #82144
Invalid LLVM coverage data produced when compiled with -C opt-level=1
Existing tests now work up to at least `opt-level=3`. This required a
detailed analysis of the LLVM IR, comparisons with Clang C++ LLVM IR
when compiled with coverage, and a lot of trial and error with codegen
adjustments.
The biggest hurdle was figuring out how to continue to support coverage
results for unused functions and generics. Rust's coverage results have
three advantages over Clang's coverage results:
1. Rust's coverage map does not include any overlapping code regions,
making coverage counting unambiguous.
2. Rust generates coverage results (showing zero counts) for all unused
functions, including generics. (Clang does not generate coverage for
uninstantiated template functions.)
3. Rust's unused functions produce minimal stubbed functions in LLVM IR,
sufficient for including in the coverage results; while Clang must
generate the complete LLVM IR for each unused function, even though
it will never be called.
This PR removes the previous hack of attempting to inject coverage into
some other existing function instance, and generates dedicated instances
for each unused function. This change, and a few other adjustments
(similar to what is required for `-C link-dead-code`, but with lower
impact), makes it possible to support LLVM optimizations.
Fixes: #79651
Coverage report: "Unexecuted instantiation:..." for a generic function
from multiple crates
Fixed by removing the aforementioned hack. Some "Unexecuted
instantiation" notices are unavoidable, as explained in the
`used_crate.rs` test, but `-Zinstrument-coverage` has new options to
back off support for either unused generics, or all unused functions,
which avoids the notice, at the cost of less coverage of unused
functions.
Fixes: #82875
Invalid LLVM coverage data produced with crate brotli_decompressor
Fixed by disabling the LLVM function attribute that forces inlining, if
`-Z instrument-coverage` is enabled. This attribute is applied to
Rust functions with `#[inline(always)], and in some cases, the forced
inlining breaks coverage instrumentation and reports.
Let a portion of DefPathHash uniquely identify the DefPath's crate.
This allows to directly map from a `DefPathHash` to the crate it originates from, without constructing side tables to do that mapping -- something that is useful for incremental compilation where we deal with `DefPathHash` instead of `DefId` a lot.
It also allows to reliably and cheaply check for `DefPathHash` collisions which allows the compiler to gracefully abort compilation instead of running into a subsequent ICE at some random place in the code.
The following new piece of documentation describes the most interesting aspects of the changes:
```rust
/// A `DefPathHash` is a fixed-size representation of a `DefPath` that is
/// stable across crate and compilation session boundaries. It consists of two
/// separate 64-bit hashes. The first uniquely identifies the crate this
/// `DefPathHash` originates from (see [StableCrateId]), and the second
/// uniquely identifies the corresponding `DefPath` within that crate. Together
/// they form a unique identifier within an entire crate graph.
///
/// There is a very small chance of hash collisions, which would mean that two
/// different `DefPath`s map to the same `DefPathHash`. Proceeding compilation
/// with such a hash collision would very probably lead to an ICE and, in the
/// worst case, to a silent mis-compilation. The compiler therefore actively
/// and exhaustively checks for such hash collisions and aborts compilation if
/// it finds one.
///
/// `DefPathHash` uses 64-bit hashes for both the crate-id part and the
/// crate-internal part, even though it is likely that there are many more
/// `LocalDefId`s in a single crate than there are individual crates in a crate
/// graph. Since we use the same number of bits in both cases, the collision
/// probability for the crate-local part will be quite a bit higher (though
/// still very small).
///
/// This imbalance is not by accident: A hash collision in the
/// crate-local part of a `DefPathHash` will be detected and reported while
/// compiling the crate in question. Such a collision does not depend on
/// outside factors and can be easily fixed by the crate maintainer (e.g. by
/// renaming the item in question or by bumping the crate version in a harmless
/// way).
///
/// A collision between crate-id hashes on the other hand is harder to fix
/// because it depends on the set of crates in the entire crate graph of a
/// compilation session. Again, using the same crate with a different version
/// number would fix the issue with a high probability -- but that might be
/// easier said then done if the crates in questions are dependencies of
/// third-party crates.
///
/// That being said, given a high quality hash function, the collision
/// probabilities in question are very small. For example, for a big crate like
/// `rustc_middle` (with ~50000 `LocalDefId`s as of the time of writing) there
/// is a probability of roughly 1 in 14,750,000,000 of a crate-internal
/// collision occurring. For a big crate graph with 1000 crates in it, there is
/// a probability of 1 in 36,890,000,000,000 of a `StableCrateId` collision.
```
Given the probabilities involved I hope that no one will ever actually see the error messages. Nonetheless, I'd be glad about some feedback on how to improve them. Should we create a GH issue describing the problem and possible solutions to point to? Or a page in the rustc book?
r? `@pnkfelix` (feel free to re-assign)
Only store a LocalDefId in some HIR nodes
Some HIR nodes are guaranteed to be HIR owners: Item, TraitItem, ImplItem, ForeignItem and MacroDef.
As a consequence, we do not need to store the `HirId`'s `local_id`, and we can directly store a `LocalDefId`.
This allows to avoid a bit of the dance with `tcx.hir().local_def_id` and `tcx.hir().local_def_id_to_hir_id` mappings.
This allows a build system to indicate a location in its own dependency
specification files (eg Cargo's `Cargo.toml`) which can be reported
along side any unused crate dependency.
This supports several types of location:
- 'json' - provide some json-structured data, which is included in the json diagnostics
in a `tool_metadata` field
- 'raw' - emit the provided string into the output. This also appears as a json string in
`tool_metadata`.
If no `--extern-location` is explicitly provided then a default json entry of the form
`"tool_metadata":{"name":<cratename>,"path":<cratepath>}` is emitted.
Encode MIR metadata by iterating on DefId instead of traversing the HIR tree
Split out of https://github.com/rust-lang/rust/pull/80347.
This part only traverses `mir_keys` and encodes MIR according to the def kind.
r? `@oli-obk`
This allows to directly map from a DefPathHash to the crate it
originates from, without constructing side tables to do that mapping.
It also allows to reliably and cheaply check for DefPathHash collisions.
Remove DepKind::CrateMetadata and pre-allocation of DepNodes
Remove much of the special-case handling around crate metadata
dependency tracking by replacing `DepKind::CrateMetadata` and the
pre-allocation of corresponding `DepNodes` with on-demand invocation
of the `crate_hash` query.
Rework diagnostics for wrong number of generic args (fixes#66228 and #71924)
This PR reworks the `wrong number of {} arguments` message, so that it provides more details and contextual hints.
Consistently avoid constructing optimized MIR when not doing codegen
The optimized MIR for closures is being encoded unconditionally, while
being unnecessary for cargo check. This turns out to be especially
costly with MIR inlining enabled, since it triggers computation of
optimized MIR for all callees that are being examined for inlining
purposes https://github.com/rust-lang/rust/pull/77307#issuecomment-751915450.
Skip encoding of optimized MIR for closures, enum constructors, struct
constructors, and trait fns when not doing codegen, like it is already
done for other items since 49433.
Separate out a `hir::Impl` struct
This makes it possible to pass the `Impl` directly to functions, instead
of having to pass each of the many fields one at a time. It also
simplifies matches in many cases.
See `rustc_save_analysis::dump_visitor::process_impl` or `rustdoc::clean::clean_impl` for a good example of how this makes `impl`s easier to work with.
r? `@petrochenkov` maybe?
