DefaultImpl is a highly confusing name for what we now call auto impls,
as in `impl Send for ..`. The name auto impl is not formally decided
but for sanity anything is better than `DefaultImpl` which refers
neither to `default impl` nor to `impl Default`.
incr.comp.: Implement compiler diagnostic persistence.
This PR implements storing and loading diagnostics that the compiler generates and thus allows for emitting warnings during incremental compilation without actually re-evaluating the thing the warning originally came from. It also lays some groundwork for storing and loading type information and MIR in the incr. comp. cache.
~~It is still work in progress:~~
- ~~There's still some documentation to be added.~~
- ~~The way anonymous queries are handled might lead to duplicated emissions of warnings. Not sure if there is a better way or how frequent such duplication would be in practice.~~
Diagnostic message duplication is addressed separately in #45519.
r? @nikomatsakis
- Don't hash traits in scope as part of HIR hashing any more.
- Some queries returned DefIndexes from other crates.
- Provide a generic way of stably hashing maps (not used everywhere yet).
This is a big map that ends up inside of a `CrateContext` during translation for
all codegen units. This means that any change to the map may end up causing an
incremental recompilation of a codegen unit! In order to reduce the amount of
dependencies here between codegen units and the actual input crate this commit
refactors dealing with exported symbols and such into various queries.
The new queries are largely based on existing queries with filled out
implementations for the local crate in addition to external crates, but the main
idea is that while translating codegen untis no unit needs the entire set of
exported symbols, instead they only need queries about particulare `DefId`
instances every now and then.
The linking stage, however, still generates a full list of all exported symbols
from all crates, but that's going to always happen unconditionally anyway, so no
news there!
This commit moves the definition of the `ExportedSymbols` structure to the
`rustc` crate and then creates a query that'll be used to construct the
`ExportedSymbols` set. This in turn uses the reachablity query exposed in the
previous commit.
rustc: Make `CrateStore` private to `TyCtxt`
This commit makes the `CrateStore` object private to the `ty/context.rs` module and also absent on the `Session` itself.
cc #44390
cc #44341 (initial commit pulled and rebased from here)
Autodetect the type of allocator crate used
Annotate the allocator crates (allocator_system, allocator_jemalloc) by the type of allocator they are. If one is requested as an exe allocator, detect its type by the flags.
This has the effect that using this (de jure wrong) configuration in the target spec works instead of producing a really unhelpful and arcane linker error:
"exe-allocation-crate": "alloc_system"
Fixes#43524.
There are two yet unsolved FIXME's, I'll be glad for some advice on what to do with them.
Evaluate fixed-length array length expressions lazily.
This is in preparation for polymorphic array lengths (aka `[T; T::A]`) and const generics.
We need deferred const-evaluation to break cycles when array types show up in positions which require knowing the array type to typeck the array length, e.g. the array type is in a `where` clause.
The final step - actually passing bounds in scope to array length expressions from the parent - is not done because it still produces cycles when *normalizing* `ParamEnv`s, and @nikomatsakis' in-progress lazy normalization work is needed to deal with that uniformly.
However, the changes here are still useful to unlock work on const generics, which @EpicatSupercell manifested interest in, and I might be mentoring them for that, but we need this baseline first.
r? @nikomatsakis cc @oli-obk
This'll allow us to reconstruct query parameters purely from the `DepNode`
they're associated with. Some queries could move straight to `HirId` but others
that don't always have a correspondance between `HirId` and `DefId` moved to
two-level maps where the query operates over a `DefIndex`, returning a map,
which is then keyed off `ItemLocalId`.
Closes#44414
Annotate the allocator crates (allocator_system, allocator_jemalloc) by
the type of allocator they are. If one is requested as an exe allocator,
detect its type by the flags.
This has the effect that using this (de jure wrong) configuration in the
target spec works instead of producing a really unhelpful and arcane
linker error:
"exe-allocation-crate": "alloc_system"
Fixes#43524.
This commit moves the `crates` method to a query and then migrates all callers
to use a query instead of the now-renamed `crates_untracked` method where
possible.
Closes#41417
Previously a `Symbol` was stored there, but this ended up causing hash
collisions in situations that otherwise shouldn't have a hash collision. Only
the symbol's string value was hashed, but it was possible for distinct symbols
to have the same string value, fooling various calcuations into thinking that
these paths *didn't* need disambiguating data when in fact they did!
By storing `InternedString` instead we're hopefully triggering all the exising
logic to disambiguate paths with same-name `Symbol` but actually distinct
locations.
This commit primarily removes the `stability` field from `TyCtxt` as well as its
internal mutable state, instead using a query to build the stability index as
well as primarily using queries for other related lookups.
Like previous commits the calculation of the stability index is wrapped in a
`with_ignore` node to avoid regressing the current tests, and otherwise this
commit also introduces #44232 but somewhat intentionally so.
Should hopefully more accurately reflect what's happening! This commit also
removes the cache in the cstore implementation as it's already cached through
the query infrastructure.
These are only called pre-TyCtxt (e.g. lowering/resolve), so make it explicit in
the name that they're untracked and therefore unsuitable to called elsewhere.
This commit moves the calculation of the `LanguageItems` structure into a
query rather than being calculated before the `TyCtxt` exists, with the eventual
end goal of removing some `CrateStore` methods.
The main use of `CrateStore` *before* the `TyCtxt` is created is during
resolution, but we want to be sure that any methods used before resolution are
not used after the `TyCtxt` is created. This commit starts moving the methods
used by resolve to all be named `{name}_untracked` where the rest of the
compiler uses just `{name}` as a query.
During this transition a number of new queries were added to account for
post-resolve usage of these methods.
This commit started by moving methods from `CrateStore` to queries, but it ended
up necessitating some deeper refactorings to move more items in general to
queries.
Before this commit the *resolver* would walk over the AST and process foreign
modules (`extern { .. }` blocks) and collect `#[link]` annotations. It would
then also process the command line `-l` directives and such. This information
was then stored as precalculated lists in the `CrateStore` object for iterating
over later.
After this, commit, however, this pass no longer happens during resolution but
now instead happens through queries. A query for the linked libraries of a crate
will crawl the crate for `extern` blocks and then process the linkage
annotations at that time.
This comit applies the following changes:
* Deletes the `is_allocator` query as it's no longer used
* Moves the `is_sanitizer_runtime` method to a query
* Moves the `is_profiler_runtime` method to a query
* Moves the `panic_strategy` method to a query
* Moves the `is_no_builtins` method to a query
* Deletes the cstore method of `is_compiler_builtins`. The query was added in
#42588 but the `CrateStore` method was not deleted
A good bit of these methods were used late in linking during trans so a new
dedicated structure was created to ship a calculated form of this information
over to the linker rather than having to ship the whole of `TyCtxt` over to
linking.
This map, like `trait_map`, is calculated in resolve, but we want to be sure to
track it for incremental compliation. Hide it behind a query to get more
refactorings later.
Implement From<&[T]> and others for Arc/Rc (RFC 1845)
* Implements `From<`{`&[T]`, `&str`, `String`, `Box<T> where T: ?Sized`, `Vec<T>`}`>` for `Arc`/`Rc`
* Removes `rustc_private`-marked methods `Rc::__from_array` and `Rc::__from_str`, replacing their use with `Rc::from`
Tracking issue: #40475
Implements RFC 1845, adding implementations of:
* `From<&[T]>` for `Rc<[T]>`
* `From<&str>` for `Rc<str>`
* `From<String>` for `Rc<str>`
* `From<Box<T: ?Sized>>` for `Rc<T>`
* `From<Vec<T>>` for `Rc<[T]>`
* and likewise for `Arc<_>`
Also removes now-obsolete internal methods `Rc::__from_array` and
`Rc::__from_str`, replacing their use with `Rc::from`.
Encode proper module spans in crate metadata.
The spans previously encoded only span the first token after the opening
brace, up to the closing brace of inline `mod` declarations. Thus, when
examining exports from an external crate, the spans don't include the
header of inline `mod` declarations.
r? @eddyb
The spans previously encoded only span the first token after the opening
brace, up to the closing brace of inline `mod` declarations. Thus, when
examining exports from an external crate, the spans don't include the
header of inline `mod` declarations.
This commit adds a new field to the `Item` AST node in libsyntax to optionally
contain the original token stream that the item itself was parsed from. This is
currently `None` everywhere but is intended for use later with procedural
macros.
rustc: Add some build scripts for librustc crates
This commit adds some "boilerplate" build scripts to librustc/libsyntax crates
to declare dependencies on various environment variables that are configured
throughout the build. Cargo recently gained the ability to depend on environment
variables in build scripts which can help trigger recompilation of a crate.
This should fix weird bugs where after you make a commit or a few days later
you'll get weird "not built with the same compiler" errors hopefully.
This commit adds some "boilerplate" build scripts to librustc/libsyntax crates
to declare dependencies on various environment variables that are configured
throughout the build. Cargo recently gained the ability to depend on environment
variables in build scripts which can help trigger recompilation of a crate.
This should fix weird bugs where after you make a commit or a few days later
you'll get weird "not built with the same compiler" errors hopefully.
Use similar compression settings as before updating to use flate2
Fixes#42879
(My first PR to rust-lang yay)
This changes the compression settings back to how they were before the change to use the flate2 crate rather than the in-tree flate library. The specific changes are to use the `Fast` compression level (which should be equivialent to what was used before), and use a raw deflate stream rather than wrapping the stream in a zlib wrapper. The [zlib](https://tools.ietf.org/html/rfc1950) wrapper adds an extra 2 bytes of header data, and 4 bytes for a checksum at the end. The change to use a faster compression level did give some compile speedups in the past (see #37298). Having to calculate a checksum also added a small overhead, which didn't exist before the change to flate2.
r? @alexcrichton
rustc: Implement the #[global_allocator] attribute
This PR is an implementation of [RFC 1974] which specifies a new method of
defining a global allocator for a program. This obsoletes the old
`#![allocator]` attribute and also removes support for it.
[RFC 1974]: https://github.com/rust-lang/rfcs/pull/1974
The new `#[global_allocator]` attribute solves many issues encountered with the
`#![allocator]` attribute such as composition and restrictions on the crate
graph itself. The compiler now has much more control over the ABI of the
allocator and how it's implemented, allowing much more freedom in terms of how
this feature is implemented.
cc #27389
This PR is an implementation of [RFC 1974] which specifies a new method of
defining a global allocator for a program. This obsoletes the old
`#![allocator]` attribute and also removes support for it.
[RFC 1974]: https://github.com/rust-lang/rfcs/pull/197
The new `#[global_allocator]` attribute solves many issues encountered with the
`#![allocator]` attribute such as composition and restrictions on the crate
graph itself. The compiler now has much more control over the ABI of the
allocator and how it's implemented, allowing much more freedom in terms of how
this feature is implemented.
cc #27389
Replaced by adding extra imports, adding hidden code (`# ...`), modifying
examples to be runnable (sorry Homura), specifying non-Rust code, and
converting to should_panic, no_run, or compile_fail.
Remaining "```ignore"s received an explanation why they are being ignored.
A long time coming this commit removes the `flate` crate in favor of the
`flate2` crate on crates.io. The functionality in `flate2` originally flowered
out of `flate` itself and is additionally the namesake for the crate. This will
leave a gap in the naming (there's not `flate` crate), which will likely cause a
particle collapse of some form somewhere.
Implement lazy loading of external crates' sources. Fixes#38875Fixes#38875. This is a follow-up to #42507. When a (now correctly translated) span from an external crate is referenced in a error, warning or info message, we still don't have the source code being referenced.
Since stuffing the source in the serialized metadata of an rlib is extremely wasteful, the following scheme has been implemented:
* File maps now contain a source hash that gets serialized as well.
* When a span is rendered in a message, the source hash in the corresponding file map(s) is used to try and load the source from the corresponding file on disk. If the file is not found or the hashes don't match, the failed attempt is recorded (and not retried).
* The machinery fetching source lines from file maps is augmented to use the lazily loaded external source as a secondary fallback for file maps belonging to external crates.
This required a small change to the expected stderr of one UI test (it now renders a span, where previously was none).
Further work can be done based on this - some of the machinery previously used to hide external spans is possibly obsolete and the hashing code can be reused in different places as well.
r? @eddyb
Build instruction profiler runtime as part of compiler-rt
r? @alexcrichton
This is #38608 with some fixes.
Still missing:
- [x] testing with profiler enabled on some builders (on which ones? Should I add the option to some of the already existing configurations, or create a new configuration?);
- [x] enabling distribution (on which builders?);
- [x] documentation.
incr.comp.: Make DepNode `Copy` and valid across compilation sessions
This PR moves `DepNode` to a representation that does not need retracing and thus simplifies comparing dep-graphs from different compilation sessions. The code also gets a lot simpler in many places, since we don't need the generic parameter on `DepNode` anymore. See https://github.com/rust-lang/rust/issues/42294 for details.
~~NOTE: Only the last commit of this is new, the rest is already reviewed in https://github.com/rust-lang/rust/pull/42504.~~
This PR is almost done but there are some things I still want to do:
- [x] Add some module-level documentation to `dep_node.rs`, explaining especially what the `define_dep_nodes!()` macro is about.
- [x] Do another pass over the dep-graph loading logic. I suspect that we can get rid of building the `edges` map and also use arrays instead of hash maps in some places.
cc @rust-lang/compiler
r? @nikomatsakis
We can use these to perform lazy loading of source files belonging to
external crates. That way we will be able to show the source code of
external spans that have been translated.
Fix translation of external spans
Previously, I noticed that spans from external crates don't generate any output. This limitation is problematic if analysis is performed on one or more external crates, as is the case with [rust-semverver](https://github.com/ibabushkin/rust-semverver). This change should address this behaviour, with the potential drawback that a minor performance hit is to be expected, as spans from potentially large crates have to be translated now.
incr.comp.: Track expanded spans instead of FileMaps.
This PR removes explicit tracking of FileMaps in response to #42101. The reasoning behind being able to just *not* track access to FileMaps is similar to why we don't track access to the `DefId->DefPath` map:
1. One can only get ahold of a `Span` value by accessing the HIR (for local things) or a `metadata::schema::Entry` (for things from external crates).
2. For both of these things we compute a hash that incorporates the *expanded spans*, that is, what we hash is in the (FileMap independent) format `filename:line:col`.
3. Consequently, everything that emits a span should already be tracked via its dependency to something that has the span included in its hash and changes would be detected via that hash.
One caveat here is that we have to be conservative when exporting things in metadata. A crate can be built without debuginfo and would thus by default not incorporate most spans into the metadata hashes. However, a downstream crate can make an inline copy of things in the upstream crate and span changes in the upstream crate would then go undetected, even if the downstream uses them (e.g. by emitting debuginfo for an inlined function). For this reason, we always incorporate spans into metadata hashes for now (there might be more efficient ways to handle this safely when red-green tracking is implemented).
r? @nikomatsakis
Remove interior mutability from TraitDef by turning fields into queries
This PR gets rid of anything `std::cell` in `TraitDef` by
- moving the global list of trait impls from `TraitDef` into a query,
- moving the list of trait impls relevent for some self-type from `TraitDef` into a query
- moving the specialization graph of trait impls into a query, and
- moving `TraitDef::object_safety` into a query.
I really like how querifying things not only helps with incremental compilation and on-demand, but also just plain makes the code cleaner `:)`
There are also some smaller fixes in the PR. Commits can be reviewed separately.
r? @eddyb or @nikomatsakis
Move the code for loading metadata from rlibs and dylibs from
rustc_metadata into rustc_trans, and introduce a trait to avoid
introducing a direct dependency on rustc_trans.
This means rustc_metadata is no longer rebuilt when LLVM changes.
rustc: Add a new `-Z force-unstable-if-unmarked` flag
This commit adds a new `-Z` flag to the compiler for use when bootstrapping the
compiler itself. We want to be able to use crates.io crates, but we also want
the usage of such crates to be as ergonomic as possible! To that end compiler
crates are a little tricky in that the crates.io crates are not annotated as
unstable, nor do they expect to pull in unstable dependencies.
To cover all these situations it's intended that the compiler will forever now
bootstrap with `-Z force-unstable-if-unmarked`. This flags serves a dual purpose
of forcing crates.io crates to themselves be unstable while also allowing them
to use other "unstable" crates.io crates. This should mean that adding a
dependency to compiler no longer requires upstream modification with
unstable/staged_api attributes for inclusion!
This commit adds a new `-Z` flag to the compiler for use when bootstrapping the
compiler itself. We want to be able to use crates.io crates, but we also want
the usage of such crates to be as ergonomic as possible! To that end compiler
crates are a little tricky in that the crates.io crates are not annotated as
unstable, nor do they expect to pull in unstable dependencies.
To cover all these situations it's intended that the compiler will forever now
bootstrap with `-Z force-unstable-if-unmarked`. This flags serves a dual purpose
of forcing crates.io crates to themselves be unstable while also allowing them
to use other "unstable" crates.io crates. This should mean that adding a
dependency to compiler no longer requires upstream modification with
unstable/staged_api attributes for inclusion!
incr.comp.: Hash more pieces of crate metadata to detect changes there.
This PR adds incr. comp. hashes for non-`Entry` pieces of data in crate metadata.
The first part of it I like: `EntryBuilder` is refactored into the more generally applicable `IsolatedEncoder` which provides means of encoding something into metadata while also feeding the encoded data into an incr. comp. hash. We already did this for `Entry`, now we are doing it for various other pieces of data too, like the set of exported symbols and so on. The hashes generated there are persisted together with the per-`Entry` hashes and are also used for dep-graph dirtying the same way.
The second part of the PR I'm not entirely happy with: In order to make sure that we don't forget registering a read to the new `DepNodes` introduced here, I added the `Tracked<T>` struct. This struct wraps a value and requires a `DepNode` when accessing the wrapped value. This makes it harder to overlook adding read edges in the right places and works just fine.
However, crate metadata is already used in places where there is no `tcx` yet or even in places where no `cnum` has been assigned -- this makes it harder to apply this feature consistently or implement it ergonomically. The result is not too bad but there's a bit more code churn and a bit more opportunity to get something wrong than I would have liked. On the other hand, wrapping things in `Tracked<T>` already has revealed some bugs, so there's definitely some value in it.
This is still a work in progress:
- [x] I need to write some test cases.
- [x] Accessing the CodeMap should really be dependency tracked too, especially with the new path-remapping feature.
cc @nikomatsakis
Refactor variance and remove last `[pub]` map
This PR refactors variance to work in a more red-green friendly way. Because red-green doesn't exist yet, it has to be a bit hacky. The basic idea is this:
- We compute a big map with the variance for all items in the crate; when you request variances for a particular item, we read it from the crate
- We now hard-code that traits are invariant (which they are, for deep reasons, not gonna' change)
- When building constraints, we compute the transitive closure of all things within the crate that depend on what using `TransitiveRelation`
- this lets us gin up the correct dependencies when requesting variance of a single item
Ah damn, just remembered, one TODO:
- [x] Update the variance README -- ah, I guess the README updates I did are sufficient
r? @michaelwoerister
