Build instruction profiler runtime as part of compiler-rt
r? @alexcrichton
This is #38608 with some fixes.
Still missing:
- [x] testing with profiler enabled on some builders (on which ones? Should I add the option to some of the already existing configurations, or create a new configuration?);
- [x] enabling distribution (on which builders?);
- [x] documentation.
incr.comp.: Make DepNode `Copy` and valid across compilation sessions
This PR moves `DepNode` to a representation that does not need retracing and thus simplifies comparing dep-graphs from different compilation sessions. The code also gets a lot simpler in many places, since we don't need the generic parameter on `DepNode` anymore. See https://github.com/rust-lang/rust/issues/42294 for details.
~~NOTE: Only the last commit of this is new, the rest is already reviewed in https://github.com/rust-lang/rust/pull/42504.~~
This PR is almost done but there are some things I still want to do:
- [x] Add some module-level documentation to `dep_node.rs`, explaining especially what the `define_dep_nodes!()` macro is about.
- [x] Do another pass over the dep-graph loading logic. I suspect that we can get rid of building the `edges` map and also use arrays instead of hash maps in some places.
cc @rust-lang/compiler
r? @nikomatsakis
We can use these to perform lazy loading of source files belonging to
external crates. That way we will be able to show the source code of
external spans that have been translated.
Fix translation of external spans
Previously, I noticed that spans from external crates don't generate any output. This limitation is problematic if analysis is performed on one or more external crates, as is the case with [rust-semverver](https://github.com/ibabushkin/rust-semverver). This change should address this behaviour, with the potential drawback that a minor performance hit is to be expected, as spans from potentially large crates have to be translated now.
incr.comp.: Track expanded spans instead of FileMaps.
This PR removes explicit tracking of FileMaps in response to #42101. The reasoning behind being able to just *not* track access to FileMaps is similar to why we don't track access to the `DefId->DefPath` map:
1. One can only get ahold of a `Span` value by accessing the HIR (for local things) or a `metadata::schema::Entry` (for things from external crates).
2. For both of these things we compute a hash that incorporates the *expanded spans*, that is, what we hash is in the (FileMap independent) format `filename:line:col`.
3. Consequently, everything that emits a span should already be tracked via its dependency to something that has the span included in its hash and changes would be detected via that hash.
One caveat here is that we have to be conservative when exporting things in metadata. A crate can be built without debuginfo and would thus by default not incorporate most spans into the metadata hashes. However, a downstream crate can make an inline copy of things in the upstream crate and span changes in the upstream crate would then go undetected, even if the downstream uses them (e.g. by emitting debuginfo for an inlined function). For this reason, we always incorporate spans into metadata hashes for now (there might be more efficient ways to handle this safely when red-green tracking is implemented).
r? @nikomatsakis
Remove interior mutability from TraitDef by turning fields into queries
This PR gets rid of anything `std::cell` in `TraitDef` by
- moving the global list of trait impls from `TraitDef` into a query,
- moving the list of trait impls relevent for some self-type from `TraitDef` into a query
- moving the specialization graph of trait impls into a query, and
- moving `TraitDef::object_safety` into a query.
I really like how querifying things not only helps with incremental compilation and on-demand, but also just plain makes the code cleaner `:)`
There are also some smaller fixes in the PR. Commits can be reviewed separately.
r? @eddyb or @nikomatsakis
Move the code for loading metadata from rlibs and dylibs from
rustc_metadata into rustc_trans, and introduce a trait to avoid
introducing a direct dependency on rustc_trans.
This means rustc_metadata is no longer rebuilt when LLVM changes.
rustc: Add a new `-Z force-unstable-if-unmarked` flag
This commit adds a new `-Z` flag to the compiler for use when bootstrapping the
compiler itself. We want to be able to use crates.io crates, but we also want
the usage of such crates to be as ergonomic as possible! To that end compiler
crates are a little tricky in that the crates.io crates are not annotated as
unstable, nor do they expect to pull in unstable dependencies.
To cover all these situations it's intended that the compiler will forever now
bootstrap with `-Z force-unstable-if-unmarked`. This flags serves a dual purpose
of forcing crates.io crates to themselves be unstable while also allowing them
to use other "unstable" crates.io crates. This should mean that adding a
dependency to compiler no longer requires upstream modification with
unstable/staged_api attributes for inclusion!
This commit adds a new `-Z` flag to the compiler for use when bootstrapping the
compiler itself. We want to be able to use crates.io crates, but we also want
the usage of such crates to be as ergonomic as possible! To that end compiler
crates are a little tricky in that the crates.io crates are not annotated as
unstable, nor do they expect to pull in unstable dependencies.
To cover all these situations it's intended that the compiler will forever now
bootstrap with `-Z force-unstable-if-unmarked`. This flags serves a dual purpose
of forcing crates.io crates to themselves be unstable while also allowing them
to use other "unstable" crates.io crates. This should mean that adding a
dependency to compiler no longer requires upstream modification with
unstable/staged_api attributes for inclusion!
incr.comp.: Hash more pieces of crate metadata to detect changes there.
This PR adds incr. comp. hashes for non-`Entry` pieces of data in crate metadata.
The first part of it I like: `EntryBuilder` is refactored into the more generally applicable `IsolatedEncoder` which provides means of encoding something into metadata while also feeding the encoded data into an incr. comp. hash. We already did this for `Entry`, now we are doing it for various other pieces of data too, like the set of exported symbols and so on. The hashes generated there are persisted together with the per-`Entry` hashes and are also used for dep-graph dirtying the same way.
The second part of the PR I'm not entirely happy with: In order to make sure that we don't forget registering a read to the new `DepNodes` introduced here, I added the `Tracked<T>` struct. This struct wraps a value and requires a `DepNode` when accessing the wrapped value. This makes it harder to overlook adding read edges in the right places and works just fine.
However, crate metadata is already used in places where there is no `tcx` yet or even in places where no `cnum` has been assigned -- this makes it harder to apply this feature consistently or implement it ergonomically. The result is not too bad but there's a bit more code churn and a bit more opportunity to get something wrong than I would have liked. On the other hand, wrapping things in `Tracked<T>` already has revealed some bugs, so there's definitely some value in it.
This is still a work in progress:
- [x] I need to write some test cases.
- [x] Accessing the CodeMap should really be dependency tracked too, especially with the new path-remapping feature.
cc @nikomatsakis
Refactor variance and remove last `[pub]` map
This PR refactors variance to work in a more red-green friendly way. Because red-green doesn't exist yet, it has to be a bit hacky. The basic idea is this:
- We compute a big map with the variance for all items in the crate; when you request variances for a particular item, we read it from the crate
- We now hard-code that traits are invariant (which they are, for deep reasons, not gonna' change)
- When building constraints, we compute the transitive closure of all things within the crate that depend on what using `TransitiveRelation`
- this lets us gin up the correct dependencies when requesting variance of a single item
Ah damn, just remembered, one TODO:
- [x] Update the variance README -- ah, I guess the README updates I did are sufficient
r? @michaelwoerister
This is a more principled version of the `RefCell` we were using
before. We now allocate a `Steal<Mir<'tcx>>` for each intermediate MIR
pass; when the next pass steals the entry, any later attempts to use it
will panic (there is no way to *test* if MIR is stolen, you're just
supposed to *know*).
The new setup is as follows. There is a pipeline of MIR passes that each
run **per def-id** to optimize a particular function. You are intended
to request MIR at whatever stage you need it. At the moment, there is
only one stage you can request:
- `optimized_mir(def_id)`
This yields the final product. Internally, it pulls the MIR for the
given def-id through a series of steps. Right now, these are still using
an "interned ref-cell" but they are intended to "steal" from one
another:
- `mir_build` -- performs the initial construction for local MIR
- `mir_pass_set` -- performs a suite of optimizations and transformations
- `mir_pass` -- an individual optimization within a suite
So, to construct the optimized MIR, we invoke:
mir_pass_set((MIR_OPTIMIZED, def_id))
which will build up the final MIR.
On demandify region mapping
This is an adaptation of @cramertj's PR. I am sort of tempted to keep simplifying it, but also tempted to land it so and we can refactor more in follow-up PRs. As is, it does the following things:
- makes the region-maps an on-demand query, per function `tcx.region_maps(def_id)`
- interns code extents instead of of having them be integers
- remove the "root region extent" and (to some extent) item extents; instead we use `Option<CodeExtent<'tcx>>` in a few places (no space inefficiency since `CodeExtent<'tcx>` is now a pointer).
I'm not entirely happy with the way I have it setup though. Here are some of the changes I was considering (I'm not sure if they would work out well):
1. Removing `item_extents` entirely -- they are rarely used now, because most of the relevant places now accept an `Option<Region<'tcx>>` or an `Option<CodeExtent<'tcx>>`, but I think still used in a few places.
2. Merging `RegionMaps` into the typeck tables, instead of having it be its own query.
3. Change `CodeExtent<'tcx>` to store the parent pointer. This would mean that fewer places in the code actually *need* a `RegionMaps` anyhow, since most of them just want to be able to walk "up the tree". On the other hand, you wouldn't be able to intern a `CodeExtent<'tcx>` for some random node-id, you'd need to look it up in the table (since there'd be more information).
Most of this code is semi-temporary -- I expect it to largely go away as we move to NLL -- so I'm also not *that* concerned with making it perfect.
r? @eddyb
When -Z profile is passed, the GCDAProfiling LLVM pass is added
to the pipeline, which uses debug information to instrument the IR.
After compiling with -Z profile, the $(OUT_DIR)/$(CRATE_NAME).gcno
file is created, containing initial profiling information.
After running the program built, the $(OUT_DIR)/$(CRATE_NAME).gcda
file is created, containing branch counters.
The created *.gcno and *.gcda files can be processed using
the "llvm-cov gcov" and "lcov" tools. The profiling data LLVM
generates does not faithfully follow the GCC's format for *.gcno
and *.gcda files, and so it will probably not work with other tools
(such as gcov itself) that consume these files.
make *most* maps private
Currently we access the `DepTrackingMap` fields directly rather than using the query accessors. This seems bad. This branch removes several such uses, but not all, and extends the macro so that queries can hide their maps (so we can prevent regressions). The extension to the macro is kind of ugly :/ but couldn't find a simple way to do it otherwise (I guess I could use a nested macro...). Anyway I figure it's only temporary.
r? @eddyb
Implement a file-path remapping feature in support of debuginfo and reproducible builds
This PR adds the `-Zremap-path-prefix-from`/`-Zremap-path-prefix-to` commandline option pair and is a more general implementation of #41419. As opposed to the previous attempt, this implementation should enable reproducible builds regardless of the working directory of the compiler.
This implementation of the feature is more general in the sense that the re-mapping will affect *all* paths the compiler emits, including the ones in error messages.
r? @alexcrichton
#37653 support `default impl` for specialization
this commit implements the first step of the `default impl` feature:
> all items in a `default impl` are (implicitly) `default` and hence
> specializable.
In order to test this feature I've copied all the tests provided for the
`default` method implementation (in run-pass/specialization and
compile-fail/specialization directories) and moved the `default` keyword
from the item to the impl.
See [referenced](https://github.com/rust-lang/rust/issues/37653) issue for further info
r? @aturon
Support AddressSanitizer and ThreadSanitizer on x86_64-apple-darwin
[ASan](https://clang.llvm.org/docs/AddressSanitizer.html#supported-platforms) and [TSan](https://clang.llvm.org/docs/ThreadSanitizer.html#supported-platforms) are supported on macOS, and this commit enables their support.
The sanitizers are always built as `*.dylib` on Apple platforms, so they cannot be statically linked into the corresponding `rustc_?san.rlib`. The dylibs are directly copied to `lib/rustlib/x86_64-apple-darwin/lib/` instead.
Note, although Xcode also ships with their own copies of ASan/TSan dylibs, we cannot use them due to version mismatch.
----
~~There is a caveat: the sanitizer libraries are linked as `@rpath/` (due to https://reviews.llvm.org/D6018), so the user needs to additionally pass `-C rpath`:~~
**Edit:** Passing rpath is now automatic.
ASan and TSan are supported on macOS, and this commit enables their
support.
The sanitizers are always built as *.dylib on Apple platforms, so they
cannot be statically linked into the corresponding `rustc_?san.rlib`. The
dylibs are directly copied to `lib/rustlib/x86_64-apple-darwin/lib/`
instead.
Note, although Xcode also ships with their own copies of ASan/TSan dylibs,
we cannot use them due to version mismatch.
There is a caveat: the sanitizer libraries are linked as @rpath, so the
user needs to additionally pass `-C rpath`:
rustc -Z sanitizer=address -C rpath file.rs
^~~~~~~~
Otherwise there will be a runtime error:
dyld: Library not loaded: @rpath/libclang_rt.asan_osx_dynamic.dylib
Referenced from: /path/to/executable
Reason: image not found
Abort trap: 6
The next commit includes a temporary change in compiler to force the linker
to emit a usable @rpath.
this commit implements the first step of the `default impl` feature:
all items in a `default impl` are (implicitly) `default` and hence
specializable.
In order to test this feature I've copied all the tests provided for the
`default` method implementation (in run-pass/specialization and
compile-fail/specialization directories) and moved the `default` keyword
from the item to the impl.
See referenced issue for further info
this avoids parsing item attributes on each call to `item_attrs`, which takes
off 33% (!) of translation time and 50% (!) of trans-item collection time.
This may seem like overkill, but it's exactly what we want/need for
incremental compilation I think. In particular, while generating code
for some codegen unit X, we can wind up querying about any number of
external items, and we only want to be forced to rebuild X is some of
those changed from a foreign item to otherwise. Factoring this into a
query means we would re-run only if some `false` became `true` (or vice
versa).
Implement global_asm!() (RFC 1548)
This is a first attempt. ~~One (potential) problem I haven't solved is how to handle multiple usages of `global_asm!` in a module/crate. It looks like `LLVMSetModuleInlineAsm` overwrites module asm, and `LLVMAppendModuleInlineAsm` is not provided in LLVM C headers 😦~~
I can provide more detail as needed, but honestly, there's not a lot going on here.
r? @eddyb
CC @Amanieu @jackpot51
Tracking issue: #35119
A number of things were using `crate_hash` that really ought to be using
`crate_disambiguator` (e.g., to create the plugin symbol names). They
have been updated.
It is important to remove `LinkMeta` from `SharedCrateContext` since it
contains a hash of the entire crate, and hence it will change
whenever **anything** changes (which would then require
rebuilding **everything**).
Windows builder croaked. This change tries to fix that by actually
calling the global_asm-defined function so the symbol doesn't get
optimized away, if that is in fact what was happening.
Additionally, we provide an empty main() for non-x86 arches.
Instead of collecting all potential inputs to some metadata entry and
hashing those, we directly hash the values we are storing in metadata.
This is more accurate and doesn't suffer from quadratic blow-up when
many entries have the same dependencies.
cstore: return an immutable borrow from `visible_parent_map`
This prevents an ICE when `visible_parent_map` is called multiple times, for example when an item referenced in an impl signature is imported from an `extern crate` statement occurs within an impl.
Fixes#41053.
r? @eddyb
rustbuild: Update bootstrap compiler
Now that we've also updated cargo's release process this commit also changes the
download location of Cargo from Cargos archives back to the static.r-l.o
archives. This should ensure that the Cargo download is the exact Cargo paired
with the rustc that we release.
Now that we've also updated cargo's release process this commit also changes the
download location of Cargo from Cargos archives back to the static.r-l.o
archives. This should ensure that the Cargo download is the exact Cargo paired
with the rustc that we release.
on-demand-ify `custom_coerce_unsized_kind` and `inherent-impls`
This "on-demand" task both checks for errors and computes the custom unsized kind, if any. This task is only defined on impls of `CoerceUnsized`; invoking it on any other kind of impl results in a bug. This is just to avoid having an `Option`, could easily be changed.
r? @eddyb
This commit deletes the internal liblog in favor of the implementation that
lives on crates.io. Similarly it's also setting a convention for adding crates
to the compiler. The main restriction right now is that we want compiler
implementation details to be unreachable from normal Rust code (e.g. requires a
feature), and by default everything in the sysroot is reachable via `extern
crate`.
The proposal here is to require that crates pulled in have these lines in their
`src/lib.rs`:
#![cfg_attr(rustbuild, feature(staged_api, rustc_private))]
#![cfg_attr(rustbuild, unstable(feature = "rustc_private", issue = "27812"))]
This'll mean that by default they're not using these attributes but when
compiled as part of the compiler they do a few things:
* Mark themselves as entirely unstable via the `staged_api` feature and the
`#![unstable]` attribute.
* Allow usage of other unstable crates via `feature(rustc_private)` which is
required if the crate relies on any other crates to compile (other than std).
There are now 3 queries:
- inherent_impls(def-id): for a given type, get a `Rc<Vec<DefId>>` with
all its inherent impls. This internally uses `crate_inherent_impls`,
doing some hacks to keep the current deps (which, btw, are not clearly
correct).
- crate_inherent_impls(crate): gathers up a map from types
to `Rc<Vec<DefId>>`, touching the entire krate, possibly generating
errors.
- crate_inherent_impls_overlap_check(crate): performs overlap checks
between the inherent impls for a given type, generating errors.
This "on-demand" task both checks for errors and computes the custom
unsized kind, if any. This task is only defined on impls of
`CoerceUnsized`; invoking it on any other kind of impl results in a bug.
This is just to avoid having an `Option`, could easily be changed.
Remove unused adt-def insertion by constructor DefIndex
It looks to me like ADT definitions weren't being looked up by constructor id, and a test run supports my theory.
In any case, I'm not sure it would have worked in its current configuration. If I understand correctly, the `adt_def` map entry from constructor id -> adt def would only be present after a successful call to `queries::adt_def::get` with the proper ADT `DefIndex`. Trying to look up an adt_def by the constructor index prior to a successful lookup by ADT index would fail since `item.kind` would be `EntryKind::Fn` (for the constructor function) and so would trigger the `bug!`.
r? @nikomatsakis
Correctly get source for metatdata-only crate type
Closes#40535
However, I'm not sure how to approach writing a regression test since I'm still working on a reduced test case from the code that caused the ICE in the first place. It's not enough to have an unknown `extern crate` in a metadata crate, it depends on a few extra arguments but I'm not sure which yet.
Also replaced the `unwrap()` with a more informative `expect()`.
r? @jseyfried
Introduce HirId, a replacement for ast::NodeId after lowering to HIR
This is the first step towards implementing #40303. This PR introduces the `HirId` type and generates a `HirId` for everything that would be assigned one (i.e. stuff in the HIR), but the HIR data types still use `NodeId` for now. Changing that is a big refactoring that I want to do in a separate PR.
A `HirId` uniquely identifies a node in the HIR of the current crate. It is composed of the `owner`, which is the `DefIndex` of the directly enclosing `hir::Item`, `hir::TraitItem`, or `hir::ImplItem` (i.e. the closest "item-like"), and the `local_id` which is unique within the given owner.
This PR is also running a number of consistency checks for the generated `HirId`s:
- Does `NodeId` in the HIR have a corresponding `HirId`?
- Is the `owner` part of each `HirId` consistent with its position in the HIR?
- Do the numerical values of the `local_id` part all lie within a dense range of integers?
cc @rust-lang/compiler
r? @eddyb or @nikomatsakis
This commit is a random stab in the dark to fix the spurious failures on #39518.
The leading theory of the spurious failures on Windows is that the compiler is
loading a path in the `deps` folder, passing it to `link.exe`, and then this is
racing with Cargo itself updating those paths.
This race, however, has a few unique properties:
* It's isolated to just libstd. Most crates are never passed to the linker and
simultaneously being worked on by Cargo. Cargo's typical execution of the
dependency graph never hits this problem.
* The crates are already all located in the sysroot in addition to the `deps`
folder. This means that the compiler actually has two candidates of crates to
load, and it's just arbitrarily rejecting one.
Together this means that we shouldn't need to fix this problem "in the large"
and we can instead just fix it in this isolated situation (hopefully). To solve
this the compiler's been updated to prefer crates from the sysroot to leave
Cargo's structure to itself.
We'll see if this actually allows the PR to land...
convert AdtDef::destructor to on-demand
This removes the `Cell` from `AdtDef`. Also, moving destructor validity
checking to on-demand (forced during item-type checking) ensures that
invalid destructors can't cause ICEs.
Fixes#38868.
Fixes#40132.
r? @eddyb
Implement function-like procedural macros ( `#[proc_macro]`)
Adds the `#[proc_macro]` attribute, which expects bare functions of the kind `fn(TokenStream) -> TokenStream`, which can be invoked like `my_macro!()`.
cc rust-lang/rfcs#1913, #38356
r? @jseyfried
cc @nrc
This removes the Cell from AdtDef. Also, moving destructor validity
checking to on-demand (forced during item-type checking) ensures that
invalid destructors can't cause ICEs.
Fixes#38868.
Fixes#40132.
[MIR] SwitchInt Everywhere
Something I've been meaning to do for a very long while. This PR essentially gets rid of 3 kinds of conditional branching and only keeps the most general one - `SwitchInt`. Primary benefits are such that dealing with MIR now does not involve dealing with 3 different ways to do conditional control flow. On the other hand, constructing a `SwitchInt` currently requires more code than what previously was necessary to build an equivalent `If` terminator. Something trivially "fixable" with some constructor methods somewhere (MIR needs stuff like that badly in general).
Some timings (tl;dr: slightly faster^1 (unexpected), but also uses slightly more memory at peak (expected)):
^1: Not sure if the speed benefits are because of LLVM liking the generated code better or the compiler itself getting compiled better. Either way, its a net benefit. The CORE and SYNTAX timings done for compilation without optimisation.
```
AFTER:
Building stage1 std artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
Finished release [optimized] target(s) in 31.50 secs
Finished release [optimized] target(s) in 31.42 secs
Building stage1 compiler artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
Finished release [optimized] target(s) in 439.56 secs
Finished release [optimized] target(s) in 435.15 secs
CORE: 99% (24.81 real, 0.13 kernel, 24.57 user); 358536k resident
CORE: 99% (24.56 real, 0.15 kernel, 24.36 user); 359168k resident
SYNTAX: 99% (49.98 real, 0.48 kernel, 49.42 user); 653416k resident
SYNTAX: 99% (50.07 real, 0.58 kernel, 49.43 user); 653604k resident
BEFORE:
Building stage1 std artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
Finished release [optimized] target(s) in 31.84 secs
Building stage1 compiler artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
Finished release [optimized] target(s) in 451.17 secs
CORE: 99% (24.66 real, 0.20 kernel, 24.38 user); 351096k resident
CORE: 99% (24.36 real, 0.17 kernel, 24.18 user); 352284k resident
SYNTAX: 99% (52.24 real, 0.56 kernel, 51.66 user); 645544k resident
SYNTAX: 99% (51.55 real, 0.48 kernel, 50.99 user); 646428k resident
```
cc @nikomatsakis @eddyb
Previously AdtDef variants contained ConstInt for each discriminant, which did not really reflect
the actual type of the discriminants. Moving the type into AdtDef allows to easily put the type
into metadata and also saves bytes from ConstVal overhead for each discriminant.
Also arguably the code is cleaner now :)
Make reprs use a structured representation instead of a slice
This is needed for `-z reorder-fields`. The old design uses a slice taken from HIR, plus a cache that lazily parses. The new design stores it directly in the `AdtDef` as a `ReprOptions`. We're doing this now because we need to be able to add reprs that don't necessarily exist in HIR for `-z reorder-fields`, but it needs to happen anyway.
`lookup_repr_hints` should be mostly deprecated. I want to remove it from `layout` before closing this, unless people think that should be a separate PR. The `[WIP]` is because of this. The problem with closing this as-is is that the code here isn't actually testable until some parts of the compiler start using it.
r? @eddyb
Expand derive macros in the MacroExpander
This removes the expand_derives function, and sprinkles the functionality throughout the Invocation Collector, Expander and Resolver.
Fixes https://github.com/rust-lang/rust/issues/39326
r? @jseyfried
Implement kind="static-nobundle" (RFC 1717)
This implements the "static-nobundle" library kind (last item from #37403).
Rustc handles "static-nobundle" libs very similarly to dylibs, except that on Windows, uses of their symbols do not get marked with "dllimport". Which is the whole point of this feature.
This commit updates the version number to 1.17.0 as we're not on that version of
the nightly compiler, and at the same time this updates src/stage0.txt to
bootstrap from freshly minted beta compiler and beta Cargo.
Refactor the parser to consume token trees
This is groundwork for efficiently parsing attribute proc macro invocations, bang macro invocations, and `TokenStream`-based attributes and fragment matchers.
This improves parsing performance by 8-15% and expansion performance by 0-5% on a sampling of the compiler's crates.
r? @nrc
These are static libraries that are not bundled (as the name implies) into rlibs and staticlibs that rustc generates,
and must be present when the final binary artifact is being linked.
* Add support for `#[proc_macro]`
* Reactivate `proc_macro` feature and gate `#[proc_macro_attribute]` under it
* Have `#![feature(proc_macro)]` imply `#![feature(use_extern_macros)]`,
error on legacy import of proc macros via `#[macro_use]`
[11/n] Separate ty::Tables into one per each body.
_This is part of a series ([prev](https://github.com/rust-lang/rust/pull/38449) | [next]()) of patches designed to rework rustc into an out-of-order on-demand pipeline model for both better feature support (e.g. [MIR-based](https://github.com/solson/miri) early constant evaluation) and incremental execution of compiler passes (e.g. type-checking), with beneficial consequences to IDE support as well.
If any motivation is unclear, please ask for additional PR description clarifications or code comments._
<hr>
In order to track the results of type-checking and inference for incremental recompilation, they must be stored separately for each function or constant value, instead of lumped together.
These side-`Tables` also have to be tracked by various passes, as they visit through bodies (all of which have `Tables`, even if closures share the ones from their parent functions). This is usually done by switching a `tables` field in an override of `visit_nested_body` before recursing through `visit_body`, to the relevant one and then restoring it - however, in many cases the nesting is unnecessary and creating the visitor for each body in the crate and then visiting that body, would be a much cleaner solution.
To simplify handling of inlined HIR & its side-tables, their `NodeId` remapping and entries HIR map were fully stripped out, which means that `NodeId`s from inlined HIR must not be used where a local `NodeId` is expected. It might be possible to make the nodes (`Expr`, `Block`, `Pat`, etc.) that only show up within a `Body` have IDs that are scoped to that `Body`, which would also allow `Tables` to use `Vec`s.
That last part also fixes#38790 which was accidentally introduced in a previous refactor.
Remove not(stage0) from deny(warnings)
Historically this was done to accommodate bugs in lints, but there hasn't been a
bug in a lint since this feature was added which the warnings affected. Let's
completely purge warnings from all our stages by denying warnings in all stages.
This will also assist in tracking down `stage0` code to be removed whenever
we're updating the bootstrap compiler.
This commit stabilizes the `proc_macro` and `proc_macro_lib` features in the
compiler to stabilize the "Macros 1.1" feature of the language. Many more
details can be found on the tracking issue, #35900.
Closes#35900
This commit introduces 128-bit integers. Stage 2 builds and produces a working compiler which
understands and supports 128-bit integers throughout.
The general strategy used is to have rustc_i128 module which provides aliases for iu128, equal to
iu64 in stage9 and iu128 later. Since nowhere in rustc we rely on large numbers being supported,
this strategy is good enough to get past the first bootstrap stages to end up with a fully working
128-bit capable compiler.
In order for this strategy to work, number of locations had to be changed to use associated
max_value/min_value instead of MAX/MIN constants as well as the min_value (or was it max_value?)
had to be changed to use xor instead of shift so both 64-bit and 128-bit based consteval works
(former not necessarily producing the right results in stage1).
This commit includes manual merge conflict resolution changes from a rebase by @est31.
Historically this was done to accommodate bugs in lints, but there hasn't been a
bug in a lint since this feature was added which the warnings affected. Let's
completely purge warnings from all our stages by denying warnings in all stages.
This will also assist in tracking down `stage0` code to be removed whenever
we're updating the bootstrap compiler.
Use `DefId`s instead of `NodeId`s for `pub(restricted)` visibilities
This is groundwork for hygiene 2.0, specifically privacy checking hygienic intercrate name resolutions.
r? @nrc
The Markdown engine used by the book can cope with a single leading space
on the list marker:
Like this:
* List item
Rather than like this:
* List item
… but it’s not the typical convention employed in the book, and moreover
the Markdown engine used for producing the error index *can’t* cope with
it (its behaviour looks like a bug, as it appears to lose one of the two
line breaks as well, but that’s immaterial here).
So, we shift to a single convention which doesn’t trigger bugs in the
Markdown renderer.
Cleanup refactoring around DefPath handling
This PR makes two big changes:
* All DefPaths of a crate are now stored in metadata in their own table (as opposed to `DefKey`s as part of metadata `Entry`s.
* The compiler will no longer allocate a pseudo-local DefId for inlined HIR nodes (because those are gross). Inlined HIR nodes will have a NodeId but they don't have there own DefId anymore. Turns out they were not needed anymore either. Hopefully HIR inlining will be gone completely one day but if until then we start needing to be able to map inlined NodeIds to original DefIds, we can add an additional table to metadata that allows for reconstructing this.
Overall this makes for some nice simplifications and removal of special cases.
r? @eddyb
cc @rust-lang/compiler
Fix unaligned loads in librustc_metadata
Creating a `&[u32]` from an `&[u8]` is not necessarily valid, and crashes
on certain platforms if the data is not well aligned.
[9/n] rustc: move type information out of AdtDef and TraitDef.
_This is part of a series ([prev](https://github.com/rust-lang/rust/pull/37688) | [next]()) of patches designed to rework rustc into an out-of-order on-demand pipeline model for both better feature support (e.g. [MIR-based](https://github.com/solson/miri) early constant evaluation) and incremental execution of compiler passes (e.g. type-checking), with beneficial consequences to IDE support as well.
If any motivation is unclear, please ask for additional PR description clarifications or code comments._
<hr>
Both `AdtDef` and `TraitDef` contained type information (field types, generics and predicates) which was required to create them, preventing their use before that type information exists, or in the case of field types, *mutation* was required, leading to a variance-magicking implementation of `ivar`s.
This PR takes that information out and the resulting cleaner setup could even eventually end up merged with HIR, because, just like `AssociatedItem` before it, there's no dependency on types anymore.
(With one exception, variant discriminants should probably be moved into their own map later.)
rustdoc: link to cross-crate sources directly.
Fixes#37684 by implementing proper support for getting the `Span` of definitions across crates.
In rustdoc this is used to generate direct links to the original source instead of fragile redirects.
This functionality could be expanded further for making error reporting code more uniform and seamless across crates, although at the moment there is no actual source to print, only file/line/column information.
Closes#37870 which is also "fixes" #37684 by throwing away the builtin macro docs from libcore.
After this lands, #37727 could be reverted, although it doesn't matter much either way.
Refactor TraitObject to Slice<ExistentialPredicate>
For reference, the primary types changes in this PR are shown below. They may add in the understanding of what is discussed below, though they should not be required.
We change `TraitObject` into a list of `ExistentialPredicate`s to allow for a couple of things:
- Principal (ExistentialPredicate::Trait) is now optional.
- Region bounds are moved out of `TraitObject` into `TyDynamic`. This permits wrapping only the `ExistentialPredicate` list in `Binder`.
- `BuiltinBounds` and `BuiltinBound` are removed entirely from the codebase, to permit future non-constrained auto traits. These are replaced with `ExistentialPredicate::AutoTrait`, which only requires a `DefId`. For the time being, only `Send` and `Sync` are supported; this constraint can be lifted in a future pull request.
- Binder-related logic is extracted from `ExistentialPredicate` into the parent (`Binder<Slice<EP>>`), so `PolyX`s are inside `TraitObject` are replaced with `X`.
The code requires a sorting order for `ExistentialPredicate`s in the interned `Slice`. The sort order is asserted to be correct during interning, but the slices are not sorted at that point.
1. `ExistentialPredicate::Trait` are defined as always equal; **This may be wrong; should we be comparing them and sorting them in some way?**
1. `ExistentialPredicate::Projection`: Compared by `ExistentialProjection::sort_key`.
1. `ExistentialPredicate::AutoTrait`: Compared by `TraitDef.def_path_hash`.
Construction of `ExistentialPredicate`s is conducted through `TyCtxt::mk_existential_predicates`, which interns a passed iterator as a `Slice`. There are no convenience functions to construct from a set of separate iterators; callers must pass an iterator chain. The lack of convenience functions is primarily due to few uses and the relative difficulty in defining a nice API due to optional parts and difficulty in recognizing which argument goes where. It is also true that the current situation isn't significantly better than 4 arguments to a constructor function; but the extra work is deemed unnecessary as of this time.
```rust
// before this PR
struct TraitObject<'tcx> {
pub principal: PolyExistentialTraitRef<'tcx>,
pub region_bound: &'tcx ty::Region,
pub builtin_bounds: BuiltinBounds,
pub projection_bounds: Vec<PolyExistentialProjection<'tcx>>,
}
// after
pub enum ExistentialPredicate<'tcx> {
// e.g. Iterator
Trait(ExistentialTraitRef<'tcx>),
// e.g. Iterator::Item = T
Projection(ExistentialProjection<'tcx>),
// e.g. Send
AutoTrait(DefId),
}
```
They don't implement FnLikeNode anymore, instead are handled differently
further up in the call tree. Also, keep less information (just def ids
for the args).
Avoid loading needless proc-macro dependencies
Fixes#37958 when no proc-macros are exported; in particular, without `pub extern crate proc_macros;`, `#![feature(macro_reexport)]`, or `#![feature(use_extern_macros)]`.
I opened https://github.com/rust-lang/cargo/issues/3334 for exported proc macros.
r? @alexcrichton
When cross compiling with procedural macros, the crate loader starts by
looking for a target crate, before trying with a host crate.
Rather than emitting an error immediately if the host and target
extension differ, the compiler should delay it until both attempts have
failed.
Fixes#37899
r? @jseyfried
rustc_metadata: don't break the version check when CrateRoot changes.
In #36551 I made `rustc_version` a field of `CrateRoot`, but despite it being the first field, one could still break the version check by changing `CrateRoot` so older compilers couldn't fully decode it (e.g. #37463).
This PR fixes#37803 by moving the version string back at the beginning of metadata, right after the 32-bit big-endian absolute position of `CrateRoot`, and by incrementing `METADATA_VERSION`.
Clean up `ast::Attribute`, `ast::CrateConfig`, and string interning
This PR
- removes `ast::Attribute_` (changing `Attribute` from `Spanned<Attribute_>` to a struct),
- moves a `MetaItem`'s name from the `MetaItemKind` variants to a field of `MetaItem`,
- avoids needlessly wrapping `ast::MetaItem` with `P`,
- moves string interning into `syntax::symbol` (`ast::Name` is a reexport of `symbol::Symbol` for now),
- replaces `InternedString` with `Symbol` in the AST, HIR, and various other places, and
- refactors `ast::CrateConfig` from a `Vec` to a `HashSet`.
r? @eddyb
Separate impl items from the parent impl
This change separates impl item bodies out of the impl itself. This gives incremental more resolution. In so doing, it refactors how the visitors work, and cleans up a bit of the collect/check logic (mostly by moving things out of collect that didn't really belong there, because they were just checking conditions).
However, this is not as effective as I expected, for a kind of frustrating reason. In particular, when invoking `foo.bar()` you still wind up with dependencies on private items. The problem is that the method resolution code scans that list for methods with the name `bar` -- and this winds up touching *all* the methods, even private ones.
I can imagine two obvious ways to fix this:
- separating fn bodies from fn sigs (#35078, currently being pursued by @flodiebold)
- a more aggressive model of incremental that @michaelwoerister has been advocating, in which we hash the intermediate results (e.g., the outputs of collect) so that we can see that the intermediate result hasn't changed, even if a particular impl item has changed.
So all in all I'm not quite sure whether to land this or not. =) It still seems like it has to be a win in some cases, but not with the test cases we have just now. I can try to gin up some test cases, but I'm not sure if they will be totally realistic. On the other hand, some of the early refactorings to the visitor trait seem worthwhile to me regardless.
cc #36349 -- well, this is basically a fix for that issue, I guess
r? @michaelwoerister
NB: Based atop of @eddyb's PR https://github.com/rust-lang/rust/pull/37402; don't land until that lands.
