This commit moves the actual code generation in the compiler behind a query
keyed by a codegen unit's name. This ended up entailing quite a few internal
refactorings to enable this, along with a few cut corners:
* The `OutputFilenames` structure is now tracked in the `TyCtxt` as it affects a
whole bunch of trans and such. This is now behind a query and threaded into
the construction of the `TyCtxt`.
* The `TyCtxt` now has a channel "out the back" intended to send data to worker
threads in rustc_trans. This is used as a sort of side effect of the codegen
query but morally what's happening here is the return value of the query
(currently unit but morally a path) is only valid once the background threads
have all finished.
* Dispatching work items to the codegen threads was refactored to only rely on
data in `TyCtxt`, which mostly just involved refactoring where data was
stored, moving it from the translation thread to the controller thread's
`CodegenContext` or the like.
* A new thread locals was introduced in trans to work around the query
system. This is used in the implementation of `assert_module_sources` which
looks like an artifact of the old query system and will presumably go away
once red/green is up and running.
This commit attaches a channel to the LLVM workers to the `TyCtxt` which will
later be used during the codegen query to actually send work to LLVM workers.
Otherwise this commit is just plumbing this channel throughout the compiler to
ensure it reaches the right consumers.
This is a big map that ends up inside of a `CrateContext` during translation for
all codegen units. This means that any change to the map may end up causing an
incremental recompilation of a codegen unit! In order to reduce the amount of
dependencies here between codegen units and the actual input crate this commit
refactors dealing with exported symbols and such into various queries.
The new queries are largely based on existing queries with filled out
implementations for the local crate in addition to external crates, but the main
idea is that while translating codegen untis no unit needs the entire set of
exported symbols, instead they only need queries about particulare `DefId`
instances every now and then.
The linking stage, however, still generates a full list of all exported symbols
from all crates, but that's going to always happen unconditionally anyway, so no
news there!
Otherwise we may emit double errors related to the `#[export_name]` attribute,
for example, and using a query should ensure that it's only emitted at most
once.
This commit moves the `collect_and_partition_translation_items` function into a
query on `TyCtxt` instead of a free function in trans, allowing us to track
dependencies and such of the function.
This commit moves the definition of the `ExportedSymbols` structure to the
`rustc` crate and then creates a query that'll be used to construct the
`ExportedSymbols` set. This in turn uses the reachablity query exposed in the
previous commit.
rustc: Spawn `cmd /c emcc.bat` explicitly
In #42436 the behavior for spawning processes on Windows was tweaked slightly to
fix various bugs, but this caused #42791 as a regression, namely that to spawn
batch scripts they need to be manually spawned with `cmd /c` instead now. This
updates the compiler to handle this case explicitly for Emscripten.
Closes#42791
This commit removes the `dep_graph` field from the `Session` type according to
issue #44390. Most of the fallout here was relatively straightforward and the
`prepare_session_directory` function was rejiggered a bit to reuse the results
in the later-called `load_dep_graph` function.
Closes#44390
In #42436 the behavior for spawning processes on Windows was tweaked slightly to
fix various bugs, but this caused #42791 as a regression, namely that to spawn
batch scripts they need to be manually spawned with `cmd /c` instead now. This
updates the compiler to handle this case explicitly for Emscripten.
Closes#42791
Migrate a slew of metadata methods to queries
This PR intends to make more progress on #41417, knocking off some low-hanging fruit.
Closes#44190
cc #44137
This commit moves the `crates` method to a query and then migrates all callers
to use a query instead of the now-renamed `crates_untracked` method where
possible.
Closes#41417
This commit adds logic to the compiler to attempt to handle super long linker
invocations by falling back to the `@`-file syntax if the invoked command is too
large. Each OS has a limit on how many arguments and how large the arguments can
be when spawning a new process, and linkers tend to be one of those programs
that can hit the limit!
The logic implemented here is to unconditionally attempt to spawn a linker and
then if it fails to spawn with an error from the OS that indicates the command
line is too big we attempt a fallback. The fallback is roughly the same for all
linkers where an argument pointing to a file, prepended with `@`, is passed.
This file then contains all the various arguments that we want to pass to the
linker.
Closes#41190
Previously a `Symbol` was stored there, but this ended up causing hash
collisions in situations that otherwise shouldn't have a hash collision. Only
the symbol's string value was hashed, but it was possible for distinct symbols
to have the same string value, fooling various calcuations into thinking that
these paths *didn't* need disambiguating data when in fact they did!
By storing `InternedString` instead we're hopefully triggering all the exising
logic to disambiguate paths with same-name `Symbol` but actually distinct
locations.
The main use of `CrateStore` *before* the `TyCtxt` is created is during
resolution, but we want to be sure that any methods used before resolution are
not used after the `TyCtxt` is created. This commit starts moving the methods
used by resolve to all be named `{name}_untracked` where the rest of the
compiler uses just `{name}` as a query.
During this transition a number of new queries were added to account for
post-resolve usage of these methods.
This commit started by moving methods from `CrateStore` to queries, but it ended
up necessitating some deeper refactorings to move more items in general to
queries.
Before this commit the *resolver* would walk over the AST and process foreign
modules (`extern { .. }` blocks) and collect `#[link]` annotations. It would
then also process the command line `-l` directives and such. This information
was then stored as precalculated lists in the `CrateStore` object for iterating
over later.
After this, commit, however, this pass no longer happens during resolution but
now instead happens through queries. A query for the linked libraries of a crate
will crawl the crate for `extern` blocks and then process the linkage
annotations at that time.
This comit applies the following changes:
* Deletes the `is_allocator` query as it's no longer used
* Moves the `is_sanitizer_runtime` method to a query
* Moves the `is_profiler_runtime` method to a query
* Moves the `panic_strategy` method to a query
* Moves the `is_no_builtins` method to a query
* Deletes the cstore method of `is_compiler_builtins`. The query was added in
#42588 but the `CrateStore` method was not deleted
A good bit of these methods were used late in linking during trans so a new
dedicated structure was created to ship a calculated form of this information
over to the linker rather than having to ship the whole of `TyCtxt` over to
linking.
Compact display of static lib dependencies
Fixes#33173
Instead of displaying one dependency per line, I've changed the format to display them all in one line.
As a bonus they're in format of linker flags (`-lfoo`), so the output can be copy&pasted if one is actually going to link as suggested.
Static PIE support, while supported on musl, requires a patch to GCC.
Until/unless it is merged, adding '-pie' to the linker command line will
override '-static' and create a binary that requires a dynamic
interpreter (ld.so).
Improve LLVM/trans scheduling a bit
Currently it's possible that the main thread is waiting on LLVM threads to finish work while its implicit token is going to waste. This PR let's the main thread take over, so one of the running LLVM threads can free its token earlier.
r? @alexcrichton
Thread through the original error when opening archives
This updates the management of opening archives to thread through the original
piece of error information from LLVM over to the end consumer, trans.
