Commit Graph

35 Commits

Author SHA1 Message Date
bors
a8f7075532 Auto merge of - Aaron1011:feature/query-result-debug, r=estebank
Enforce that query results implement Debug

Currently, we require that query keys implement `Debug`, but we do not do the same for query values. This can make incremental compilation bugs difficult to debug - there isn't a good place to print out the result loaded from disk.

This PR adds `Debug` bounds to several query-related functions, allowing us to debug-print the query value when an 'unstable fingerprint' error occurs. This required adding `#[derive(Debug)]` to a fairly large number of types - hopefully, this doesn't have much of an impact on compiler bootstrapping times.
2021-01-26 05:47:23 +00:00
bors
c5a96fb797 Auto merge of - tgnottingham:direct_serialize_depgraph, r=michaelwoerister
Serialize dependency graph directly from DepGraph

Reduce memory usage by serializing dep graph directly from `DepGraph`,
rather than copying it into `SerializedDepGraph` and serializing that.
2021-01-19 19:36:41 +00:00
Aaron Hill
056fbbf7ee
Undo assertion change 2021-01-16 21:37:53 -05:00
Aaron Hill
6417760632
Run fmt 2021-01-16 17:53:02 -05:00
Aaron Hill
93ab705655
Print result on unstable fingerprint error 2021-01-16 17:53:02 -05:00
Aaron Hill
7afb32557d
Enforce that query results implement Debug 2021-01-16 17:53:02 -05:00
LingMan
a56bffb4f9 Use Option::map_or instead of .map(..).unwrap_or(..) 2021-01-14 19:23:59 +01:00
Tyson Nottingham
09067db8a0 Serialize dependency graph directly from DepGraph
Reduce memory usage by serializing dep graph directly from `DepGraph`,
rather than copying it into `SerializedDepGraph` and serializing that.
2021-01-12 22:20:29 -08:00
Joshua Nelson
0215b3a456 Don't mark force_query_with_job as inline(always)
It's rather large, and using `inline(always)` forces it to be recompiled
in each calling crate.
2021-01-08 18:38:33 -05:00
Camille GILLOT
016ea6b319 Use a side-table of consts instead of matching on the DepKind enum. 2021-01-08 17:48:02 +01:00
Camille GILLOT
d1220fdedf Simplify DepNodeParams. 2021-01-08 17:29:49 +01:00
Tyson Nottingham
03eb75f759 rustc_query_system: avoid race condition when using edge_count 2020-12-22 14:12:57 -08:00
Tyson Nottingham
22ed75158b rustc_query_system: add more comments for dependency graph 2020-12-22 14:12:57 -08:00
Tyson Nottingham
d6b2aaed7d rustc_query_system: rename intern_node to intern_new_node 2020-12-22 14:12:57 -08:00
Tyson Nottingham
712fcae13a rustc_query_system: remove inline annotation from edge_count
This isn't called frequently enough to justify inlining.
2020-12-22 14:12:57 -08:00
Tyson Nottingham
4f76266295 rustc_query_system: minor cleanup
Remove effectively unused parameter and delete out of date comment.
2020-12-22 14:12:57 -08:00
Tyson Nottingham
dd1ab840d2 rustc_query_system: use more space-efficient edges representation
Use single vector of edges rather than per-node vector. There is a small
hit to instruction counts (< 0.5%), but the memory savings make up for it.
2020-12-22 14:12:57 -08:00
Tyson Nottingham
ea47269f5f rustc_query_system: share previous graph edges with current graph
Reduce memory consumption by sharing the previous dependency graph's
edges with the current graph when it is known to be valid to do so. It
is known to be valid whenever we mark a node green because all of its
dependencies were green. It is *not* known to be valid when we mark a
node green because we re-executed its query and its result was the same
as in the previous compilation session. In that case, the dependency set
might have changed (we don't try to determine whether or not it changed
and whether or not we can share).
2020-12-22 14:12:57 -08:00
Tyson Nottingham
f6d6b0c96d rustc_query_system: share previous graph data with current graph
Reduce memory consumption by taking advantage of red/green algorithm
properties to share the previous dependency graph's node data with the
current graph instead of storing node data redundantly. Red nodes can
share the `DepNode`, and green nodes can share the `DepNode` and
`Fingerprint`. Edges will be shared when possible in a later change.
2020-12-22 14:12:57 -08:00
Tyson Nottingham
7795801902 rustc_query_system: explicitly register reused dep nodes
Register nodes that we've reused from the previous session explicitly
with `OnDiskCache`. Previously, we relied on this happening as a side
effect of accessing the nodes in the `PreviousDepGraph`. For the sake of
performance and avoiding unintended side effects, register explictily.
2020-12-18 18:53:12 -08:00
Aaron Hill
3918b82993
Use def_path_hash_to_def_id when re-using a RawDefId
Fixes 

Previously, we just copied a `RawDefId` from the 'old' map to the 'new'
map. However, the `RawDefId` for a given `DefPathHash` may be different
in the current compilation session. Using `def_path_hash_to_def_id`
ensures that the `RawDefId` we use is valid in the current session.
2020-12-10 16:04:19 -05:00
Aaron Hill
c2946402ff
Properly re-use def path hash in incremental mode
Fixes 

In incremental compilation mode, we update a `DefPathHash -> DefId`
mapping every time we create a `DepNode` for a foreign `DefId`.
This mapping is written out to the on-disk incremental cache, and is
read by the next compilation session to allow us to lazily decode
`DefId`s.

When we decode a `DepNode` from the current incremental cache, we need
to ensure that any previously-recorded `DefPathHash -> DefId` mapping
gets recorded in the new mapping that we write out. However, PR 
didn't do this in all cases, leading to us being unable to decode a
`DefPathHash` in certain circumstances.

This PR refactors some of the code around `DepNode` deserialization to
prevent this kind of mistake from happening again.
2020-12-04 22:16:40 -05:00
Aaron Hill
7a9aa4f980
Fix rebase fallout 2020-11-25 15:08:51 -05:00
Aaron Hill
e935d3832c
Lazy DefPath decoding for incremental compilation 2020-11-25 14:49:15 -05:00
Dániel Buga
db8b86b2df Fix typos 2020-11-21 09:06:45 +01:00
Tyson Nottingham
05dde137ca Make PackedFingerprint's Fingerprint private 2020-11-18 15:10:43 -08:00
Tyson Nottingham
f09d474836 Use PackedFingerprint in DepNode to reduce memory consumption 2020-11-18 12:49:09 -08:00
Joshua Nelson
57c6ed0c07 Fix even more clippy warnings 2020-10-30 10:13:39 -04:00
Camille GILLOT
0a4d948b4a Remove unused ProfileCategory. 2020-10-22 22:35:32 +02:00
bors
500ddc5efd Auto merge of - Julian-Wollersberger:less-query-context, r=oli-obk
Make fewer types generic over QueryContext

While trying to refactor `rustc_query_system::query::QueryContext` to make it dyn-safe, I noticed some smaller things:
* QueryConfig doesn't need to be generic over QueryContext
* ~~The `kind` field on QueryJobId is unused~~
* Some unnecessary where clauses
* Many types in `job.rs` where generic over `QueryContext` but only needed `QueryContext::Query`.
  If handle_cycle_error() could be refactored to not take `error: CycleError<CTX::Query>`, all those bounds could be removed as well.

Changing `find_cycle_in_stack()` in job.rs to not take a `tcx` argument is the only functional change here. Everything else is just updating type signatures. (aka compile-error driven development ^^)

~~Currently there is a weird bug where memory usage suddenly skyrockets when running UI tests. I'll investigate that tomorrow.
A perf run probably won't make sense before that is fixed.~~

EDIT: `kind` actually is used by `Eq`, and re-adding it fixed the memory issue.
2020-10-22 12:24:55 +00:00
Julian Wollersberger
52cedcab92 Remove <CTX: QueryContext> in a bunch of places.
It was only needed by `find_cycle_in_stack()` in job.rs, but needed to be forwarded through dozens of types.
2020-10-19 11:11:09 +02:00
est31
338fad2162 Remove unused code from rustc_query_system 2020-10-14 04:14:32 +02:00
Julian Wollersberger
39b0e79285 Remove generic argument from QueryConfig. 2020-10-12 16:04:49 +02:00
Andreas Jonson
b8752fff19 update the version of itertools and parking_lot
this is to avoid compiling multiple version of the crates in rustc
2020-09-12 08:26:53 +02:00
mark
9e5f7d5631 mv compiler to compiler/ 2020-08-30 18:45:07 +03:00