Commit Graph

320 Commits

Author SHA1 Message Date
Josh Triplett
0fc5c524f5 Stabilize bool::then_some 2022-05-04 13:22:08 +02:00
Elliot Roberts
7907385999 fix most compiler/ doctests 2022-05-02 17:40:30 -07:00
Camille GILLOT
443333dc1f Remove NodeIdHashingMode. 2022-04-12 19:59:32 +02:00
bors
e980c62955 Auto merge of #95524 - oli-obk:cached_stable_hash_cleanups, r=nnethercote
Cached stable hash cleanups

r? `@nnethercote`

Add a sanity assertion in debug mode to check that the cached hashes are actually the ones we get if we compute the hash each time.

Add a new data structure that bundles all the hash-caching work to make it easier to re-use it for different interned data structures
2022-04-09 02:31:24 +00:00
David Wood
7f91697b50 errors: implement fallback diagnostic translation
This commit updates the signatures of all diagnostic functions to accept
types that can be converted into a `DiagnosticMessage`. This enables
existing diagnostic calls to continue to work as before and Fluent
identifiers to be provided. The `SessionDiagnostic` derive just
generates normal diagnostic calls, so these APIs had to be modified to
accept Fluent identifiers.

In addition, loading of the "fallback" Fluent bundle, which contains the
built-in English messages, has been implemented.

Each diagnostic now has "arguments" which correspond to variables in the
Fluent messages (necessary to render a Fluent message) but no API for
adding arguments has been added yet. Therefore, diagnostics (that do not
require interpolation) can be converted to use Fluent identifiers and
will be output as before.
2022-04-05 07:01:02 +01:00
Oli Scherer
00c24dd8ce Move stable hash from TyS into a datastructure that can be shared with other interned types. 2022-03-31 14:54:04 +00:00
Yuri Astrakhan
5160f8f843 Spellchecking compiler comments
This PR cleans up the rest of the spelling mistakes in the compiler comments. This PR does not change any literal or code spelling issues.
2022-03-30 15:14:15 -04:00
bors
f132bcf3bd Auto merge of #94081 - oli-obk:lazy_tait_take_two, r=nikomatsakis
Lazy type-alias-impl-trait take two

### user visible change 1: RPIT inference from recursive call sites

Lazy TAIT has an insta-stable change. The following snippet now compiles, because opaque types can now have their hidden type set from wherever the opaque type is mentioned.

```rust
fn bar(b: bool) -> impl std::fmt::Debug {
    if b {
        return 42
    }
    let x: u32 = bar(false); // this errors on stable
    99
}
```

The return type of `bar` stays opaque, you can't do `bar(false) + 42`, you need to actually mention the hidden type.

### user visible change 2: divergence between RPIT and TAIT in return statements

Note that `return` statements and the trailing return expression are special with RPIT (but not TAIT). So

```rust
#![feature(type_alias_impl_trait)]
type Foo = impl std::fmt::Debug;

fn foo(b: bool) -> Foo {
    if b {
        return vec![42];
    }
    std::iter::empty().collect() //~ ERROR `Foo` cannot be built from an iterator
}

fn bar(b: bool) -> impl std::fmt::Debug {
    if b {
        return vec![42]
    }
    std::iter::empty().collect() // Works, magic (accidentally stabilized, not intended)
}
```

But when we are working with the return value of a recursive call, the behavior of RPIT and TAIT is the same:

```rust
type Foo = impl std::fmt::Debug;

fn foo(b: bool) -> Foo {
    if b {
        return vec![];
    }
    let mut x = foo(false);
    x = std::iter::empty().collect(); //~ ERROR `Foo` cannot be built from an iterator
    vec![]
}

fn bar(b: bool) -> impl std::fmt::Debug {
    if b {
        return vec![];
    }
    let mut x = bar(false);
    x = std::iter::empty().collect(); //~ ERROR `impl Debug` cannot be built from an iterator
    vec![]
}
```

### user visible change 3: TAIT does not merge types across branches

In contrast to RPIT, TAIT does not merge types across branches, so the following does not compile.

```rust
type Foo = impl std::fmt::Debug;

fn foo(b: bool) -> Foo {
    if b {
        vec![42_i32]
    } else {
        std::iter::empty().collect()
        //~^ ERROR `Foo` cannot be built from an iterator over elements of type `_`
    }
}
```

It is easy to support, but we should make an explicit decision to include the additional complexity in the implementation (it's not much, see a721052457cf513487fb4266e3ade65c29b272d2 which needs to be reverted to enable this).

### PR formalities

previous attempt: #92007

This PR also includes #92306 and #93783, as they were reverted along with #92007 in #93893

fixes #93411
fixes #88236
fixes #89312
fixes #87340
fixes #86800
fixes #86719
fixes #84073
fixes #83919
fixes #82139
fixes #77987
fixes #74282
fixes #67830
fixes #62742
fixes #54895
2022-03-30 05:04:45 +00:00
Oli Scherer
264cd05b16 Revert "Auto merge of #93893 - oli-obk:sad_revert, r=oli-obk"
This reverts commit 6499c5e7fc, reversing
changes made to 78450d2d60.
2022-03-28 16:27:14 +00:00
klensy
008fc79dcd Propagate parallel_compiler feature through rustc crates. Turned off feature gives change of builded crates: 238 -> 224. 2022-03-28 08:41:12 +03:00
Camille GILLOT
056951d628 Take &mut Diagnostic in emit_diagnostic.
Taking a Diagnostic by move would break the usual pattern
`diag.label(..).emit()`.
2022-03-20 20:36:08 +01:00
mark
e489a94dee rename ErrorReported -> ErrorGuaranteed 2022-03-02 09:45:25 -06:00
bors
3b1fe7e7c9 Auto merge of #94084 - Mark-Simulacrum:drop-sharded, r=cjgillot
Avoid query cache sharding code in single-threaded mode

In non-parallel compilers, this is just adding needless overhead at compilation time (since there is only one shard statically anyway). This amounts to roughly ~10 seconds reduction in bootstrap time, with overall neutral (some wins, some losses) performance results.

Parallel compiler performance should be largely unaffected by this PR; sharding is kept there.
2022-02-27 14:04:07 +00:00
Mark Rousskov
22c3a71de1 Switch bootstrap cfgs 2022-02-25 08:00:52 -05:00
Eduard-Mihai Burtescu
b7e95dee65 rustc_errors: let DiagnosticBuilder::emit return a "guarantee of emission". 2022-02-23 06:38:52 +00:00
bors
58a721af9f Auto merge of #93839 - Mark-Simulacrum:delete-json-rust-deserialization, r=nnethercote
Simplify rustc_serialize by dropping support for decoding into JSON

This PR currently bundles two (somewhat separate) tasks.

First, it removes the JSON Decoder trait impl, which permitted going from JSON to Rust structs. For now, we keep supporting JSON deserialization, but only to `Json` (an equivalent of serde_json::Value). The primary hard to remove user there is for custom targets -- which need some form of JSON deserialization -- but they already have a custom ad-hoc pass for moving from Json to a Rust struct.

A [comment](e7aca89598/compiler/rustc_target/src/spec/mod.rs (L1653)) there suggests that it would be impractical to move them to a Decodable-based impl, at least without backwards compatibility concerns. I suspect that if we were widely breaking compat there, it would make sense to use serde_json at this point which would produce better error messages; the types in rustc_target are relatively isolated so we would not particularly suffer from using serde_derive.

The second part of the PR (all but the first commit) is to simplify the Decoder API by removing the non-primitive `read_*` functions. These primarily add indirection (through a closure), which doesn't directly cause a performance issue (the unique closure types essentially guarantee monomorphization), but does increase the amount of work rustc and LLVM need to do. This could be split out to a separate PR, but is included here in part to help motivate the first part.

Future work might consist of:

* Specializing enum discriminant encoding to avoid leb128 for small enums (since we know the variant count, we can directly use read/write u8 in almost all cases)
* Adding new methods to support faster deserialization (e.g., access to the underlying byte stream)
   * Currently these are somewhat ad-hoc supported by specializations for e.g. `Vec<u8>`, but other types which could benefit don't today.
* Removing the Decoder trait entirely in favor of a concrete type -- today, we only really have one impl of it modulo wrappers used for specialization-based dispatch.

Highly recommend review with whitespace changes off, as the removal of closures frequently causes things to be de-indented.
2022-02-22 07:54:22 +00:00
Mark Rousskov
42904b0219 Delete Decoder::read_seq 2022-02-20 18:58:23 -05:00
Mark Rousskov
24dc052132 Delete Decoder::read_seq_elt 2022-02-20 18:58:22 -05:00
Mark Rousskov
19288951e1 Delete Decoder::read_struct_field 2022-02-20 18:58:22 -05:00
Mark Rousskov
c021ba48a7 Delete Decoder::read_struct 2022-02-20 18:58:22 -05:00
Mark Rousskov
594ea74bf0 Refactor Sharded out of non-parallel active query map 2022-02-20 15:10:19 -05:00
Mark Rousskov
41f124c824 Avoid sharding query caches entirely in single-threaded mode 2022-02-20 14:57:34 -05:00
Mark Rousskov
8443816176 Inline QueryStateShard into QueryState 2022-02-20 12:11:28 -05:00
Mark Rousskov
75ef068920 Delete QueryLookup
This was largely just caching the shard value at this point, which is not
particularly useful -- in the use sites the key was being hashed nearby anyway.
2022-02-20 12:11:28 -05:00
Mark Rousskov
9deed6f74e Move Sharded maps into each QueryCache impl 2022-02-20 12:10:46 -05:00
Mark Rousskov
ddda851fd5 Remove SimpleDefKind 2022-02-17 18:08:45 -05:00
Oli Scherer
d54195db22 Revert "Auto merge of #92007 - oli-obk:lazy_tait2, r=nikomatsakis"
This reverts commit e7cc3bddbe, reversing
changes made to 734368a200.
2022-02-11 07:18:06 +00:00
bors
e7aca89598 Auto merge of #93741 - Mark-Simulacrum:global-job-id, r=cjgillot
Refactor query system to maintain a global job id counter

This replaces the per-shard counters with a single global counter, simplifying
the JobId struct down to just a u64 and removing the need to pipe a DepKind
generic through a bunch of code. The performance implications on non-parallel
compilers are likely minimal (this switches to `Cell<u64>` as the backing
storage over a `u64`, but the latter was already inside a `RefCell` so it's not
really a significance divergence). On parallel compilers, the cost of a single
global u64 counter may be more significant: it adds a serialization point in
theory. On the other hand, we can imagine changing the counter to have a
thread-local component if it becomes worrisome or some similar structure.

The new design is sufficiently simpler that it warrants the potential for slight
changes down the line if/when we get parallel compilation to be more of a
default.

A u64 counter, instead of u32 (the old per-shard width), is chosen to avoid
possibly overflowing it and causing problems; it is effectively impossible that
we would overflow a u64 counter in this context.
2022-02-09 18:54:30 +00:00
Mark Rousskov
e240783a4d Switch QueryJobId to a single global counter
This replaces the per-shard counters with a single global counter, simplifying
the JobId struct down to just a u64 and removing the need to pipe a DepKind
generic through a bunch of code. The performance implications on non-parallel
compilers are likely minimal (this switches to `Cell<u64>` as the backing
storage over a `u64`, but the latter was already inside a `RefCell` so it's not
really a significance divergence). On parallel compilers, the cost of a single
global u64 counter may be more significant: it adds a serialization point in
theory. On the other hand, we can imagine changing the counter to have a
thread-local component if it becomes worrisome or some similar structure.

The new design is sufficiently simpler that it warrants the potential for slight
changes down the line if/when we get parallel compilation to be more of a
default.

A u64 counter, instead of u32 (the old per-shard width), is chosen to avoid
possibly overflowing it and causing problems; it is effectively impossible that
we would overflow a u64 counter in this context.
2022-02-08 18:49:55 -05:00
bors
e7cc3bddbe Auto merge of #92007 - oli-obk:lazy_tait2, r=nikomatsakis
Lazy type-alias-impl-trait

Previously opaque types were processed by

1. replacing all mentions of them with inference variables
2. memorizing these inference variables in a side-table
3. at the end of typeck, resolve the inference variables in the side table and use the resolved type as the hidden type of the opaque type

This worked okayish for `impl Trait` in return position, but required lots of roundabout type inference hacks and processing.

This PR instead stops this process of replacing opaque types with inference variables, and just keeps the opaque types around.
Whenever an opaque type `O` is compared with another type `T`, we make the comparison succeed and record `T` as the hidden type. If `O` is compared to `U` while there is a recorded hidden type for it, we grab the recorded type (`T`) and compare that against `U`. This makes implementing

* https://github.com/rust-lang/rfcs/pull/2515

much simpler (previous attempts on the inference based scheme were very prone to ICEs and general misbehaviour that was not explainable except by random implementation defined oddities).

r? `@nikomatsakis`

fixes #93411
fixes #88236
2022-02-07 23:40:26 +00:00
Oli Scherer
0f6e06b7c0 Lazily resolve type-alias-impl-trait defining uses
by using an opaque type obligation to bubble up comparisons between opaque types and other types

Also uses proper obligation causes so that the body id works, because out of some reason nll uses body ids for logic instead of just diagnostics.
2022-02-02 15:40:11 +00:00
est31
08be313feb Use Option::then in two places 2022-02-02 16:10:16 +01:00
lcnr
a1a30f7548 add a rustc::query_stability lint 2022-02-01 10:15:59 +01:00
Nicholas Nethercote
37fbd91eb5 Address review comments. 2022-01-22 10:38:34 +11:00
Nicholas Nethercote
416399dc10 Make Decodable and Decoder infallible.
`Decoder` has two impls:
- opaque: this impl is already partly infallible, i.e. in some places it
  currently panics on failure (e.g. if the input is too short, or on a
  bad `Result` discriminant), and in some places it returns an error
  (e.g. on a bad `Option` discriminant). The number of places where
  either happens is surprisingly small, just because the binary
  representation has very little redundancy and a lot of input reading
  can occur even on malformed data.
- json: this impl is fully fallible, but it's only used (a) for the
  `.rlink` file production, and there's a `FIXME` comment suggesting it
  should change to a binary format, and (b) in a few tests in
  non-fundamental ways. Indeed #85993 is open to remove it entirely.

And the top-level places in the compiler that call into decoding just
abort on error anyway. So the fallibility is providing little value, and
getting rid of it leads to some non-trivial performance improvements.

Much of this commit is pretty boring and mechanical. Some notes about
a few interesting parts:
- The commit removes `Decoder::{Error,error}`.
- `InternIteratorElement::intern_with`: the impl for `T` now has the same
  optimization for small counts that the impl for `Result<T, E>` has,
  because it's now much hotter.
- Decodable impls for SmallVec, LinkedList, VecDeque now all use
  `collect`, which is nice; the one for `Vec` uses unsafe code, because
  that gave better perf on some benchmarks.
2022-01-22 10:38:31 +11:00
bors
42852d7857 Auto merge of #92740 - cuviper:update-rayons, r=Mark-Simulacrum
Update rayon and rustc-rayon

This updates rayon for various tools and rustc-rayon for the compiler's parallel mode.

- rayon v1.3.1 -> v1.5.1
- rayon-core v1.7.1 -> v1.9.1
- rustc-rayon v0.3.1 -> v0.3.2
- rustc-rayon-core v0.3.1 -> v0.3.2

... and indirectly, this updates all of crossbeam-* to their latest versions.

Fixes #92677 by removing crossbeam-queue, but there's still a lingering question about how tidy discovers "runtime" dependencies. None of this is truly in the standard library's dependency tree at all.
2022-01-16 08:12:23 +00:00
bors
02c9e73e6c Auto merge of #92681 - Aaron1011:task-deps-ref, r=cjgillot
Introduce new `TaskDepsRef` enum to track allow/ignore/forbid status
2022-01-14 14:20:17 +00:00
Josh Stone
f3b8812f24 Update rayon and rustc-rayon 2022-01-10 11:34:07 -08:00
bors
d63a8d965e Auto merge of #92278 - Aaron1011:fix-fingerprint-caching, r=michaelwoerister
Ensure that `Fingerprint` caching respects hashing configuration

Fixes #92266

In some `HashStable` impls, we use a cache to avoid re-computing
the same `Fingerprint` from the same structure (e.g. an `AdtDef`).
However, the `StableHashingContext` used can be configured to
perform hashing in different ways (e.g. skipping `Span`s). This
configuration information is not included in the cache key,
which will cause an incorrect `Fingerprint` to be used if
we hash the same structure with different `StableHashingContext`
settings.

To fix this, the configuration settings of `StableHashingContext`
are split out into a separate `HashingControls` struct. This
struct is used as part of the cache key, ensuring that our caches
always produce the correct result for the given settings.

With this in place, we now turn off `Span` hashing during the
entire process of computing the hash included in legacy symbols.
This current has no effect, but will matter when a future PR
starts hashing more `Span`s that we currently skip.
2022-01-10 00:26:07 +00:00
Aaron Hill
f64cd87ca6
Introduce new TaskDepsRef enum to track allow/ignore/forbid status 2022-01-08 18:22:06 -05:00
bors
a7e2e33960 Auto merge of #91919 - Aaron1011:query-recursive-read, r=michaelwoerister
Don't perform any new queries while reading a query result on disk

In addition to being very confusing, this can cause us to add dep node edges between two queries that would not otherwise have an edge.

We now panic if any new dep node edges are created during the deserialization of a query result. This requires serializing the full `AdtDef` to disk, instead of just serializing the `DefId` and invoking the `adt_def` query during deserialization.

I'll probably split this up into several smaller PRs for perf runs.
2022-01-08 18:32:31 +00:00
Aaron Hill
4ca275add0
Address review comments 2022-01-05 10:30:49 -05:00
Aaron Hill
560c90f5df
Adjust assert_default_hashing_controls 2022-01-05 10:13:29 -05:00
Aaron Hill
5580e5e1dd
Ensure that Fingerprint caching respects hashing configuration
Fixes #92266

In some `HashStable` impls, we use a cache to avoid re-computing
the same `Fingerprint` from the same structure (e.g. an `AdtDef`).
However, the `StableHashingContext` used can be configured to
perform hashing in different ways (e.g. skipping `Span`s). This
configuration information is not included in the cache key,
which will cause an incorrect `Fingerprint` to be used if
we hash the same structure with different `StableHashingContext`
settings.

To fix this, the configuration settings of `StableHashingContext`
are split out into a separate `HashingControls` struct. This
struct is used as part of the cache key, ensuring that our caches
always produce the correct result for the given settings.

With this in place, we now turn off `Span` hashing during the
entire process of computing the hash included in legacy symbols.
This current has no effect, but will matter when a future PR
starts hashing more `Span`s that we currently skip.
2022-01-05 10:13:28 -05:00
bors
2b681ac06b Auto merge of #92259 - Aaron1011:normal-mod-hashing, r=michaelwoerister
Remove special-cased stable hashing for HIR module

All other 'containers' (e.g. `impl` blocks) hashed their contents
in the normal, order-dependent way. However, `Mod` was hashing
its contents in a (sort-of) order-independent way. However, the
exact order is exposed to consumers through `Mod.item_ids`,
and through query results like `hir_module_items`. Therefore,
stable hashing needs to take the order of items into account,
to avoid fingerprint ICEs.

Unforuntately, I was unable to directly build a reproducer
for the ICE, due to the behavior of `Fingerprint::combine_commutative`.
This operation swaps the upper and lower `u64` when constructing the
result, which makes the function non-associative. Since we start
the hashing of module items by combining `Fingerprint::ZERO` with
the first item, it's difficult to actually build an example where
changing the order of module items leaves the final hash unchanged.

However, this appears to have been hit in practice in #92218
While we're not able to reproduce it, the fact that proc-macros
are involved (which can give an entire module the same span, preventing
any span-related invalidations) makes me confident that the root
cause of that issue is our method of hashing module items.

This PR removes all of the special handling for `Mod`, instead deriving
a `HashStable` implementation. This makes `Mod` consistent with other
'contains' like `Impl`, which hash their contents through the typical
derive of `HashStable`.
2022-01-04 00:25:23 +00:00
Aaron Hill
da3f196a4e
Remove special-cased stable hashing for HIR module
All other 'containers' (e.g. `impl` blocks) hashed their contents
in the normal, order-dependent way. However, `Mod` was hashing
its contents in a (sort-of) order-independent way. However, the
exact order is exposed to consumers through `Mod.item_ids`,
and through query results like `hir_module_items`. Therefore,
stable hashing needs to take the order of items into account,
to avoid fingerprint ICEs.

Unforuntately, I was unable to directly build a reproducer
for the ICE, due to the behavior of `Fingerprint::combine_commutative`.
This operation swaps the upper and lower `u64` when constructing the
result, which makes the function non-associative. Since we start
the hashing of module items by combining `Fingerprint::ZERO` with
the first item, it's difficult to actually build an example where
changing the order of module items leaves the final hash unchanged.

However, this appears to have been hit in practice in #92218
While we're not able to reproduce it, the fact that proc-macros
are involved (which can give an entire module the same span, preventing
any span-related invalidations) makes me confident that the root
cause of that issue is our method of hashing module items.

This PR removes all of the special handling for `Mod`, instead deriving
a `HashStable` implementation. This makes `Mod` consistent with other
'contains' like `Impl`, which hash their contents through the typical
derive of `HashStable`.
2021-12-24 12:38:29 -05:00
Aaron Hill
27ed52c0a2
Adjust wording of comment 2021-12-23 13:44:04 -05:00
Aaron Hill
28f19f62c7
Address review comments 2021-12-23 13:38:54 -05:00
Aaron Hill
ab168e69ac
Some cleanup 2021-12-23 13:38:53 -05:00
Aaron Hill
49560e9c49
Ban deps only during query loading from disk 2021-12-23 13:38:53 -05:00
Aaron Hill
75181dc22f
Error if we try to read dep during deserialization 2021-12-23 13:38:53 -05:00
Aaron Hill
f1d682334d
Add #[rustc_clean(loaded_from_disk)] to assert loading of query result
Currently, you can use `#[rustc_clean]` to assert to that a particular
query (technically, a `DepNode`) is green or red. However, a green
`DepNode` does not mean that the query result was actually deserialized
from disk - we might have never re-run a query that needed the result.

Some incremental tests are written as regression tests for ICEs that
occured during query result decoding. Using
`#[rustc_clean(loaded_from_disk="typeck")]`, you can now assert
that the result of a particular query (e.g. `typeck`) was actually
loaded from disk, in addition to being green.
2021-12-21 16:34:12 -05:00
PFPoitras
304ede6bcc Stabilize iter::zip. 2021-12-14 18:50:31 -04:00
Alex Crichton
a0c959750a std: Stabilize the thread_local_const_init feature
This commit is intended to follow the stabilization disposition of the
FCP that has now finished in #84223. This stabilizes the ability to flag
thread local initializers as `const` expressions which enables the macro
to generate more efficient code for accessing it, notably removing
runtime checks for initialization.

More information can also be found in #84223 as well as the tests where
the feature usage was removed in this PR.

Closes #84223
2021-11-29 07:23:46 -08:00
Mark Rousskov
dc65b22901 Manually outline error on incremental_verify_ich
This reduces codegen for rustc_query_impl by 169k lines of LLVM IR, representing
a 1.2% improvement.
2021-11-22 21:32:20 -05:00
bors
495322d776 Auto merge of #90361 - Mark-Simulacrum:always-verify, r=michaelwoerister
Enable verification for 1/32th of queries loaded from disk

This is a limited enabling of incremental verification for query results loaded from disk, which previously did not run without -Zincremental-verify-ich. If enabled for all queries, we see a probably unacceptable hit of ~50% in the worst case, so this pairs back the verification to a more limited set based on the hash key.

Per collected [perf results](https://github.com/rust-lang/rust/pull/84227#issuecomment-953350582), this is a regression of at most 7% on coercions opt incr-unchanged, and typically less than 0.5% on other benchmarks (largely limited to incr-unchanged). I believe this is acceptable performance to land, and we can either ratchet it up or down fairly easily.

We have no real sense of whether this will lead to a large amount of assertions in the wild, but since those assertions may lead to miscompilations today, it seems potentially warranted. We have a good bit of lead time until the next stable release, though the holiday season will also start soon; we may wish to discuss the timing of enabling this and weigh the desire to prevent (possible) miscompilations against assertions.

cc `@rust-lang/wg-incr-comp`
2021-11-08 13:38:08 +00:00
bors
88a5a984fe Auto merge of #90380 - Mark-Simulacrum:revert-89558-query-stable-lint, r=lcnr
Revert "Add rustc lint, warning when iterating over hashmaps"

Fixes perf regressions introduced in https://github.com/rust-lang/rust/pull/90235 by temporarily reverting the relevant PR.
2021-10-29 04:55:51 +00:00
Mark Rousskov
3215eeb99f
Revert "Add rustc lint, warning when iterating over hashmaps" 2021-10-28 11:01:42 -04:00
Mark Rousskov
49e7c993ee Enable verification for 1/32th of queries loaded from disk 2021-10-28 09:57:31 -04:00
bors
c4ff03f689 Auto merge of #90145 - cjgillot:sorted-map, r=michaelwoerister
Use SortedMap in HIR.

Closes https://github.com/rust-lang/rust/issues/89788
r? `@ghost`
2021-10-28 13:04:40 +00:00
bors
28d0e75269 Auto merge of #90210 - cjgillot:qarray2, r=Mark-Simulacrum
Build the query vtable directly.

Continuation of https://github.com/rust-lang/rust/pull/89978.

This shrinks the query interface and attempts to reduce the amount of function pointer calls.
2021-10-25 01:10:50 +00:00
Matthias Krüger
87822b27ee
Rollup merge of #89558 - lcnr:query-stable-lint, r=estebank
Add rustc lint, warning when iterating over hashmaps

r? rust-lang/wg-incr-comp
2021-10-24 15:48:42 +02:00
Camille GILLOT
138e96b719 Do not require QueryCtxt for cache_on_disk. 2021-10-23 18:12:43 +02:00
Camille GILLOT
7c0920f5fb Build the query vtable directly. 2021-10-23 16:59:19 +02:00
Camille GILLOT
6f6fa8b954 Use SortedMap in HIR. 2021-10-21 23:08:57 +02:00
Camille GILLOT
0a5666b838 Do not depend on the stored value when trying to cache on disk. 2021-10-21 20:00:45 +02:00
Camille GILLOT
b11ec29e28 Address review. 2021-10-20 18:51:15 +02:00
Camille GILLOT
8785b70774 Inline DepNodeParams methods. 2021-10-20 18:46:25 +02:00
Camille GILLOT
df71d0874a Compute query vtable manually. 2021-10-20 18:41:28 +02:00
Camille GILLOT
69a3594635 Store node_intern_event_id in CurrentDepGraph. 2021-10-20 18:37:11 +02:00
Camille GILLOT
bd5c107672 Build jump table at runtime. 2021-10-20 18:32:29 +02:00
Camille GILLOT
602d3cbce3 Invoke callbacks from rustc_middle. 2021-10-20 18:29:33 +02:00
Camille GILLOT
b09de95fab Merge two query callbacks arrays. 2021-10-20 18:29:27 +02:00
Camille GILLOT
dc7143367c Drop has_params. 2021-10-20 18:29:22 +02:00
Camille GILLOT
aa404c24dd Make hash_result an Option. 2021-10-20 18:29:18 +02:00
Yuki Okushi
3d95330230
Rollup merge of #87404 - rylev:artifact-size-profiling, r=wesleywiser
Add support for artifact size profiling

This adds support for profiling artifact file sizes (incremental compilation artifacts and query cache to begin with).

Eventually we want to track this in perf.rlo so we can ensure that file sizes do not change dramatically on each pull request.

This relies on support in measureme: https://github.com/rust-lang/measureme/pull/169. Once that lands we can update this PR to not point to a git dependency.

This was worked on together with `@michaelwoerister.`

r? `@wesleywiser`
2021-10-20 04:35:11 +09:00
bors
1af55d19c7 Auto merge of #89933 - est31:let_else, r=michaelwoerister
Adopt let_else across the compiler

This performs a substitution of code following the pattern:

```
let <id> = if let <pat> = ... { identity } else { ... : ! };
```

To simplify it to:

```
let <pat> = ... { identity } else { ... : ! };
```

By adopting the `let_else` feature (cc #87335).

The PR also updates the syn crate because the currently used version of the crate doesn't support `let_else` syntax yet.

Note: Generally I'm the person who *removes* usages of unstable features from the compiler, not adds more usages of them, but in this instance I think it hopefully helps the feature get stabilized sooner and in a better state. I have written a [comment](https://github.com/rust-lang/rust/issues/87335#issuecomment-944846205) on the tracking issue about my experience and what I feel could be improved before stabilization of `let_else`.
2021-10-19 14:41:39 +00:00
bors
bd41e09da3 Auto merge of #89124 - cjgillot:owner-info, r=michaelwoerister
Index and hash HIR as part of lowering

Part of https://github.com/rust-lang/rust/pull/88186
~Based on https://github.com/rust-lang/rust/pull/88880 (see merge commit).~

Once HIR is lowered, it is later indexed by the `index_hir` query and hashed for `crate_hash`. This PR moves those post-processing steps to lowering itself. As a side objective, the HIR crate data structure is refactored as an `IndexVec<LocalDefId, Option<OwnerInfo<'hir>>>` where `OwnerInfo` stores all the relevant information for an HIR owner.

r? `@michaelwoerister`
cc `@petrochenkov`
2021-10-18 19:53:05 +00:00
est31
1418df5888 Adopt let_else across the compiler
This performs a substitution of code following the pattern:

let <id> = if let <pat> = ... { identity } else { ... : ! };

To simplify it to:

let <pat> = ... { identity } else { ... : ! };

By adopting the let_else feature.
2021-10-16 07:18:05 +02:00
lcnr
00e5abe9b6 allow potential_query_instability everywhere 2021-10-15 10:58:18 +02:00
Mark Rousskov
127373822e Remove built-in cache_hit tracking
This was already only enabled in debug_assertions builds. Generally, it seems
like most use cases that would use this could also use the -Zself-profile flag
which also tracks cache hits (in all builds), and so the extra cfg's and such
are not really necessary.

This is largely just a small cleanup though, which primarily is intended to make
other changes easier by avoiding the need to deal with this field.
2021-10-11 16:33:49 -04:00
Camille GILLOT
0431fdb113 Compute full HIR hash during lowering. 2021-10-10 00:05:49 +02:00
Camille GILLOT
457de08487 Forbid hashing HIR outside of indexing. 2021-10-09 18:38:28 +02:00
Ryan Levick
947a33bf20 Add support for artifact size profiling 2021-10-07 14:22:29 +02:00
Mark Rousskov
6f78eed1c7 Query the fingerprint style during key reconstruction
Keys can be reconstructed from fingerprints that are not DefPathHash, but then
we cannot extract a DefId from them.
2021-10-06 22:19:48 -04:00
Camille GILLOT
b2ed9c4007 Add some inlining. 2021-10-03 16:08:57 +02:00
Camille GILLOT
fedd7785fe Access StableHashingContext in rustc_query_system. 2021-10-03 16:08:55 +02:00
Camille GILLOT
c355b2e5cd Move ICH to rustc_query_system. 2021-10-03 16:08:53 +02:00
Mark Rousskov
c746be2219 Migrate to 2021 2021-09-20 22:21:42 -04:00
bors
d6cd2c6c87 Auto merge of #82183 - michaelwoerister:lazier-defpathhash-loading2, r=wesleywiser
Simplify lazy DefPathHash decoding by using an on-disk hash table.

This PR simplifies the logic around mapping `DefPathHash` values encountered during incremental compilation to valid `DefId`s in the current session. It is able to do so by using an on-disk hash table encoding that allows for looking up values directly, i.e. without deserializing the entire table.

The main simplification comes from not having to keep track of `DefPathHashes` being used during the compilation session.
2021-09-18 14:37:39 +00:00
Tomasz Miąsko
b6b19f3b6c Use explicit log level in tracing instrument macro
Specify a log level in tracing instrument macro explicitly.

Additionally reduce the used log level from a default info level to a
debug level (all of those appear to be developer oriented logs, so there
should be no need to include them in release builds).
2021-09-15 19:02:10 +02:00
Michael Woerister
5445715c20 Remove RawDefId tracking infrastructure from incr. comp. framework.
This infrastructure is obsolete now with the new encoding scheme for
the DefPathHash->DefIndex maps in crate metadata.
2021-09-14 13:56:33 +02:00
bors
8c2b6ea37d Auto merge of #78780 - cjgillot:req, r=Mark-Simulacrum
Refactor query forcing

The control flow in those functions was very complex, with several layers of continuations.

I tried to simplify the implementation, while keeping essentially the same logic.
Now, all code paths go through `try_execute_query` for the actual query execution.
Communication with the `dep_graph` and the live caches are the only difference between query getting/ensuring/forcing.
2021-09-11 20:39:47 +00:00
Noah Lev
4553a4baf2 Remove redundant Span in QueryJobInfo
Previously, `QueryJobInfo` was composed of two parts: a `QueryInfo` and
a `QueryJob`. However, both `QueryInfo` and `QueryJob` have a `span`
field, which seem to be the same. So, the `span` was recorded twice.

Now, `QueryJobInfo` is composed of a `QueryStackFrame` (the other field
of `QueryInfo`) and a `QueryJob`. So, now, the `span` is only recorded
once.
2021-09-01 11:10:58 -07:00
Noah Lev
c861964735 Note that trait aliases cannot be recursive 2021-08-27 14:50:52 -07:00
Noah Lev
cd0fc444fb Note that type aliases cannot be recursive 2021-08-27 14:50:51 -07:00
Camille GILLOT
31330bfce1 Use variable. 2021-08-22 20:23:32 +02:00
Camille GILLOT
eeb3c8f4b7 Unify with_task functions.
Remove with_eval_always_task.
2021-08-22 20:23:32 +02:00
Camille GILLOT
f2c8707abb Remove force_query_with_job. 2021-08-22 20:23:31 +02:00
Camille GILLOT
ef4becdce4 Split try_execute_query. 2021-08-22 20:23:31 +02:00
Camille GILLOT
307aacaf05 Decouple JobOwner from cache. 2021-08-22 20:23:31 +02:00
Camille GILLOT
d2304008c1 Complete job outside of force_query_with_job. 2021-08-22 20:23:30 +02:00
Camille GILLOT
13d4eb92b8 Do not compute the dep_node twice. 2021-08-22 20:23:30 +02:00
Camille GILLOT
283a8e1445 Make all query forcing go through try_execute_query.
try_execute_query is now able to centralize the path for query
get/ensure/force.

try_execute_query now takes the dep_node as a parameter, so it can
accommodate `force`. This dep_node is an Option to avoid computing it in
the `get` fast path.

try_execute_query now returns both the result and the dep_node_index to
allow the caller to handle the dep graph.

The caller is responsible for marking the dependency.
2021-08-22 20:23:29 +02:00
Camille GILLOT
45d6decc19 Remove try_mark_green_and_read. 2021-08-22 20:23:29 +02:00
Camille GILLOT
c3bf3969d4 Move assertion inwards.
`with_taks_impl` is only called from `with_eval_always_task` and
`with_task` . The former is only used in query invocation, while the
latter is also used to start the `tcx` and to trigger codegen.

This move should not change significantly the number of calls to this
assertion.
2021-08-22 20:23:29 +02:00
Camille GILLOT
cd1cb3449e Simplify control flow. 2021-08-22 20:23:20 +02:00
Noah Lev
2f48bfa88c Improve errors for recursive type aliases 2021-08-21 18:30:25 -07:00
Camille GILLOT
0edc775b90 Only clone key when needed. 2021-08-22 01:06:19 +02:00
Camille GILLOT
5e35fadddb Move dep_graph checking into try_load_from_disk_and_cache_in_memory. 2021-08-22 01:00:01 +02:00
Aaron Hill
77b02eed7b
Prevent double panic when handling incremental fingerprint mismatch
When an incremental fingerprint mismatch occurs, we debug-print
our `DepNode` and query result. Unfortunately, the debug printing
process may cause us to run additional queries, which can result
in a re-entrant fingerprint mismatch error.

To avoid a double panic, this commit adds a thread-local variable
to detect re-entrant calls.
2021-08-12 15:11:39 -05:00
bors
b53a93db2d Auto merge of #87535 - lf-:authors, r=Mark-Simulacrum
rfc3052 followup: Remove authors field from Cargo manifests

Since RFC 3052 soft deprecated the authors field, hiding it from
crates.io, docs.rs, and making Cargo not add it by default, and it is
not generally up to date/useful information for contributors, we may as well
remove it from crates in this repo.
2021-08-02 05:49:17 +00:00
bors
aadd6189ad Auto merge of #87449 - matthiaskrgr:clippyy_v2, r=nagisa
more clippy::complexity fixes

(also a couple of clippy::perf fixes)
2021-08-01 09:15:15 +00:00
Jade
3cf820e17d rfc3052: Remove authors field from Cargo manifests
Since RFC 3052 soft deprecated the authors field anyway, hiding it from
crates.io, docs.rs, and making Cargo not add it by default, and it is
not generally up to date/useful information, we should remove it from
crates in this repo.
2021-07-29 14:56:05 -07:00
Aaron Hill
87740bac64
Restrict field visibility 2021-07-25 20:43:27 -05:00
Aaron Hill
e6a5231238
Create QuerySideEffects and use it for diagnostics 2021-07-25 20:27:58 -05:00
Matthias Krüger
3fd8cbb404 clippy::useless_format 2021-07-25 12:26:03 +02:00
Ryan Levick
b5bec17184 Add docs to new methods 2021-07-07 11:14:14 +02:00
Ryan Levick
6e33dce9c2 Profile incremental hashing 2021-07-07 10:43:30 +02:00
bors
12d0849f9d Auto merge of #85154 - cjgillot:lessfn, r=bjorn3
Reduce amount of function pointers in query invocation.

r? `@ghost`
2021-06-15 14:52:58 +00:00
bors
e4a6032706 Auto merge of #85903 - bjorn3:rustc_serialize_cleanup, r=varkor
Remove unused functions and arguments from rustc_serialize
2021-06-07 14:40:26 +00:00
Yuki Okushi
36f1ed6de2
Rollup merge of #85850 - bjorn3:less_feature_gates, r=jyn514
Remove unused feature gates

The first commit removes a usage of a feature gate, but I don't expect it to be controversial as the feature gate was only used to workaround a limitation of rust in the past. (closures never being `Clone`)

The second commit uses `#[allow_internal_unstable]` to avoid leaking the `trusted_step` feature gate usage from inside the index newtype macro. It didn't work for the `min_specialization` feature gate though.

The third commit removes (almost) all feature gates from the compiler that weren't used anyway.
2021-06-04 13:42:54 +09:00
Camille GILLOT
b51f24f021 Make the reasoning more explicit. 2021-06-01 21:46:30 +02:00
Camille GILLOT
3a6d5c2beb Avoid creating anonymous nodes with zero or one dependency. 2021-06-01 21:43:30 +02:00
bjorn3
a2c4affe86 Remove unused functions and arguments from rustc_serialize 2021-06-01 19:29:11 +02:00
bjorn3
312f964478 Remove unused feature gates 2021-05-31 13:55:43 +02:00
Camille GILLOT
fd318a2f9b Reduce amount of function pointers. 2021-05-30 15:15:22 +02:00
bors
f60a670256 Auto merge of #85319 - cjgillot:query-simp, r=Mark-Simulacrum
Simplification of query forcing

Extracted from #78780
2021-05-30 10:11:23 +00:00
bors
9a72afa7dd Auto merge of #83772 - jhpratt:revamp-step-trait, r=Mark-Simulacrum
Make `Step` trait safe to implement

This PR makes a few modifications to the `Step` trait that I believe better position it for stabilization in the short term. In particular,

1. `unsafe trait TrustedStep` is introduced, indicating that the implementation of `Step` for a given type upholds all stated invariants (which have remained unchanged). This is gated behind a new `trusted_step` feature, as stabilization is realistically blocked on min_specialization.
2. The `Step` trait is internally specialized on the `TrustedStep` trait, which avoids a serious performance regression.
3. `TrustedLen` is implemented for `T: TrustedStep` as the latter's invariants subsume the former's.
4. The `Step` trait is no longer `unsafe`, as the invariants must not be relied upon by unsafe code (unless the type implements `TrustedStep`).
5. `TrustedStep` is implemented for all types that implement `Step` in the standard library and compiler.
6. The `step_trait_ext` feature is merged into the `step_trait` feature. I was unable to find any reasoning for the features being split; the `_unchecked` methods need not necessarily be stabilized at the same time, but I think it is useful to have them under the same feature flag.

All existing implementations of `Step` will be broken, as it is not possible to `unsafe impl` a safe trait. Given this trait only exists on nightly, I feel this breakage is acceptable. The blanket `impl<T: Step> TrustedLen for T` will likely cause some minor breakage, but this should be covered by the equivalent impl for `TrustedStep`.

Hopefully these changes are sufficient to place `Step` in decent position for stabilization, which would allow user-defined types to be used with `a..b` syntax.
2021-05-30 01:21:39 +00:00
Camille GILLOT
f3ed997254 Move reconstruct test inwards. 2021-05-29 22:38:51 +02:00
Jacob Pratt
bc2f0fb5a9
Specialize implementations
Implementations in stdlib are now optimized as they were before.
2021-05-26 18:07:09 -04:00
Camille GILLOT
a50f1e949b Get rid of PreviousDepGraph. 2021-05-22 14:14:23 +02:00
Camille GILLOT
c95a5682f7 Remove def_path_str. 2021-05-15 10:37:30 +02:00
Camille GILLOT
eb82187b13 Make the fast path faster. 2021-05-15 10:36:37 +02:00
Camille GILLOT
91444af87a Refactor try_mark_previous_green. 2021-05-15 10:27:27 +02:00
Camille GILLOT
c2c59ae304 Move key recovering into force_query. 2021-05-15 10:20:56 +02:00
Aaron Hill
a4c0793551 Show nicer error when an 'unstable fingerprints' error occurs 2021-05-10 17:43:51 -04:00
bors
777bb2f612 Auto merge of #84806 - Mark-Simulacrum:try-start-entry, r=cjgillot
Streamline try_start code

This shifts some branches around and avoids interleaving parallel and
non-parallel versions of the function too much.
2021-05-06 22:35:06 +00:00
Mark Rousskov
981135ae8e Streamline try_start code
This shifts some branches around and avoids interleaving parallel and
non-parallel versions of the function too much.
2021-05-02 12:25:48 -04:00
Mark Rousskov
61fd56fdb9 Avoid generating QueryMap::extend for each key type 2021-05-01 20:13:18 -04:00
Mark Rousskov
a1d7367429 Move iter_results to dyn FnMut rather than a generic
This means that we're no longer generating the iteration/locking code for each
invocation site of iter_results, rather just once per query.

This is a 15% win in instruction counts when compiling the rustc_query_impl crate.
2021-04-29 17:26:46 -04:00
Ralf Jung
bd9556956a fix feature use in rustc libs 2021-04-18 22:05:45 +02:00
pierwill
0019ca9141 Fix outdated crate names in compiler docs
Changes `librustc_X` to `rustc_X`, only in documentation comments.
Plain code comments are left unchanged.

Also fix incorrect file paths.
2021-04-08 11:12:14 -05:00
Camille GILLOT
f3dde45d2a Enable debugging the dep-graph without debug-assertions.
It may also be useful in these cases,
and some CI configurations test without debug assertions.
2021-03-31 17:12:06 +02:00
Camille GILLOT
8ee9322c10 Also profile finishing the encoding. 2021-03-30 18:10:08 +02:00
Camille GILLOT
df24315ddf Adjust profiling. 2021-03-30 18:10:08 +02:00
Camille GILLOT
fe89f3236c Address review. 2021-03-30 18:10:08 +02:00
Camille GILLOT
65a8681a17 Add documentation. 2021-03-30 18:10:07 +02:00
Camille GILLOT
c5c935af92 Simplify tracking the encoder state. 2021-03-30 18:10:07 +02:00
Camille GILLOT
e1c99e5fcc Remove the parallel version. 2021-03-30 18:10:06 +02:00