Before this PR, type names could depend on the cratenum being used
for a given crate and also on the source location of closures.
Both are undesirable for incremental compilation where we cache
LLVM IR and don't want it to depend on formatting or in which
order crates are loaded.
Replace FNV with a faster hash function.
Hash table lookups are very hot in rustc profiles and the time taken within `FnvHash` itself is a big part of that. Although FNV is a simple hash, it processes its input one byte at a time. In contrast, Firefox has a homespun hash function that is also simple but works on multiple bytes at a time. So I tried it out and the results are compelling:
```
futures-rs-test 4.326s vs 4.212s --> 1.027x faster (variance: 1.001x, 1.007x)
helloworld 0.233s vs 0.232s --> 1.004x faster (variance: 1.037x, 1.016x)
html5ever-2016- 5.397s vs 5.210s --> 1.036x faster (variance: 1.009x, 1.006x)
hyper.0.5.0 5.018s vs 4.905s --> 1.023x faster (variance: 1.007x, 1.006x)
inflate-0.1.0 4.889s vs 4.872s --> 1.004x faster (variance: 1.012x, 1.007x)
issue-32062-equ 0.347s vs 0.335s --> 1.035x faster (variance: 1.033x, 1.019x)
issue-32278-big 1.717s vs 1.622s --> 1.059x faster (variance: 1.027x, 1.028x)
jld-day15-parse 1.537s vs 1.459s --> 1.054x faster (variance: 1.005x, 1.003x)
piston-image-0. 11.863s vs 11.482s --> 1.033x faster (variance: 1.060x, 1.002x)
regex.0.1.30 2.517s vs 2.453s --> 1.026x faster (variance: 1.011x, 1.013x)
rust-encoding-0 2.080s vs 2.047s --> 1.016x faster (variance: 1.005x, 1.005x)
syntex-0.42.2 32.268s vs 31.275s --> 1.032x faster (variance: 1.014x, 1.022x)
syntex-0.42.2-i 17.629s vs 16.559s --> 1.065x faster (variance: 1.013x, 1.021x)
```
(That's a stage1 compiler doing debug builds. Results for a stage2 compiler are similar.)
The attached commit is not in a state suitable for landing because I changed the implementation of FnvHasher without changing its name (because that would have required touching many lines in the compiler). Nonetheless, it is a good place to start discussions.
Profiles show very clearly that this new hash function is a lot faster to compute than FNV. The quality of the new hash function is less clear -- it seems to do better in some cases and worse in others (judging by the number of instructions executed in `Hash{Map,Set}::get`).
CC @brson, @arthurprs
Stabilize `..` in tuple (struct) patterns
I'd like to nominate `..` in tuple and tuple struct patterns for stabilization.
This feature is a relatively small extension to existing stable functionality and doesn't have known blockers.
The feature first appeared in Rust 1.10 6 months ago.
An example of use: https://github.com/rust-lang/rust/pull/36203
Closes https://github.com/rust-lang/rust/issues/33627
r? @nikomatsakis
Reduce the number of bytes hashed by IchHasher.
IchHasher uses blake2b hashing, which is expensive, so the fewer bytes hashed
the better. There are two big ways to reduce the number of bytes hashed.
- Filenames in spans account for ~66% of all bytes (for builds with debuginfo).
The vast majority of spans have the same filename for the start of the span
and the end of the span, so hashing the filename just once in those cases is
a big win.
- u32 and u64 and usize values account for ~25%--33% of all bytes (for builds
with debuginfo). The vast majority of these are small, i.e. fit in a u8, so
shrinking them down before hashing is also a big win.
This PR implements these two optimizations. I'm certain the first one is safe.
I'm about 90% sure that the second one is safe.
Here are measurements of the number of bytes hashed when doing
debuginfo-enabled builds of stdlib and
rustc-benchmarks/syntex-0.42.2-incr-clean.
```
stdlib syntex-incr
------ -----------
original 156,781,386 255,095,596
half-SawSpan 106,744,403 176,345,419
short-ints 45,890,534 118,014,227
no-SawSpan[*] 6,831,874 45,875,714
[*] don't hash the SawSpan at all. Not part of this PR, just implemented for
comparison's sake.
```
For debug builds of syntex-0.42.2-incr-clean, the two changes give a 1--2%
speed-up.
ICH: Hash expression spans if their source location is captured for panics.
Since the location of some expressions is captured in error message constants, it has an influence on machine code and consequently we need to take them into account by the incr. comp. hash. This PR makes this happen for `+, -, *, /, %` and for array indexing -- let me know if I forgot anything.
In the future we might want to change the codegen strategy for those error messages, so that they are stored in a separate object file with a stable symbol name, so that only this object file has to be regenerated when source locations change. This strategy would also eliminate unnecessary duplications due to monomorphization, as @arielb1 has pointed out on IRC. I opened https://github.com/rust-lang/rust/issues/37512, so we don't forget about this.
r? @nikomatsakis
std: Stabilize and deprecate APIs for 1.13
This commit is intended to be backported to the 1.13 branch, and works with the
following APIs:
Stabilized
* `i32::checked_abs`
* `i32::wrapping_abs`
* `i32::overflowing_abs`
* `RefCell::try_borrow`
* `RefCell::try_borrow_mut`
Deprecated
* `BinaryHeap::push_pop`
* `BinaryHeap::replace`
* `SipHash13`
* `SipHash24`
* `SipHasher` - use `DefaultHasher` instead in the `std::collections::hash_map`
module
Closes#28147Closes#34767Closes#35057Closes#35070
This commit is intended to be backported to the 1.13 branch, and works with the
following APIs:
Stabilized
* `i32::checked_abs`
* `i32::wrapping_abs`
* `i32::overflowing_abs`
* `RefCell::try_borrow`
* `RefCell::try_borrow_mut`
* `DefaultHasher`
* `DefaultHasher::new`
* `DefaultHasher::default`
Deprecated
* `BinaryHeap::push_pop`
* `BinaryHeap::replace`
* `SipHash13`
* `SipHash24`
* `SipHasher` - use `DefaultHasher` instead in the `std::collections::hash_map`
module
Closes#28147Closes#34767Closes#35057Closes#35070
fix a few errant `Krate` edges
Exploring the effect of small changes on `syntex` reuse, I discovered the following sources of unnecessary edges from `Krate`
r? @michaelwoerister
incr. comp.: Take spans into account for ICH
This PR makes the ICH (incr. comp. hash) take spans into account when debuginfo is enabled.
A side-effect of this is that the SVH (which is based on the ICHs of all items in the crate) becomes sensitive to the tiniest change in a code base if debuginfo is enabled. Since we are not trying to model ABI compatibility via the SVH anymore (this is done via the crate disambiguator now), this should be not be a problem.
Fixes#33888.
Fixes#32753.
Implement synchronization scheme for incr. comp. directory
This PR implements a copy-on-write-based synchronization scheme for the incremental compilation cache directory. For technical details, see the documentation at the beginning of `rustc_incremental/persist/fs.rs`.
The PR contains unit tests for some functions but for testing whether the scheme properly handles races, a more elaborate test setup would be needed. It would probably involve a small tool that allows to manipulate the incremental compilation directory in a controlled way and then letting a compiler instance run against directories in different states. I don't know if it's worth the trouble of adding another test category to `compiletest`, but I'd be happy to do so.
Fixes#32754Fixes#34957
Experimentally, this fixes the poor re-use observed in
libsyntex-syntax. I'm not sure how to make a regression test for this,
though, given the non-deterministic nature of it.
This avoids the compile-time overhead of computing them twice. It also fixes
an issue where the hash computed after typeck is differen than the hash before,
because typeck mutates the def-map in place.
Fixes#35549.
Fixes#35593.
This massively speeds up serialization. It also
seems to produce deterministic metadata hashes
(before I was seeing inconsistent results).
Fixes#35232.
When we hash the inputs to a MetaData node, we have to hash them in a
consistent order. We achieve this by sorting the stringfied `DefPath`
entries. Also, micro-optimie by cache more results across the saving
process.
