rust/compiler/rustc_metadata
bors d64c84562f Auto merge of #113542 - saethlin:adaptive-tables, r=b-naber
Adapt table sizes to the contents

This is an implementation of https://github.com/rust-lang/compiler-team/issues/666

The objective of this PR is to permit the rmeta format to accommodate larger crates that need offsets larger than a `u32` can store without compromising performance for crates that do not need such range. The second commit is a number of tiny optimization opportunities I noticed while looking at perf recordings of the first commit.

The rmeta tables need to have fixed-size elements to permit lazy random access. But the size only needs to be fixed _per table_, not per element type. This PR adds another `usize` to the table header which indicates the table element size. As each element of a table is set, we keep track of the widest encoded table value, then don't bother encoding all the unused trailing bytes on each value. When decoding table elements, we copy them to a full-width array if they are not already full-width.

`LazyArray` needs some special treatment. Most other values that are encoded in tables are indexes or offsets, and those tend to be small so we get to drop a lot of zero bytes off the end. But `LazyArray` encodes _two_ small values in a fixed-width table element: A position of the table and the length of the table. The treatment described above could trim zero bytes off the table length, but any nonzero length shields the position bytes from the optimization. To improve this, we interleave the bytes of position and length. This change is responsible for about half of the crate metadata win on many crates.

Fixes https://github.com/rust-lang/rust/issues/112934 (probably)
Fixes https://github.com/rust-lang/rust/issues/103607
2023-08-30 02:40:37 +00:00
..
src Auto merge of #113542 - saethlin:adaptive-tables, r=b-naber 2023-08-30 02:40:37 +00:00
Cargo.toml
messages.ftl Suggest x build library for a custom toolchain that fails to load core 2023-07-04 09:37:40 +08:00