rust/compiler
bors f00c139998 Auto merge of #110050 - saethlin:better-u32-encoding, r=nnethercote
Use a specialized varint + bitpacking scheme for DepGraph encoding

The previous scheme here uses leb128 to encode the edge tables that represent the incr comp dependency graph. The problem with that scheme is that leb128 has overhead for larger values, and generally relies on the distribution of encoded values being heavily skewed towards smaller values. That is definitely not the case for a dep node index, since they are handed out sequentially and the whole range is covered, the distribution is actually biased in the opposite direction: Most dep nodes are large.

This PR implements a different varint encoding scheme. Instead of applying varint encoding to individual dep node indices (which is extremely branchy) we now apply it per node.

While being built, each node now stores its edges in a `SmallVec` with a bit of extra logic to track the max value of each edge. Then we varint encode the whole batch. This is a gamble: We save on space by only claiming 2 bits per node instead of ~3 bits per edge which is a nice savings but needs to balance out with the space overhead that a single large index in a node with a lot of edges will encode unnecessary bytes in each of that node's edge indices.

Then, to keep the runtime overhead of this encoding scheme down we deserialize our indices by loading 4 bytes for each then masking off the bytes that are't ours. This is much less code and branches than leb128, but relies on having some readable bytes past the end of each edge list. We explicitly add such padding to the in-memory data during decoding. And we also do this decoding lazily, turning a dense on-disk encoding into a peak memory reduction.

Then we apply a bit-packing scheme; since in https://github.com/rust-lang/rust/pull/115391 we now have unused bits on `DepKind`, we use those unused bits (currently there are 7!) to store the 2 bits that we need for the byte width of the edges in each node, then use the remaining bits to store the length of the edge list, if it fits.

r? `@nnethercote`
2023-09-07 02:09:41 +00:00
..
rustc
rustc_abi there seems to be no reason to treat ZST specially in these cases 2023-08-29 08:58:58 +02:00
rustc_arena Bump cfg(bootstrap) 2023-08-23 20:05:14 -04:00
rustc_ast improve AttrTokenStream 2023-09-04 20:07:28 +08:00
rustc_ast_lowering Don't hold the definitions' lock across index_hir 2023-09-02 08:13:07 +02:00
rustc_ast_passes Auto merge of #115131 - frank-king:feature/unnamed-fields-lite, r=petrochenkov 2023-08-24 12:52:35 +00:00
rustc_ast_pretty Auto merge of #115131 - frank-king:feature/unnamed-fields-lite, r=petrochenkov 2023-08-24 12:52:35 +00:00
rustc_attr Auto merge of #114201 - Centri3:explicit-repr-rust, r=WaffleLapkin 2023-08-25 00:02:54 +00:00
rustc_baked_icu_data
rustc_borrowck Rollup merge of #114511 - chenyukang:yukang-fix-114374-fmt-args, r=b-naber 2023-09-06 19:31:48 +02:00
rustc_builtin_macros Load include_bytes! directly into an Lrc 2023-08-27 20:16:19 -04:00
rustc_codegen_cranelift Auto merge of #115580 - eduardosm:stdarch-intrinsics, r=davidtwco,bjorn3 2023-09-06 22:26:37 +00:00
rustc_codegen_gcc Rollup merge of #111580 - atsuzaki:layout-ice, r=oli-obk 2023-08-29 20:49:02 +02:00
rustc_codegen_llvm Use relative positions inside a SourceFile. 2023-09-03 12:56:10 +00:00
rustc_codegen_ssa Refactor how MIR represents composite debuginfo. 2023-09-05 17:20:07 +00:00
rustc_const_eval Refactor how MIR represents composite debuginfo. 2023-09-05 17:20:07 +00:00
rustc_data_structures Use a reference to the lock in the guards 2023-09-06 11:44:06 +02:00
rustc_driver
rustc_driver_impl Rollup merge of #113565 - workingjubilee:better-signal-handler-message, r=pnkfelix 2023-08-30 07:18:10 +02:00
rustc_error_codes Auto merge of #113126 - Bryanskiy:delete_old, r=petrochenkov 2023-09-01 12:40:01 +00:00
rustc_error_messages Bump cfg(bootstrap) 2023-08-23 20:05:14 -04:00
rustc_errors Add OnceHelp lint level (same as OnceNote, except for help) 2023-09-04 13:54:20 +02:00
rustc_expand Bump cfg(bootstrap) 2023-08-23 20:05:14 -04:00
rustc_feature fix version for abi_thiscall to 1.73.0, which was forgotten to change when stabilized 2023-09-01 19:46:09 +03:00
rustc_fluent_macro
rustc_fs_util
rustc_graphviz
rustc_hir replace doc occurrences of ItemLikeVisitor 2023-09-05 22:40:36 +02:00
rustc_hir_analysis Auto merge of #115401 - Zoxc:freeze, r=oli-obk 2023-09-06 11:48:43 +00:00
rustc_hir_pretty
rustc_hir_typeck Remove allow_private entirely. 2023-08-29 16:36:13 -07:00
rustc_incremental Ignore unexpected incr-comp session dirs 2023-08-18 19:57:41 +02:00
rustc_index Auto merge of #115094 - Mark-Simulacrum:bootstrap-update, r=ozkanonur 2023-08-24 11:10:52 +00:00
rustc_infer Test and note unsafe ctor to fn ptr coercion 2023-08-30 15:09:40 -07:00
rustc_interface Rename Freeze to FreezeLock 2023-09-02 08:14:06 +02:00
rustc_lexer
rustc_lint Auto merge of #115166 - Urgau:invalid_ref_casting-invalid-unsafecell-usage, r=est31 2023-09-07 00:24:45 +00:00
rustc_lint_defs Rollup merge of #115578 - ouz-a:rustc_clarify, r=oli-obk 2023-09-06 19:31:49 +02:00
rustc_llvm Auto merge of #114946 - anforowicz:generic-fix-for-asan-lto, r=tmiasko 2023-09-06 20:04:03 +00:00
rustc_log
rustc_macros Bump cfg(bootstrap) 2023-08-23 20:05:14 -04:00
rustc_metadata Register the file while computing its start position. 2023-09-03 15:02:46 +00:00
rustc_middle Auto merge of #110050 - saethlin:better-u32-encoding, r=nnethercote 2023-09-07 02:09:41 +00:00
rustc_mir_build Refactor how MIR represents composite debuginfo. 2023-09-05 17:20:07 +00:00
rustc_mir_dataflow Take &mut Results in ResultsVisitor 2023-09-02 19:35:51 -04:00
rustc_mir_transform Auto merge of #115615 - matthiaskrgr:rollup-49fosdf, r=matthiaskrgr 2023-09-06 18:16:06 +00:00
rustc_monomorphize Rollup merge of #115492 - Enselic:large-box-move, r=oli-obk 2023-09-05 07:15:16 +02:00
rustc_parse Rollup merge of #115596 - nnethercote:two-small-changes, r=lqd 2023-09-06 19:31:50 +02:00
rustc_parse_format Replace data_structures dependency with index in rustc_parse_format 2023-09-05 19:11:50 +02:00
rustc_passes add rustc_abi debugging attribute 2023-08-27 11:55:49 +02:00
rustc_plugin_impl
rustc_privacy Auto merge of #113126 - Bryanskiy:delete_old, r=petrochenkov 2023-09-01 12:40:01 +00:00
rustc_query_impl remove some unused crate deps 2023-09-01 19:13:09 +03:00
rustc_query_system Auto merge of #110050 - saethlin:better-u32-encoding, r=nnethercote 2023-09-07 02:09:41 +00:00
rustc_resolve Auto merge of #115429 - compiler-errors:assoc-ct-lt-fallthrough, r=cjgillot 2023-09-02 10:55:49 +00:00
rustc_serialize Use a specialized varint + bitpacking scheme for DepGraph encoding 2023-09-04 12:16:50 -04:00
rustc_session Rename Freeze to FreezeLock 2023-09-02 08:14:06 +02:00
rustc_smir Rollup merge of #115605 - ouz-a:smir_better_debug, r=oli-obk 2023-09-06 19:31:51 +02:00
rustc_span Auto merge of #115166 - Urgau:invalid_ref_casting-invalid-unsafecell-usage, r=est31 2023-09-07 00:24:45 +00:00
rustc_symbol_mangling Fix CFI: f32 and f64 are encoded incorrectly for c 2023-08-24 21:02:06 -07:00
rustc_target Auto merge of #114114 - keith:ks/always-add-lc_build_version-for-metadata-object-files, r=wesleywiser 2023-08-29 21:17:13 +00:00
rustc_trait_selection Auto merge of #115529 - chenyukang:yukang-fix-115402-overflowsize, r=compiler-errors 2023-09-06 02:37:41 +00:00
rustc_traits remove some unused crate deps 2023-09-01 19:13:09 +03:00
rustc_transmute Fix error report for size overflow from transmute 2023-09-06 06:48:34 +08:00
rustc_ty_utils some more is_zst that should be is_1zst 2023-08-29 14:11:27 +02:00
rustc_type_ir Bump cfg(bootstrap) 2023-08-23 20:05:14 -04:00