mikros/rust
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Rollup merge of #122245 - saethlin:check-dep-graph-size, r=petrochenkov

Browse Source 
Detect truncated DepGraph files

I suspect that the following issues are caused by truncated incr comp files:

* https://github.com/rust-lang/rust/issues/120582
* https://github.com/rust-lang/rust/issues/121499
* https://github.com/rust-lang/rust/issues/122210

We fail with an allocation failure or capacity overflow in this case because we assume that the ending bytes of an DepGraph file are the lengths of arrays. If the file has somehow been truncated then the ending bytes are probably some of our varint encoding, which tries to eliminate zero bytes, so interpreting a random 8 bytes as an array length has a very high chance of producing a byte capacity over `isize::MAX`.

Now theoretically since https://github.com/rust-lang/rust/pull/119510 merged I have fixed the out-of-disk issues and yet in https://github.com/rust-lang/rust/issues/120894#issuecomment-1945126700 I still see some decoding failures that look like out-of-disk ICEs, for example https://crater-reports.s3.amazonaws.com/beta-1.77-1/beta-2024-02-10/gh/scottfones.aoc_2022/log.txt

So this PR should ensure that we get an ICE that clearly identifies if the file in question is truncated.
This commit is contained in:
Jubilee 2024-03-12 09:04:00 -07:00 committed by GitHub
commit f54350a9ba
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 7 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
compiler/rustc_query_system/src/dep_graph/serialized.rs
View File

@ -179,13 +179,15 @@ impl SerializedDepGraph {
pub fn decode<D: Deps>(d: &mut MemDecoder<'_>) -> SerializedDepGraph {
// The last 16 bytes are the node count and edge count.
debug!("position: {:?}", d.position());
let (node_count, edge_count) =
d.with_position(d.len() - 2 * IntEncodedWithFixedSize::ENCODED_SIZE, |d| {
let (node_count, edge_count, graph_size) =
d.with_position(d.len() - 3 * IntEncodedWithFixedSize::ENCODED_SIZE, |d| {
debug!("position: {:?}", d.position());
let node_count = IntEncodedWithFixedSize::decode(d).0 as usize;
let edge_count = IntEncodedWithFixedSize::decode(d).0 as usize;
(node_count, edge_count)
let graph_size = IntEncodedWithFixedSize::decode(d).0 as usize;
(node_count, edge_count, graph_size)
});
assert_eq!(d.len(), graph_size);
debug!("position: {:?}", d.position());
debug!(?node_count, ?edge_count);
@ -491,6 +493,8 @@ fn finish(self, profiler: &SelfProfilerRef) -> FileEncodeResult {
debug!("position: {:?}", encoder.position());
IntEncodedWithFixedSize(node_count).encode(&mut encoder);
IntEncodedWithFixedSize(edge_count).encode(&mut encoder);
let graph_size = encoder.position() + IntEncodedWithFixedSize::ENCODED_SIZE;
IntEncodedWithFixedSize(graph_size as u64).encode(&mut encoder);
debug!("position: {:?}", encoder.position());
// Drop the encoder so that nothing is written after the counts.
let result = encoder.finish();