mikros/rust
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
rust/compiler/rustc_serialize/src/leb128.rs

147 lines
4.1 KiB
Rust
Raw Normal View History

Serialize incr comp structures to file via fixed-size buffer Reduce a large memory spike that happens during serialization by writing the incr comp structures to file by way of a fixed-size buffer, rather than an unbounded vector. Effort was made to keep the instruction count close to that of the previous implementation. However, buffered writing to a file inherently has more overhead than writing to a vector, because each write may result in a handleable error. To reduce this overhead, arrangements are made so that each LEB128-encoded integer can be written to the buffer with only one capacity and error check. Higher-level optimizations in which entire composite structures can be written with one capacity and error check are possible, but would require much more work. The performance is mostly on par with the previous implementation, with small to moderate instruction count regressions. The memory reduction is significant, however, so it seems like a worth-while trade-off.
2020-12-06 17:30:55 -08:00
#![macro_use]
macro_rules! max_leb128_len {
($int_ty:ty) => {
// The longest LEB128 encoding for an integer uses 7 bits per byte.
(std::mem::size_of::<$int_ty>() * 8 + 6) / 7
};
}
// Returns the longest LEB128 encoding of all supported integer types.
pub const fn max_leb128_len() -> usize {
max_leb128_len!(u128)
}
Make leb128 coding a bit faster.
2018-01-09 16:53:35 +01:00
macro_rules! impl_write_unsigned_leb128 {
Serialize incr comp structures to file via fixed-size buffer Reduce a large memory spike that happens during serialization by writing the incr comp structures to file by way of a fixed-size buffer, rather than an unbounded vector. Effort was made to keep the instruction count close to that of the previous implementation. However, buffered writing to a file inherently has more overhead than writing to a vector, because each write may result in a handleable error. To reduce this overhead, arrangements are made so that each LEB128-encoded integer can be written to the buffer with only one capacity and error check. Higher-level optimizations in which entire composite structures can be written with one capacity and error check are possible, but would require much more work. The performance is mostly on par with the previous implementation, with small to moderate instruction count regressions. The memory reduction is significant, however, so it seems like a worth-while trade-off.
2020-12-06 17:30:55 -08:00
($fn_name:ident, $int_ty:ty) => {
Make leb128 coding a bit faster.
2018-01-09 16:53:35 +01:00
#[inline]
Serialize incr comp structures to file via fixed-size buffer Reduce a large memory spike that happens during serialization by writing the incr comp structures to file by way of a fixed-size buffer, rather than an unbounded vector. Effort was made to keep the instruction count close to that of the previous implementation. However, buffered writing to a file inherently has more overhead than writing to a vector, because each write may result in a handleable error. To reduce this overhead, arrangements are made so that each LEB128-encoded integer can be written to the buffer with only one capacity and error check. Higher-level optimizations in which entire composite structures can be written with one capacity and error check are possible, but would require much more work. The performance is mostly on par with the previous implementation, with small to moderate instruction count regressions. The memory reduction is significant, however, so it seems like a worth-while trade-off.
2020-12-06 17:30:55 -08:00
pub fn $fn_name(
out: &mut [::std::mem::MaybeUninit<u8>; max_leb128_len!($int_ty)],
mut value: $int_ty,
) -> &[u8] {
let mut i = 0;
Micro-optimize the heck out of LEB128 reading and writing. This commit makes the following writing improvements: - Removes the unnecessary `write_to_vec` function. - Reduces the number of conditions per loop from 2 to 1. - Avoids a mask and a shift on the final byte. And the following reading improvements: - Removes an unnecessary type annotation. - Fixes a dangerous unchecked slice access. Imagine a slice `[0x80]` -- the current code will read past the end of the slice some number of bytes. The bounds check at the end will subsequently trigger, unless something bad (like a crash) happens first. The cost of doing bounds check in the loop body is negligible. - Avoids a mask on the final byte. And the following improvements for both reading and writing: - Changes `for` to `loop` for the loops, avoiding an unnecessary condition on each iteration. This also removes the need for `leb128_size`. All of these changes give significant perf wins, up to 5%.
2020-02-10 18:40:02 +11:00
loop {
if value < 0x80 {
Serialize incr comp structures to file via fixed-size buffer Reduce a large memory spike that happens during serialization by writing the incr comp structures to file by way of a fixed-size buffer, rather than an unbounded vector. Effort was made to keep the instruction count close to that of the previous implementation. However, buffered writing to a file inherently has more overhead than writing to a vector, because each write may result in a handleable error. To reduce this overhead, arrangements are made so that each LEB128-encoded integer can be written to the buffer with only one capacity and error check. Higher-level optimizations in which entire composite structures can be written with one capacity and error check are possible, but would require much more work. The performance is mostly on par with the previous implementation, with small to moderate instruction count regressions. The memory reduction is significant, however, so it seems like a worth-while trade-off.
2020-12-06 17:30:55 -08:00
unsafe {
*out.get_unchecked_mut(i).as_mut_ptr() = value as u8;
}
i += 1;
Make leb128 coding a bit faster.
2018-01-09 16:53:35 +01:00
break;
Micro-optimize the heck out of LEB128 reading and writing. This commit makes the following writing improvements: - Removes the unnecessary `write_to_vec` function. - Reduces the number of conditions per loop from 2 to 1. - Avoids a mask and a shift on the final byte. And the following reading improvements: - Removes an unnecessary type annotation. - Fixes a dangerous unchecked slice access. Imagine a slice `[0x80]` -- the current code will read past the end of the slice some number of bytes. The bounds check at the end will subsequently trigger, unless something bad (like a crash) happens first. The cost of doing bounds check in the loop body is negligible. - Avoids a mask on the final byte. And the following improvements for both reading and writing: - Changes `for` to `loop` for the loops, avoiding an unnecessary condition on each iteration. This also removes the need for `leb128_size`. All of these changes give significant perf wins, up to 5%.
2020-02-10 18:40:02 +11:00
} else {
Serialize incr comp structures to file via fixed-size buffer Reduce a large memory spike that happens during serialization by writing the incr comp structures to file by way of a fixed-size buffer, rather than an unbounded vector. Effort was made to keep the instruction count close to that of the previous implementation. However, buffered writing to a file inherently has more overhead than writing to a vector, because each write may result in a handleable error. To reduce this overhead, arrangements are made so that each LEB128-encoded integer can be written to the buffer with only one capacity and error check. Higher-level optimizations in which entire composite structures can be written with one capacity and error check are possible, but would require much more work. The performance is mostly on par with the previous implementation, with small to moderate instruction count regressions. The memory reduction is significant, however, so it seems like a worth-while trade-off.
2020-12-06 17:30:55 -08:00
unsafe {
*out.get_unchecked_mut(i).as_mut_ptr() = ((value & 0x7f) | 0x80) as u8;
}
Micro-optimize the heck out of LEB128 reading and writing. This commit makes the following writing improvements: - Removes the unnecessary `write_to_vec` function. - Reduces the number of conditions per loop from 2 to 1. - Avoids a mask and a shift on the final byte. And the following reading improvements: - Removes an unnecessary type annotation. - Fixes a dangerous unchecked slice access. Imagine a slice `[0x80]` -- the current code will read past the end of the slice some number of bytes. The bounds check at the end will subsequently trigger, unless something bad (like a crash) happens first. The cost of doing bounds check in the loop body is negligible. - Avoids a mask on the final byte. And the following improvements for both reading and writing: - Changes `for` to `loop` for the loops, avoiding an unnecessary condition on each iteration. This also removes the need for `leb128_size`. All of these changes give significant perf wins, up to 5%.
2020-02-10 18:40:02 +11:00
value >>= 7;
Serialize incr comp structures to file via fixed-size buffer Reduce a large memory spike that happens during serialization by writing the incr comp structures to file by way of a fixed-size buffer, rather than an unbounded vector. Effort was made to keep the instruction count close to that of the previous implementation. However, buffered writing to a file inherently has more overhead than writing to a vector, because each write may result in a handleable error. To reduce this overhead, arrangements are made so that each LEB128-encoded integer can be written to the buffer with only one capacity and error check. Higher-level optimizations in which entire composite structures can be written with one capacity and error check are possible, but would require much more work. The performance is mostly on par with the previous implementation, with small to moderate instruction count regressions. The memory reduction is significant, however, so it seems like a worth-while trade-off.
2020-12-06 17:30:55 -08:00
i += 1;
Make leb128 coding a bit faster.
2018-01-09 16:53:35 +01:00
}
}
Serialize incr comp structures to file via fixed-size buffer Reduce a large memory spike that happens during serialization by writing the incr comp structures to file by way of a fixed-size buffer, rather than an unbounded vector. Effort was made to keep the instruction count close to that of the previous implementation. However, buffered writing to a file inherently has more overhead than writing to a vector, because each write may result in a handleable error. To reduce this overhead, arrangements are made so that each LEB128-encoded integer can be written to the buffer with only one capacity and error check. Higher-level optimizations in which entire composite structures can be written with one capacity and error check are possible, but would require much more work. The performance is mostly on par with the previous implementation, with small to moderate instruction count regressions. The memory reduction is significant, however, so it seems like a worth-while trade-off.
2020-12-06 17:30:55 -08:00
unsafe { ::std::mem::MaybeUninit::slice_assume_init_ref(&out.get_unchecked(..i)) }
Use a more efficient encoding for opaque data in RBML.
2015-12-25 13:59:02 -05:00
}
Format the world
2019-12-22 17:42:04 -05:00
};
Use StableHasher everywhere The standard implementations of Hasher have architecture-dependent results when hashing integers. This causes problems when the hashes are stored within metadata - metadata written by one host architecture can't be read by another. To fix that, implement an architecture-independent StableHasher and use it in all places an architecture-independent hasher is needed. Fixes #38177.
2016-12-14 01:45:03 +02:00
}
Make leb128 coding a bit faster.
2018-01-09 16:53:35 +01:00
impl_write_unsigned_leb128!(write_u16_leb128, u16);
impl_write_unsigned_leb128!(write_u32_leb128, u32);
impl_write_unsigned_leb128!(write_u64_leb128, u64);
impl_write_unsigned_leb128!(write_u128_leb128, u128);
impl_write_unsigned_leb128!(write_usize_leb128, usize);
macro_rules! impl_read_unsigned_leb128 {
Serialize incr comp structures to file via fixed-size buffer Reduce a large memory spike that happens during serialization by writing the incr comp structures to file by way of a fixed-size buffer, rather than an unbounded vector. Effort was made to keep the instruction count close to that of the previous implementation. However, buffered writing to a file inherently has more overhead than writing to a vector, because each write may result in a handleable error. To reduce this overhead, arrangements are made so that each LEB128-encoded integer can be written to the buffer with only one capacity and error check. Higher-level optimizations in which entire composite structures can be written with one capacity and error check are possible, but would require much more work. The performance is mostly on par with the previous implementation, with small to moderate instruction count regressions. The memory reduction is significant, however, so it seems like a worth-while trade-off.
2020-12-06 17:30:55 -08:00
($fn_name:ident, $int_ty:ty) => {
Make leb128 coding a bit faster.
2018-01-09 16:53:35 +01:00
#[inline]
pub fn $fn_name(slice: &[u8]) -> ($int_ty, usize) {
Micro-optimize the heck out of LEB128 reading and writing. This commit makes the following writing improvements: - Removes the unnecessary `write_to_vec` function. - Reduces the number of conditions per loop from 2 to 1. - Avoids a mask and a shift on the final byte. And the following reading improvements: - Removes an unnecessary type annotation. - Fixes a dangerous unchecked slice access. Imagine a slice `[0x80]` -- the current code will read past the end of the slice some number of bytes. The bounds check at the end will subsequently trigger, unless something bad (like a crash) happens first. The cost of doing bounds check in the loop body is negligible. - Avoids a mask on the final byte. And the following improvements for both reading and writing: - Changes `for` to `loop` for the loops, avoiding an unnecessary condition on each iteration. This also removes the need for `leb128_size`. All of these changes give significant perf wins, up to 5%.
2020-02-10 18:40:02 +11:00
let mut result = 0;
Make leb128 coding a bit faster.
2018-01-09 16:53:35 +01:00
let mut shift = 0;
let mut position = 0;
Micro-optimize the heck out of LEB128 reading and writing. This commit makes the following writing improvements: - Removes the unnecessary `write_to_vec` function. - Reduces the number of conditions per loop from 2 to 1. - Avoids a mask and a shift on the final byte. And the following reading improvements: - Removes an unnecessary type annotation. - Fixes a dangerous unchecked slice access. Imagine a slice `[0x80]` -- the current code will read past the end of the slice some number of bytes. The bounds check at the end will subsequently trigger, unless something bad (like a crash) happens first. The cost of doing bounds check in the loop body is negligible. - Avoids a mask on the final byte. And the following improvements for both reading and writing: - Changes `for` to `loop` for the loops, avoiding an unnecessary condition on each iteration. This also removes the need for `leb128_size`. All of these changes give significant perf wins, up to 5%.
2020-02-10 18:40:02 +11:00
loop {
let byte = slice[position];
Make leb128 coding a bit faster.
2018-01-09 16:53:35 +01:00
position += 1;
if (byte & 0x80) == 0 {
Micro-optimize the heck out of LEB128 reading and writing. This commit makes the following writing improvements: - Removes the unnecessary `write_to_vec` function. - Reduces the number of conditions per loop from 2 to 1. - Avoids a mask and a shift on the final byte. And the following reading improvements: - Removes an unnecessary type annotation. - Fixes a dangerous unchecked slice access. Imagine a slice `[0x80]` -- the current code will read past the end of the slice some number of bytes. The bounds check at the end will subsequently trigger, unless something bad (like a crash) happens first. The cost of doing bounds check in the loop body is negligible. - Avoids a mask on the final byte. And the following improvements for both reading and writing: - Changes `for` to `loop` for the loops, avoiding an unnecessary condition on each iteration. This also removes the need for `leb128_size`. All of these changes give significant perf wins, up to 5%.
2020-02-10 18:40:02 +11:00
result |= (byte as $int_ty) << shift;
return (result, position);
} else {
result |= ((byte & 0x7F) as $int_ty) << shift;
Make leb128 coding a bit faster.
2018-01-09 16:53:35 +01:00
}
shift += 7;
}
}
Format the world
2019-12-22 17:42:04 -05:00
};
Use a more efficient encoding for opaque data in RBML.
2015-12-25 13:59:02 -05:00
}
Make leb128 coding a bit faster.
2018-01-09 16:53:35 +01:00
impl_read_unsigned_leb128!(read_u16_leb128, u16);
impl_read_unsigned_leb128!(read_u32_leb128, u32);
impl_read_unsigned_leb128!(read_u64_leb128, u64);
impl_read_unsigned_leb128!(read_u128_leb128, u128);
impl_read_unsigned_leb128!(read_usize_leb128, usize);
Serialize incr comp structures to file via fixed-size buffer Reduce a large memory spike that happens during serialization by writing the incr comp structures to file by way of a fixed-size buffer, rather than an unbounded vector. Effort was made to keep the instruction count close to that of the previous implementation. However, buffered writing to a file inherently has more overhead than writing to a vector, because each write may result in a handleable error. To reduce this overhead, arrangements are made so that each LEB128-encoded integer can be written to the buffer with only one capacity and error check. Higher-level optimizations in which entire composite structures can be written with one capacity and error check are possible, but would require much more work. The performance is mostly on par with the previous implementation, with small to moderate instruction count regressions. The memory reduction is significant, however, so it seems like a worth-while trade-off.
2020-12-06 17:30:55 -08:00
macro_rules! impl_write_signed_leb128 {
($fn_name:ident, $int_ty:ty) => {
#[inline]
pub fn $fn_name(
out: &mut [::std::mem::MaybeUninit<u8>; max_leb128_len!($int_ty)],
mut value: $int_ty,
) -> &[u8] {
let mut i = 0;
Fix LEB128 to work with the stage1 Stage 1 can’t really handle negative 128-bit literals, but an equivalent bit-not is fine
2016-08-24 15:58:40 +03:00
Serialize incr comp structures to file via fixed-size buffer Reduce a large memory spike that happens during serialization by writing the incr comp structures to file by way of a fixed-size buffer, rather than an unbounded vector. Effort was made to keep the instruction count close to that of the previous implementation. However, buffered writing to a file inherently has more overhead than writing to a vector, because each write may result in a handleable error. To reduce this overhead, arrangements are made so that each LEB128-encoded integer can be written to the buffer with only one capacity and error check. Higher-level optimizations in which entire composite structures can be written with one capacity and error check are possible, but would require much more work. The performance is mostly on par with the previous implementation, with small to moderate instruction count regressions. The memory reduction is significant, however, so it seems like a worth-while trade-off.
2020-12-06 17:30:55 -08:00
loop {
let mut byte = (value as u8) & 0x7f;
value >>= 7;
let more = !(((value == 0) && ((byte & 0x40) == 0))
|| ((value == -1) && ((byte & 0x40) != 0)));
Use a more efficient encoding for opaque data in RBML.
2015-12-25 13:59:02 -05:00
Serialize incr comp structures to file via fixed-size buffer Reduce a large memory spike that happens during serialization by writing the incr comp structures to file by way of a fixed-size buffer, rather than an unbounded vector. Effort was made to keep the instruction count close to that of the previous implementation. However, buffered writing to a file inherently has more overhead than writing to a vector, because each write may result in a handleable error. To reduce this overhead, arrangements are made so that each LEB128-encoded integer can be written to the buffer with only one capacity and error check. Higher-level optimizations in which entire composite structures can be written with one capacity and error check are possible, but would require much more work. The performance is mostly on par with the previous implementation, with small to moderate instruction count regressions. The memory reduction is significant, however, so it seems like a worth-while trade-off.
2020-12-06 17:30:55 -08:00
if more {
byte |= 0x80; // Mark this byte to show that more bytes will follow.
}
Fix LEB128 to work with the stage1 Stage 1 can’t really handle negative 128-bit literals, but an equivalent bit-not is fine
2016-08-24 15:58:40 +03:00
Serialize incr comp structures to file via fixed-size buffer Reduce a large memory spike that happens during serialization by writing the incr comp structures to file by way of a fixed-size buffer, rather than an unbounded vector. Effort was made to keep the instruction count close to that of the previous implementation. However, buffered writing to a file inherently has more overhead than writing to a vector, because each write may result in a handleable error. To reduce this overhead, arrangements are made so that each LEB128-encoded integer can be written to the buffer with only one capacity and error check. Higher-level optimizations in which entire composite structures can be written with one capacity and error check are possible, but would require much more work. The performance is mostly on par with the previous implementation, with small to moderate instruction count regressions. The memory reduction is significant, however, so it seems like a worth-while trade-off.
2020-12-06 17:30:55 -08:00
unsafe {
*out.get_unchecked_mut(i).as_mut_ptr() = byte;
}
i += 1;
if !more {
break;
}
}
unsafe { ::std::mem::MaybeUninit::slice_assume_init_ref(&out.get_unchecked(..i)) }
Use a more efficient encoding for opaque data in RBML.
2015-12-25 13:59:02 -05:00
}
Serialize incr comp structures to file via fixed-size buffer Reduce a large memory spike that happens during serialization by writing the incr comp structures to file by way of a fixed-size buffer, rather than an unbounded vector. Effort was made to keep the instruction count close to that of the previous implementation. However, buffered writing to a file inherently has more overhead than writing to a vector, because each write may result in a handleable error. To reduce this overhead, arrangements are made so that each LEB128-encoded integer can be written to the buffer with only one capacity and error check. Higher-level optimizations in which entire composite structures can be written with one capacity and error check are possible, but would require much more work. The performance is mostly on par with the previous implementation, with small to moderate instruction count regressions. The memory reduction is significant, however, so it seems like a worth-while trade-off.
2020-12-06 17:30:55 -08:00
};
Use StableHasher everywhere The standard implementations of Hasher have architecture-dependent results when hashing integers. This causes problems when the hashes are stored within metadata - metadata written by one host architecture can't be read by another. To fix that, implement an architecture-independent StableHasher and use it in all places an architecture-independent hasher is needed. Fixes #38177.
2016-12-14 01:45:03 +02:00
}
Serialize incr comp structures to file via fixed-size buffer Reduce a large memory spike that happens during serialization by writing the incr comp structures to file by way of a fixed-size buffer, rather than an unbounded vector. Effort was made to keep the instruction count close to that of the previous implementation. However, buffered writing to a file inherently has more overhead than writing to a vector, because each write may result in a handleable error. To reduce this overhead, arrangements are made so that each LEB128-encoded integer can be written to the buffer with only one capacity and error check. Higher-level optimizations in which entire composite structures can be written with one capacity and error check are possible, but would require much more work. The performance is mostly on par with the previous implementation, with small to moderate instruction count regressions. The memory reduction is significant, however, so it seems like a worth-while trade-off.
2020-12-06 17:30:55 -08:00
impl_write_signed_leb128!(write_i16_leb128, i16);
impl_write_signed_leb128!(write_i32_leb128, i32);
impl_write_signed_leb128!(write_i64_leb128, i64);
impl_write_signed_leb128!(write_i128_leb128, i128);
impl_write_signed_leb128!(write_isize_leb128, isize);
Use a more efficient encoding for opaque data in RBML.
2015-12-25 13:59:02 -05:00
Inline read_{un,}signed_leb128 and opaque::Decoder functions. These functions are all hot in rustc and inlining them speeds up most of the rustc-benchmarks by 1--2%.
2016-10-11 12:18:28 +11:00
#[inline]
Such large. Very 128. Much bits. This commit introduces 128-bit integers. Stage 2 builds and produces a working compiler which understands and supports 128-bit integers throughout. The general strategy used is to have rustc_i128 module which provides aliases for iu128, equal to iu64 in stage9 and iu128 later. Since nowhere in rustc we rely on large numbers being supported, this strategy is good enough to get past the first bootstrap stages to end up with a fully working 128-bit capable compiler. In order for this strategy to work, number of locations had to be changed to use associated max_value/min_value instead of MAX/MIN constants as well as the min_value (or was it max_value?) had to be changed to use xor instead of shift so both 64-bit and 128-bit based consteval works (former not necessarily producing the right results in stage1). This commit includes manual merge conflict resolution changes from a rebase by @est31.
2016-08-23 03:56:52 +03:00
pub fn read_signed_leb128(data: &[u8], start_position: usize) -> (i128, usize) {
Fix LEB128 to work with the stage1 Stage 1 can’t really handle negative 128-bit literals, but an equivalent bit-not is fine
2016-08-24 15:58:40 +03:00
let mut result = 0;
let mut shift = 0;
let mut position = start_position;
let mut byte;
loop {
byte = data[position];
position += 1;
Fix clippy::cast_losless
2019-06-26 14:04:37 +02:00
result |= i128::from(byte & 0x7F) << shift;
Fix LEB128 to work with the stage1 Stage 1 can’t really handle negative 128-bit literals, but an equivalent bit-not is fine
2016-08-24 15:58:40 +03:00
shift += 7;
if (byte & 0x80) == 0 {
break;
}
}
if (shift < 64) && ((byte & 0x40) != 0) {
// sign extend
result |= -(1 << shift);
}
(result, position - start_position)
Use a more efficient encoding for opaque data in RBML.
2015-12-25 13:59:02 -05:00
}
Reference in New Issue Copy Permalink