Fix &mut invalidation in ptr::swap doctest
Under Stacked Borrows with raw pointer tagging, the previous code was UB
because the code which creates the the second pointer borrows the array
through a tag in the borrow stacks below the Unique tag that our first
pointer is based on, thus invalidating the first pointer.
This is not definitely a bug and may never be real UB, but I desperately
want people to write code that conforms to SB with raw pointer tagging
so that I can write good diagnostics. The alternative aliasing models
aren't possible to diagnose well due to state space explosion.
Therefore, it would be super cool if the standard library nudged people
towards writing code that is valid with respect to SB with raw pointer
tagging.
The diagnostics that I want to write are implemented in a branch of Miri and the one for this case is below:
```
error: Undefined Behavior: attempting a read access using <2170> at alloc1068[0x0], but that tag does not exist in the borrow stack for this location
--> /home/ben/rust/library/core/src/intrinsics.rs:2103:14
|
2103 | unsafe { copy_nonoverlapping(src, dst, count) }
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| attempting a read access using <2170> at alloc1068[0x0], but that tag does not exist in the borrow stack for this location
| this error occurs as part of an access at alloc1068[0x0..0x8]
|
= help: this indicates a potential bug in the program: it performed an invalid operation, but the rules it violated are still experimental
= help: see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md for further information
help: <2170> was created due to a retag at offsets [0x0..0x10]
--> ../libcore/src/ptr/mod.rs:640:9
|
8 | let x = array[0..].as_mut_ptr() as *mut [u32; 2]; // this is `array[0..2]`
| ^^^^^^^^^^^^^^^^^^^^^^^
help: <2170> was later invalidated due to a retag at offsets [0x0..0x10]
--> ../libcore/src/ptr/mod.rs:641:9
|
9 | let y = array[2..].as_mut_ptr() as *mut [u32; 2]; // this is `array[2..4]`
| ^^^^^
= note: inside `std::intrinsics::copy_nonoverlapping::<[u32; 2]>` at /home/ben/rust/library/core/src/intrinsics.rs:2103:14
= note: inside `std::ptr::swap::<[u32; 2]>` at /home/ben/rust/library/core/src/ptr/mod.rs:685:9
note: inside `main::_doctest_main____libcore_src_ptr_mod_rs_635_0` at ../libcore/src/ptr/mod.rs:12:5
--> ../libcore/src/ptr/mod.rs:644:5
|
12 | ptr::swap(x, y);
| ^^^^^^^^^^^^^^^
note: inside `main` at ../libcore/src/ptr/mod.rs:15:3
--> ../libcore/src/ptr/mod.rs:647:3
|
15 | } _doctest_main____libcore_src_ptr_mod_rs_635_0() }
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: some details are omitted, run with `MIRIFLAGS=-Zmiri-backtrace=full` for a verbose backtrace
error: aborting due to previous error
```
Under Stacked Borrows with raw pointer tagging, the previous code was UB
because the code which creates the the second pointer borrows the array
through a tag in the borrow stacks below the Unique tag that our first
pointer is based on, thus invalidating the first pointer.
This is not definitely a bug and may never be real UB, but I desperately
want people to write code that conforms to SB with raw pointer tagging
so that I can write good diagnostics. The alternative aliasing models
aren't possible to diagnose well due to state space explosion.
Therefore, it would be super cool if the standard library nudged people
towards writing code that is valid with respect to SB with raw pointer
tagging.
Improve doc example of DerefMut
It is more illustrative, after using `*x` to modify the field, to show
in the assertion that the field has indeed been modified.
Add debug assertions to some unsafe functions
As suggested by https://github.com/rust-lang/rust/issues/51713
~~Some similar code calls `abort()` instead of `panic!()` but aborting doesn't work in a `const fn`, and the intrinsic for doing dispatch based on whether execution is in a const is unstable.~~
This picked up some invalid uses of `get_unchecked` in the compiler, and fixes them.
I can confirm that they do in fact pick up invalid uses of `get_unchecked` in the wild, though the user experience is less-than-awesome:
```
Running unittests (target/x86_64-unknown-linux-gnu/debug/deps/rle_decode_fast-04b7918da2001b50)
running 6 tests
error: test failed, to rerun pass '--lib'
Caused by:
process didn't exit successfully: `/home/ben/rle-decode-helper/target/x86_64-unknown-linux-gnu/debug/deps/rle_decode_fast-04b7918da2001b50` (signal: 4, SIGILL: illegal instruction)
```
~~As best I can tell these changes produce a 6% regression in the runtime of `./x.py test` when `[rust] debug = true` is set.~~
Latest commit (6894d559bd) brings the additional overhead from this PR down to 0.5%, while also adding a few more assertions. I think this actually covers all the places in `core` that it is reasonable to check for safety requirements at runtime.
Thoughts?
Implement provenance preserving methods on NonNull
### Description
Add the `addr`, `with_addr`, `map_addr` methods to the `NonNull` type, and map the address type to `NonZeroUsize`.
### Motivation
The `NonNull` type is useful for implementing pointer types which have the 0-niche. It is currently possible to implement these provenance preserving functions by calling `NonNull::as_ptr` and `new_unchecked`. The adding these methods makes it more ergonomic.
### Testing
Added a unit test of a non-null tagged pointer type. This is based on some real code I have elsewhere, that currently routes the pointer through a `NonZeroUsize` and back out to produce a usable pointer. I wanted to produce an ideal version of the same tagged pointer struct that preserved pointer provenance.
### Related
Extension of APIs proposed in #95228 . I can also split this out into a separate tracking issue if that is better (though I may need some pointers on how to do that).
Handle rustc_const_stable attribute in library feature collector
The library feature collector in [compiler/rustc_passes/src/lib_features.rs](551b4fa395/compiler/rustc_passes/src/lib_features.rs) has only been looking at `#[stable(…)]`, `#[unstable(…)]`, and `#[rustc_const_unstable(…)]` attributes, while ignoring `#[rustc_const_stable(…)]`. The consequences of this were:
- When any const feature got stabilized (changing one or more `rustc_const_unstable` to `rustc_const_stable`), users who had previously enabled that unstable feature using `#![feature(…)]` would get told "unknown feature", rather than rustc's nicer "the feature … has been stable since … and no longer requires an attribute to enable".
This can be seen in the way that https://github.com/rust-lang/rust/pull/93957#issuecomment-1079794660 failed after rebase:
```console
error[E0635]: unknown feature `const_ptr_offset`
--> $DIR/offset_from_ub.rs:1:35
|
LL | #![feature(const_ptr_offset_from, const_ptr_offset)]
| ^^^^^^^^^^^^^^^^
```
- We weren't enforcing that a particular feature is either stable everywhere or unstable everywhere, and that a feature that has been stabilized has the same stabilization version everywhere, both of which we enforce for the other stability attributes.
This PR updates the library feature collector to handle `rustc_const_stable`, and fixes places in the standard library and test suite where `rustc_const_stable` was being used in a way that does not meet the rules for a stability attribute.
add notes about alignment-altering reallocations to Allocator docs
As I said in https://github.com/rust-lang/wg-allocators/issues/97, the fact that calls to `grow`, `grow_zeroed`, and `shrink` may request altered alignments is surprising and may be a pitfall for implementors of `Allocator` if it's left implicit. This pull request adds a note to the "Safety" section of each function's docs making it explicit.
skip slow int_log tests in Miri
Iterating over i16::MAX many things takes a long time in Miri, let's not do that.
I added https://github.com/rust-lang/miri/pull/2044 on the Miri side to still give us some test coverage.
ptr_metadata test: avoid ptr-to-int transmutes
Pointers can have provenance, integers don't, so transmuting pointers to integers creates "non-standard" values and it is unclear how well those can be supported (https://github.com/rust-lang/unsafe-code-guidelines/issues/286).
So for this test let's take the safer option and use a pointer type instead. That also makes Miri happy. :)
**Description**
Add the `addr`, `with_addr, `map_addr` methods to the `NonNull` type,
and map the address type to `NonZeroUsize`.
**Motiviation**
The `NonNull` type is useful for implementing pointer types which have
the 0-niche. It is currently possible to implement these provenance
preserving functions by calling `NonNull::as_ptr` and `new_unchecked`.
The addition of these methods simply make it more ergonomic to use.
**Testing**
Added a unit test of a nonnull tagged pointer type. This is based on
some real code I have elsewhere, that currently routes the pointer
through a `NonZeroUsize` and back out to produce a usable pointer.
Update target_has_atomic documentation for stabilization
`cfg(target_has_atomic)` was stabilized in #93824, but this small note in the docs was not updated at the time.
allow arbitrary inherent impls for builtin types in core
Part of https://github.com/rust-lang/compiler-team/issues/487. Slightly adjusted after some talks with `@m-ou-se` about the requirements of `t-libs-api`.
This adds a crate attribute `#![rustc_coherence_is_core]` which allows arbitrary impls for builtin types in core.
For other library crates impls for builtin types should be avoided if possible. We do have to allow the existing stable impls however. To prevent us from accidentally adding more of these in the future, there is a second attribute `#[rustc_allow_incoherent_impl]` which has to be added to **all impl items**. This only supports impls for builtin types but can easily be extended to additional types in a future PR.
This implementation does not check for overlaps in these impls. Perfectly checking that requires us to check the coherence of these incoherent impls in every crate, as two distinct dependencies may add overlapping methods. It should be easy enough to detect if it goes wrong and the attribute is only intended for use inside of std.
The first two commits are mostly unrelated cleanups.
This patch series examines the question: how bad would it be if we adopted
an extremely strict pointer provenance model that completely banished all
int<->ptr casts.
The key insight to making this approach even *vaguely* pallatable is the
ptr.with_addr(addr) -> ptr
function, which takes a pointer and an address and creates a new pointer
with that address and the provenance of the input pointer. In this way
the "chain of custody" is completely and dynamically restored, making the
model suitable even for dynamic checkers like CHERI and Miri.
This is not a formal model, but lots of the docs discussing the model
have been updated to try to the *concept* of this design in the hopes
that it can be iterated on.
These debug assertions are all implemented only at runtime using
`const_eval_select`, and in the error path they execute
`intrinsics::abort` instead of being a normal debug assertion to
minimize the impact of these assertions on code size, when enabled.
Of all these changes, the bounds checks for unchecked indexing are
expected to be most impactful (case in point, they found a problem in
rustc).
Refactor set_ptr_value as with_metadata_of
Replaces `set_ptr_value` (#75091) with methods of reversed argument order:
```rust
impl<T: ?Sized> *mut T {
pub fn with_metadata_of<U: ?Sized>(self, val: *mut U) -> *mut U;
}
impl<T: ?Sized> *const T {
pub fn with_metadata_of<U: ?Sized>(self, val: *const U) -> *const U;
}
```
By reversing the arguments we achieve several clarifications:
- The function closely resembles `cast` with an argument to
initialize the metadata. This is easier to teach and answers a long
outstanding question that had restricted cast to `Sized` pointee
targets. See multiples reviews of
<https://github.com/rust-lang/rust/pull/47631>
- The 'object identity', in the form of provenance, is now preserved
from the receiver argument to the result. This helps explain the method as
a builder-style, instead of some kind of setter that would modify
something in-place. Ensuring that the result has the identity of the
`self` argument is also beneficial for an intuition of effects.
- An outstanding concern, 'Correct argument type', is avoided by not
committing to any specific argument type. This is consistent with cast
which does not require its receiver to be a 'raw address'.
Hopefully the usage examples in `sync/rc.rs` serve as sufficient examples of the style to convince the reader of the readability improvements of this style, when compared to the previous order of arguments.
I want to take the opportunity to motivate inclusion of this method _separate_ from metadata API, separate from `feature(ptr_metadata)`. It does _not_ involve the `Pointee` trait in any form. This may be regarded as a very, very light form that does not commit to any details of the pointee trait, or its associated metadata. There are several use cases for which this is already sufficient and no further inspection of metadata is necessary.
- Storing the coercion of `*mut T` into `*mut dyn Trait` as a way to dynamically cast some an arbitrary instance of the same type to a dyn trait instance. In particular, one can have a field of type `Option<*mut dyn io::Seek>` to memorize if a particular writer is seekable. Then a method `fn(self: &T) -> Option<&dyn Seek>` can be provided, which does _not_ involve the static trait bound `T: Seek`. This makes it possible to create an API that is capable of utilizing seekable streams and non-seekable streams (instead of a possible less efficient manner such as more buffering) through the same entry-point.
- Enabling more generic forms of unsizing for no-`std` smart pointers. Using the stable APIs only few concrete cases are available. One can unsize arrays to `[T]` by `ptr::slice_from_raw_parts` but unsizing a custom smart pointer to, e.g., `dyn Iterator`, `dyn Future`, `dyn Debug`, can't easily be done generically. Exposing `with_metadata_of` would allow smart pointers to offer their own `unsafe` escape hatch with similar parameters where the caller provides the unsized metadata. This is particularly interesting for embedded where `dyn`-trait usage can drastically reduce code size.
Clarify that ManuallyDrop<T> has same layout as T
This PR implements the documentation change under discussion in https://github.com/rust-lang/unsafe-code-guidelines/issues/302. It should not be approved or merged until the discussion there is resolved.
By reversing the arguments we achieve several clarifications:
- The function closely resembles `cast` but with an argument to
initialized the metadata. This is easier to teach and answers an long
outstanding question that had restricted cast to `Sized` targets
initially. See multiples reviews of
<https://github.com/rust-lang/rust/pull/47631>
- The 'object identity', in the form or provenance, is now preserved
from the call receiver to the result. This helps explain the method as
a builder-style, instead of some kind of setter that would modify
something in-place. Ensuring that the result has the identity of the
`self` argument is also beneficial for an intuition of effects.
- An outstanding concern, 'Correct argument type', is avoided by not
committing to any specific argument type. This is consistent with cast
which does not require its receiver to be a raw address.
Remove unnecessary try_opt for operations that cannot fail
As indicated in the added comments, some operation cannot overflow, so using `try_opt!` for them is unnecessary.
Optimize ascii::escape_default
`ascii::escape_default` showed up as a hot function when compiling `deunicode-1.3.1` in `@nnethercote's` [analysis](https://hackmd.io/mxdn4U58Su-UQXwzOHpHag) of `@lqd's` [rustc-benchmarking-data](https://github.com/lqd/rustc-benchmarking-data).
After taking a look at the generated assembly it looked like a LUT-based approach could be faster for `hexify()`-ing ascii characters, so that's what this PR implements
The patch looks like it provides about a 1-2% improvement in instructions for that particular crate. This should definitely be verified with a perf run as I'm still getting used to the `rustc-perf` tooling and might easily have made an error!
Rename is_{some,ok,err}_with to is_{some,ok,err}_and.
This renames `is_{some,ok,err}_with` to `is_{some,ok,err}_and`. This was discussed on the [tracking issue](https://github.com/rust-lang/rust/issues/93050).
