improve case conversion happy path
Someone shared the source code for [Go's string case conversion](19156a5474/src/strings/strings.go (L558-L616)).
It features a hot path for ascii-only strings (although I assume for reasons specific to go, they've opted for a read safe hot loop).
I've borrowed these ideas and also kept our existing code to provide a fast path + seamless utf-8 correct path fallback.
(Naive) Benchmarks can be found here https://github.com/conradludgate/case-conv
For the cases where non-ascii is found near the start, the performance of this algorithm does fall back to original speeds and has not had any measurable speed loss
As currently written, when a logic error occurs in a collection's trait
parameters, this allows *completely arbitrary* misbehavior, so long as
it does not cause undefined behavior in std. However, because the extent
of misbehavior is not specified, it is allowed for *any* code in std to
start misbehaving in arbitrary ways which are not formally UB; consider
the theoretical example of a global which gets set on an observed logic
error. Because the misbehavior is only bound by not resulting in UB from
safe APIs and the crate-level encapsulation boundary of all of std, this
makes writing user unsafe code that utilizes std theoretically
impossible, as it now relies on undocumented QOI that unrelated parts of
std cannot be caused to misbehave by a misuse of std::collections APIs.
In practice, this is a nonconcern, because std has reasonable QOI and an
implementation that takes advantage of this freedom is essentially a
malicious implementation and only compliant by the most langauage-lawyer
reading of the documentation.
To close this hole, we just add a small clause to the existing logic
error paragraph that ensures that any misbehavior is limited to the
collection which observed the logic error, making it more plausible to
prove the soundness of user unsafe code.
This is not meant to be formal; a formal refinement would likely need to
mention that values derived from the collection can also misbehave after a
logic error is observed, as well as define what it means to "observe" a
logic error in the first place. This fix errs on the side of informality
in order to close the hole without complicating a normal reading which
can assume a reasonable nonmalicious QOI.
See also [discussion on IRLO][1].
[1]: https://internals.rust-lang.org/t/using-std-collections-and-unsafe-anything-can-happen/16640
Clarify slice and Vec iteration order
While already being inferable from the doc examples, it wasn't fully specified. This is the only logical way to do a slice iterator, so I think this should be uncontroversial. It also improves the `Vec::into_iter` example to better show the order and that the iterator returns owned values.
Improve codegen of String::retain method
This pull-request improve the codegen of the `String::retain` method.
Using `unwrap_unchecked` helps the optimizer to not generate a panicking path that will never be taken for valid UTF-8 like string.
Using `encode_utf8` saves us from an expensive call to `memcpy`, as the optimizer is unable to realize that `ch_len <= 4` and so can generate much better assembly code.
https://rust.godbolt.org/z/z73ohenfc
Remove libstd's calls to `C-unwind` foreign functions
Remove all libstd and its dependencies' usage of `extern "C-unwind"`.
This is a prerequiste of a WIP PR which will forbid libraries calling `extern "C-unwind"` functions to be compiled in `-Cpanic=unwind` and linked against `panic_abort` (this restriction is necessary to address soundness bug #96926).
Cargo will ensure all crates are compiled with the same `-Cpanic` but the std is only compiled `-Cpanic=unwind` but needs the ability to be linked into `-Cpanic=abort`.
Currently there are two places where `C-unwind` is used in libstd:
* `__rust_start_panic` is used for interfacing to the panic runtime. This could be `extern "Rust"`
* `_{rdl,rg}_oom`: a shim `__rust_alloc_error_handler` will be generated by codegen to call into one of these; they can also be `extern "Rust"` (in fact, the generated shim is used as `extern "Rust"`, so I am not even sure why these are not, probably because they used to `extern "C"` and was changed to `extern "C-unwind"` when we allow alloc error hooks to unwind, but they really should just be using Rust ABI).
For dependencies, there is only one `extern "C-unwind"` function call, in `unwind` crate. This can be expressed as a re-export.
More dicussions can be seen in the Zulip thread: https://rust-lang.zulipchat.com/#narrow/stream/210922-project-ffi-unwind/topic/soundness.20in.20mixed.20panic.20mode
`@rustbot` label: T-libs F-c_unwind
Use default alloc_error_handler for hermit
Hermit now properly separates kernel from userspace.
Applications for hermit can now use Rust's default `alloc_error_handler` instead of calling the kernel's `__rg_oom`.
CC: ``@stlankes``
Add `sub_ptr` on pointers (the `usize` version of `offset_from`)
We have `add`/`sub` which are the `usize` versions of `offset`, this adds the `usize` equivalent of `offset_from`. Like how `.add(d)` replaced a whole bunch of `.offset(d as isize)`, you can see from the changes here that it's fairly common that code actually knows the order between the pointers and *wants* a `usize`, not an `isize`.
As a bonus, this can do `sub nuw`+`udiv exact`, rather than `sub`+`sdiv exact`, which can be optimized slightly better because it doesn't have to worry about negatives. That's why the slice iterators weren't using `offset_from`, though I haven't updated that code in this PR because slices are so perf-critical that I'll do it as its own change.
This is an intrinsic, like `offset_from`, so that it can eventually be allowed in CTFE. It also allows checking the extra safety condition -- see the test confirming that CTFE catches it if you pass the pointers in the wrong order.
Like we have `add`/`sub` which are the `usize` version of `offset`, this adds the `usize` equivalent of `offset_from`. Like how `.add(d)` replaced a whole bunch of `.offset(d as isize)`, you can see from the changes here that it's fairly common that code actually knows the order between the pointers and *wants* a `usize`, not an `isize`.
As a bonus, this can do `sub nuw`+`udiv exact`, rather than `sub`+`sdiv exact`, which can be optimized slightly better because it doesn't have to worry about negatives. That's why the slice iterators weren't using `offset_from`, though I haven't updated that code in this PR because slices are so perf-critical that I'll do it as its own change.
This is an intrinsic, like `offset_from`, so that it can eventually be allowed in CTFE. It also allows checking the extra safety condition -- see the test confirming that CTFE catches it if you pass the pointers in the wrong order.
Implement a lint to warn about unused macro rules
This implements a new lint to warn about unused macro rules (arms/matchers), similar to the `unused_macros` lint added by #41907 that warns about entire macros.
```rust
macro_rules! unused_empty {
(hello) => { println!("Hello, world!") };
() => { println!("empty") }; //~ ERROR: 1st rule of macro `unused_empty` is never used
}
fn main() {
unused_empty!(hello);
}
```
Builds upon #96149 and #96156.
Fixes#73576
Warn on unused `#[doc(hidden)]` attributes on trait impl items
[Zulip conversation](https://rust-lang.zulipchat.com/#narrow/stream/266220-rustdoc/topic/.E2.9C.94.20Validy.20checks.20for.20.60.23.5Bdoc.28hidden.29.5D.60).
Whether an associated item in a trait impl is shown or hidden in the documentation entirely depends on the corresponding item in the trait declaration. Rustdoc completely ignores `#[doc(hidden)]` attributes on impl items. No error or warning is emitted:
```rust
pub trait Tr { fn f(); }
pub struct Ty;
impl Tr for Ty { #[doc(hidden)] fn f() {} }
// ^^^^^^^^^^^^^^ ignored by rustdoc and currently
// no error or warning issued
```
This may lead users to the wrong belief that the attribute has an effect. In fact, several such cases are found in the standard library (I've removed all of them in this PR).
There does not seem to exist any incentive to allow this in the future either: Impl'ing a trait for a type means the type *fully* conforms to its API. Users can add `#[doc(hidden)]` to the whole impl if they want to hide the implementation or add the attribute to the corresponding associated item in the trait declaration to hide the specific item. Hiding an implementation of an associated item does not make much sense: The associated item can still be found on the trait page.
This PR emits the warn-by-default lint `unused_attribute` for this case with a future-incompat warning.
`@rustbot` label T-compiler T-rustdoc A-lint
Remove `#[rustc_deprecated]`
This removes `#[rustc_deprecated]` and introduces diagnostics to help users to the right direction (that being `#[deprecated]`). All uses of `#[rustc_deprecated]` have been converted. CI is expected to fail initially; this requires #95958, which includes converting `stdarch`.
I plan on following up in a short while (maybe a bootstrap cycle?) removing the diagnostics, as they're only intended to be short-term.
Add a dedicated length-prefixing method to `Hasher`
This accomplishes two main goals:
- Make it clear who is responsible for prefix-freedom, including how they should do it
- Make it feasible for a `Hasher` that *doesn't* care about Hash-DoS resistance to get better performance by not hashing lengths
This does not change rustc-hash, since that's in an external crate, but that could potentially use it in future.
Fixes#94026
r? rust-lang/libs
---
The core of this change is the following two new methods on `Hasher`:
```rust
pub trait Hasher {
/// Writes a length prefix into this hasher, as part of being prefix-free.
///
/// If you're implementing [`Hash`] for a custom collection, call this before
/// writing its contents to this `Hasher`. That way
/// `(collection![1, 2, 3], collection![4, 5])` and
/// `(collection![1, 2], collection![3, 4, 5])` will provide different
/// sequences of values to the `Hasher`
///
/// The `impl<T> Hash for [T]` includes a call to this method, so if you're
/// hashing a slice (or array or vector) via its `Hash::hash` method,
/// you should **not** call this yourself.
///
/// This method is only for providing domain separation. If you want to
/// hash a `usize` that represents part of the *data*, then it's important
/// that you pass it to [`Hasher::write_usize`] instead of to this method.
///
/// # Examples
///
/// ```
/// #![feature(hasher_prefixfree_extras)]
/// # // Stubs to make the `impl` below pass the compiler
/// # struct MyCollection<T>(Option<T>);
/// # impl<T> MyCollection<T> {
/// # fn len(&self) -> usize { todo!() }
/// # }
/// # impl<'a, T> IntoIterator for &'a MyCollection<T> {
/// # type Item = T;
/// # type IntoIter = std::iter::Empty<T>;
/// # fn into_iter(self) -> Self::IntoIter { todo!() }
/// # }
///
/// use std:#️⃣:{Hash, Hasher};
/// impl<T: Hash> Hash for MyCollection<T> {
/// fn hash<H: Hasher>(&self, state: &mut H) {
/// state.write_length_prefix(self.len());
/// for elt in self {
/// elt.hash(state);
/// }
/// }
/// }
/// ```
///
/// # Note to Implementers
///
/// If you've decided that your `Hasher` is willing to be susceptible to
/// Hash-DoS attacks, then you might consider skipping hashing some or all
/// of the `len` provided in the name of increased performance.
#[inline]
#[unstable(feature = "hasher_prefixfree_extras", issue = "88888888")]
fn write_length_prefix(&mut self, len: usize) {
self.write_usize(len);
}
/// Writes a single `str` into this hasher.
///
/// If you're implementing [`Hash`], you generally do not need to call this,
/// as the `impl Hash for str` does, so you can just use that.
///
/// This includes the domain separator for prefix-freedom, so you should
/// **not** call `Self::write_length_prefix` before calling this.
///
/// # Note to Implementers
///
/// The default implementation of this method includes a call to
/// [`Self::write_length_prefix`], so if your implementation of `Hasher`
/// doesn't care about prefix-freedom and you've thus overridden
/// that method to do nothing, there's no need to override this one.
///
/// This method is available to be overridden separately from the others
/// as `str` being UTF-8 means that it never contains `0xFF` bytes, which
/// can be used to provide prefix-freedom cheaper than hashing a length.
///
/// For example, if your `Hasher` works byte-by-byte (perhaps by accumulating
/// them into a buffer), then you can hash the bytes of the `str` followed
/// by a single `0xFF` byte.
///
/// If your `Hasher` works in chunks, you can also do this by being careful
/// about how you pad partial chunks. If the chunks are padded with `0x00`
/// bytes then just hashing an extra `0xFF` byte doesn't necessarily
/// provide prefix-freedom, as `"ab"` and `"ab\u{0}"` would likely hash
/// the same sequence of chunks. But if you pad with `0xFF` bytes instead,
/// ensuring at least one padding byte, then it can often provide
/// prefix-freedom cheaper than hashing the length would.
#[inline]
#[unstable(feature = "hasher_prefixfree_extras", issue = "88888888")]
fn write_str(&mut self, s: &str) {
self.write_length_prefix(s.len());
self.write(s.as_bytes());
}
}
```
With updates to the `Hash` implementations for slices and containers to call `write_length_prefix` instead of `write_usize`.
`write_str` defaults to using `write_length_prefix` since, as was pointed out in the issue, the `write_u8(0xFF)` approach is insufficient for hashers that work in chunks, as those would hash `"a\u{0}"` and `"a"` to the same thing. But since `SipHash` works byte-wise (there's an internal buffer to accumulate bytes until a full chunk is available) it overrides `write_str` to continue to use the add-non-UTF-8-byte approach.
---
Compatibility:
Because the default implementation of `write_length_prefix` calls `write_usize`, the changed hash implementation for slices will do the same thing the old one did on existing `Hasher`s.
This accomplishes two main goals:
- Make it clear who is responsible for prefix-freedom, including how they should do it
- Make it feasible for a `Hasher` that *doesn't* care about Hash-DoS resistance to get better performance by not hashing lengths
This does not change rustc-hash, since that's in an external crate, but that could potentially use it in future.
Tweak the vec-calloc runtime check to only apply to shortish-arrays
r? `@Mark-Simulacrum`
`@nbdd0121` pointed out in https://github.com/rust-lang/rust/pull/95362#issuecomment-1114085395 that LLVM currently doesn't constant-fold the `IsZero` check for long arrays, so that seems like a reasonable justification for limiting it.
It appears that it's based on length, not byte size, (https://godbolt.org/z/4s48Y81dP), so that's what I used in the PR. Maybe it's a ["the number of inlining shall be three"](https://youtu.be/s4wnuiCwTGU?t=320) sort of situation.
Certainly there's more that could be done here -- that generated code that checks long arrays byte-by-byte is highly suboptimal, for example -- but this is an easy, low-risk tweak.
std::fmt: Various fixes and improvements to documentation
This PR contains the following changes:
- **Added argument index comments to examples for specifying precision**
The examples for specifying the precision have comments explaining which
argument the specifier is referring to. However, for implicit positional
arguments, the examples simply refer to "next arg". To simplify following the
comments, "next arg" was supplemented with the actual resulting argument index.
- **Fixed documentation for specifying precision via `.*`**
The documentation stated that in case of the syntax `{<arg>:<spec>.*}`, "the
`<arg>` part refers to the value to print, and the precision must come in the
input preceding `<arg>`". This is not correct: the <arg> part does indeed refer
to the value to print, but the precision does not come in the input preciding
arg, but in the next implicit input (as if specified with {}).
Fixes#96413.
- **Fix the grammar documentation**
According to the grammar documented, the format specifier `{: }` should not be
legal because of the whitespace it contains. However, in reality, this is
perfectly fine because the actual implementation allows spaces before the
closing brace. Fixes#71088.
Also, the exact meaning of most of the terminal symbols was not specified, for
example the meaning of `identifier`.
- **Removed reference to Formatter::buf and other private fields**
Formatter::buf is not a public field and therefore isn't very helpful in user-
facing documentation. Also, the other public fields of Formatter were removed
during stabilization of std::fmt (4af3494bb0) and can only be accessed via
getters.
- **Improved list of formatting macros**
Two improvements:
1. write! can not only receive a `io::Write`, but also a `fmt::Write` as first argument.
2. The description texts now contain links to the actual macros for easier
navigation.
Clarify docs for `from_raw_parts` on `Vec` and `String`
Closes#95427
Original safety explanation for `from_raw_parts` was unclear on safety for consuming a C string. This clarifies when doing so is safe.
Classify BinaryHeap & LinkedList unit tests as such
All but one of these so-called integration test case are unit tests, just like btree's were (#75531). In addition, reunite the unit tests of linked_list that were split off during #23104 because they needed to remain unit tests (they were later moved to the separate file they are in during #63207). The two sets could remain separate files, but I opted to merge them back together, more or less in the order they used to be, apart from one duplicate name `test_split_off` and one duplicate tiny function `list_from`.
Using unwrap_unchecked helps the optimizer to not generate panicking
path, that will never be taken for valid UTF-8 like string.
Using encode_utf8 saves us a call to a memcpy, as the optimizer is
unable to realize that ch_len <= 4 and so can generate much better
assembly code.
https://rust.godbolt.org/z/z73ohenfc
Two improvements:
1. write! can not only receive a `io::Write`, but also a `fmt::Write` as first argument.
2. The description texts now contain links to the actual macros for easier
navigation.
Formatter::buf is not a public field and therefore isn't very helpful in user-
facing documentation. Also, the other public fields of Formatter were made
private during stabilization of std::fmt (4af3494bb0) and can now only be read
via accessor methods.
According to the grammar documented, the format specifier `{: }` should not be
legal because of the whitespace it contains. However, in reality, this is
perfectly fine because the actual implementation allows spaces before the
closing brace. Fixes#71088.
Also, the exact meaning of most of the terminal symbols was not specified, for
example the meaning of `identifier`.
The examples for specifying the precision have comments explaining which
argument the specifier is referring to. However, for implicit positional
arguments, the examples simply talk about "next arg". To make it easier for
readers to follow the comments, "next arg" was supplemented with the actual
resulting argument index.
The documentation stated that in case of the syntax `{<arg>:<spec>.*}`, "the
`<arg>` part refers to the value to print, and the precision must come in the
input preceding `<arg>`". This is not correct: the <arg> part does indeed refer
to the value to print, but the precision does not come in the input preciding
arg, but in the next implicit input (as if specified with {}).
Fixes#96413.
Implement str to [u8] conversion for refcounted containers
This seems motivated to complete the APIs for shared containers since we already have similar allocation-free conversions for strings like `From<Box<[u8]>> for Box<str>`.
Insta-stable since it's a new trait impl?
Clarify that `Cow::into_owned` returns owned data
Two sections of the `Cow::into_owned` docs imply that `into_owned` returns a `Cow`. Clarify that it returns the underlying owned object, either cloned or extracted from the `Cow`.
Fix some confusing wording and improve slice-search-related docs
This adds more links between `contains` and `binary_search` because I do think they have some relevant connections. If your (big) slice happens to be sorted and you know it, surely you should be using `[3; 100].binary_search(&5).is_ok()` over `[3; 100].contains(&5)`?
This also fixes the confusing "searches this sorted X" wording which just sounds really weird because it doesn't know whether it's actually sorted. It should be but it may not be. The new wording should make it clearer that you will probably want to sort it and in the same sentence it also mentions the related function `contains`.
Similarly, this mentions `binary_search` on `contains`' docs.
This also fixes some other minor stuff and inconsistencies.
[test] Add test cases for untested functions for VecDeque
Added test cases of the following functions
- get
- get_mut
- swap
- reserve_exact
- try_reserve_exact
- try_reserve
- contains
- rotate_left
- rotate_right
- binary_search
- binary_search_by
- binary_search_by_key
`alloc`: make `vec!` unavailable under `no_global_oom_handling`
`alloc`: make `vec!` unavailable under `no_global_oom_handling`
The `vec!` macro has 3 rules, but two are not usable under
`no_global_oom_handling` builds of the standard library
(even with a zero size):
```rust
let _ = vec![42]; // Error: requires `exchange_malloc` lang_item.
let _ = vec![42; 0]; // Error: cannot find function `from_elem`.
```
Thus those two rules should not be available to begin with.
The remaining one, with an empty matcher, is just a shorthand for
`new()` and may not make as much sense to have alone, since the
idea behind `vec!` is to enable `Vec`s to be defined with the same
syntax as array expressions. Furthermore, the documentation can be
confusing since it shows the other rules.
Thus perhaps it is better and simpler to disable `vec!` entirely
under `no_global_oom_handling` environments, and let users call
`new()` instead:
```rust
let _: Vec<i32> = vec![];
let _: Vec<i32> = Vec::new();
```
Notwithstanding this, a `try_vec!` macro would be useful, such as
the one introduced in https://github.com/rust-lang/rust/pull/95051.
If the shorthand for `new()` is deemed worth keeping on its own,
then it may be interesting to have a separate `vec!` macro with
a single rule and different, simpler documentation.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Speed up Vec::clear().
Currently it just calls `truncate(0)`. `truncate()` is (a) not marked as
`#[inline]`, and (b) more general than needed for `clear()`.
This commit changes `clear()` to do the work itself. This modest change
was first proposed in rust-lang#74172, where the reviewer rejected it because
there was insufficient evidence that `Vec::clear()`'s performance
mattered enough to justify the change. Recent changes within rustc have
made `Vec::clear()` hot within `macro_parser.rs`, so the change is now
clearly worthwhile.
Although it doesn't show wins on CI perf runs, this seems to be because they
use PGO. But not all platforms currently use PGO. Also, local builds don't use
PGO, and `truncate` sometimes shows up in an over-represented fashion in local
profiles. So local profiling will be made easier by this change.
Note that this will also benefit `String::clear()`, because it just
calls `Vec::clear()`.
Finally, the commit removes the `vec-clear.rs` codegen test. It was
added in #52908. From before then until now, `Vec::clear()` just called
`Vec::truncate()` with a zero length. The body of Vec::truncate() has
changed a lot since then. Now that `Vec::clear()` is doing actual work
itself, and not just calling `Vec::truncate()`, it's not surprising that
its generated code includes a load and an icmp. I think it's reasonable
to remove this test.
r? `@m-ou-se`
The `vec!` macro has 3 rules, but two are not usable under
`no_global_oom_handling` builds of the standard library
(even with a zero size):
```rust
let _ = vec![42]; // Error: requires `exchange_malloc` lang_item.
let _ = vec![42; 0]; // Error: cannot find function `from_elem`.
```
Thus those two rules should not be available to begin with.
The remaining one, with an empty matcher, is just a shorthand for
`new()` and may not make as much sense to have alone, since the
idea behind `vec!` is to enable `Vec`s to be defined with the same
syntax as array expressions. Furthermore, the documentation can be
confusing since it shows the other rules.
Thus perhaps it is better and simpler to disable `vec!` entirely
under `no_global_oom_handling` environments, and let users call
`new()` instead:
```rust
let _: Vec<i32> = vec![];
let _: Vec<i32> = Vec::new();
```
Notwithstanding this, a `try_vec!` macro would be useful, such as
the one introduced in https://github.com/rust-lang/rust/pull/95051.
If the shorthand for `new()` is deemed worth keeping on its own,
then it may be interesting to have a separate `vec!` macro with
a single rule and different, simpler documentation.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Optimize RcInnerPtr::inc_strong()/inc_weak() instruction count
Inspired by this internals thread: https://internals.rust-lang.org/t/rc-optimization-on-64-bit-targets/16362
[The generated assembly is a bit smaller](https://rust.godbolt.org/z/TeTnf6144) and is a more efficient usage of the CPU's instruction cache. `unlikely` doesn't impact any of the small artificial tests I've done, but I've included it in case it might help more complex scenarios when this is inlined.
Currently it just calls `truncate(0)`. `truncate()` is (a) not marked as
`#[inline]`, and (b) more general than needed for `clear()`.
This commit changes `clear()` to do the work itself. This modest change
was first proposed in rust-lang#74172, where the reviewer rejected it because
there was insufficient evidence that `Vec::clear()`'s performance
mattered enough to justify the change. Recent changes within rustc have
made `Vec::clear()` hot within `macro_parser.rs`, so the change is now
clearly worthwhile.
Although it doesn't show wins on CI perf runs, this seems to be because they
use PGO. But not all platforms currently use PGO. Also, local builds don't use
PGO, and `truncate` sometimes shows up in an over-represented fashion in local
profiles. So local profiling will be made easier by this change.
Note that this will also benefit `String::clear()`, because it just
calls `Vec::clear()`.
Finally, the commit removes the `vec-clear.rs` codegen test. It was
added in #52908. From before then until now, `Vec::clear()` just called
`Vec::truncate()` with a zero length. The body of Vec::truncate() has
changed a lot since then. Now that `Vec::clear()` is doing actual work
itself, and not just calling `Vec::truncate()`, it's not surprising that
its generated code includes a load and an icmp. I think it's reasonable
to remove this test.
Update binary_search example to instead redirect to partition_point
Inspired by discussion in the tracking issue for `Result::into_ok_or_err`: https://github.com/rust-lang/rust/issues/82223#issuecomment-1067098167
People are surprised by us not providing a `Result<T, T> -> T` conversion, and the main culprit for this confusion seems to be the `binary_search` API. We should instead redirect people to the equivalent API that implicitly does that `Result<T, T> -> T` conversion internally which should obviate the need for the `into_ok_or_err` function and give us time to work towards a more general solution that applies to all enums rather than just `Result` such as making or_patterns usable for situations like this via postfix `match`.
I choose to duplicate the example rather than simply moving it from `binary_search` to partition point because most of the confusion seems to arise when people are looking at `binary_search`. It makes sense to me to have the example presented immediately rather than requiring people to click through to even realize there is an example. If I had to put it in only one place I'd leave it in `binary_search` and remove it from `partition_point` but it seems pretty obviously relevant to `partition_point` so I figured the best option would be to duplicate it.
**This Commit**
Adds some clarity around indexing into Strings and the constraints
driving various decisions there.
**Why?**
The [`String` documentation][0] mentions how `String`s can't be indexed
but `Range` has an implementation for `SliceIndex<str>`. This can be
confusing. There are also several statements to explain the lack of
`String` indexing:
- the inability to index into a `String` is an implication of UTF-8
encoding
- indexing into a `String` could not be constant-time with UTF-8
encoding
- indexing into a `String` does not have an obvious return type
This last statement made sense but the first two seemed contradictory to
the documentation around [`SliceIndex<str>`][1] which mention:
- one can index into a `String` with a `Range` (also called substring
slicing but it uses the same syntax and the method name is `index`)
- `Range` indexing into a `String` is constant-time
To resolve this seeming contradiction the documentation is reworked to
more clearly explain what factors drive the decision to disallow
indexing into a `String` with a single number.
[0]: https://doc.rust-lang.org/stable/std/string/struct.String.html#utf-8
[1]: https://doc.rust-lang.org/stable/std/slice/trait.SliceIndex.html#impl-SliceIndex%3Cstr%3E
Relevant commit messages from squashed history in order:
Add initial version of ThinBox
update test to actually capture failure
swap to middle ptr impl based on matthieu-m's design
Fix stack overflow in debug impl
The previous version would take a `&ThinBox<T>` and deref it once, which
resulted in a no-op and the same type, which it would then print causing
an endless recursion. I've switched to calling `deref` by name to let
method resolution handle deref the correct number of times.
I've also updated the Drop impl for good measure since it seemed like it
could be falling prey to the same bug, and I'll be adding some tests to
verify that the drop is happening correctly.
add test to verify drop is behaving
add doc examples and remove unnecessary Pointee bounds
ThinBox: use NonNull
ThinBox: tests for size
Apply suggestions from code review
Co-authored-by: Alphyr <47725341+a1phyr@users.noreply.github.com>
use handle_alloc_error and fix drop signature
update niche and size tests
add cfg for allocating APIs
check null before calculating offset
add test for zst and trial usage
prevent optimizer induced ub in drop and cleanup metadata gathering
account for arbitrary size and alignment metadata
Thank you nika and thomcc!
Update library/alloc/src/boxed/thin.rs
Co-authored-by: Josh Triplett <josh@joshtriplett.org>
Update library/alloc/src/boxed/thin.rs
Co-authored-by: Josh Triplett <josh@joshtriplett.org>
Bump bootstrap compiler to 1.61.0 beta
This PR bumps the bootstrap compiler to the 1.61.0 beta. The first commit changes the stage0 compiler, the second commit applies the "mechanical" changes and the third and fourth commits apply changes explained in the relevant comments.
r? `@Mark-Simulacrum`
Fix double drop of allocator in IntoIter impl of Vec
Fixes#95269
The `drop` impl of `IntoIter` reconstructs a `RawVec` from `buf`, `cap` and `alloc`, when that `RawVec` is dropped it also drops the allocator. To avoid dropping the allocator twice we wrap it in `ManuallyDrop` in the `InttoIter` struct.
Note this is my first contribution to the standard library, so I might be missing some details or a better way to solve this.
allow arbitrary inherent impls for builtin types in core
Part of https://github.com/rust-lang/compiler-team/issues/487. Slightly adjusted after some talks with `@m-ou-se` about the requirements of `t-libs-api`.
This adds a crate attribute `#![rustc_coherence_is_core]` which allows arbitrary impls for builtin types in core.
For other library crates impls for builtin types should be avoided if possible. We do have to allow the existing stable impls however. To prevent us from accidentally adding more of these in the future, there is a second attribute `#[rustc_allow_incoherent_impl]` which has to be added to **all impl items**. This only supports impls for builtin types but can easily be extended to additional types in a future PR.
This implementation does not check for overlaps in these impls. Perfectly checking that requires us to check the coherence of these incoherent impls in every crate, as two distinct dependencies may add overlapping methods. It should be easy enough to detect if it goes wrong and the attribute is only intended for use inside of std.
The first two commits are mostly unrelated cleanups.
Refactor set_ptr_value as with_metadata_of
Replaces `set_ptr_value` (#75091) with methods of reversed argument order:
```rust
impl<T: ?Sized> *mut T {
pub fn with_metadata_of<U: ?Sized>(self, val: *mut U) -> *mut U;
}
impl<T: ?Sized> *const T {
pub fn with_metadata_of<U: ?Sized>(self, val: *const U) -> *const U;
}
```
By reversing the arguments we achieve several clarifications:
- The function closely resembles `cast` with an argument to
initialize the metadata. This is easier to teach and answers a long
outstanding question that had restricted cast to `Sized` pointee
targets. See multiples reviews of
<https://github.com/rust-lang/rust/pull/47631>
- The 'object identity', in the form of provenance, is now preserved
from the receiver argument to the result. This helps explain the method as
a builder-style, instead of some kind of setter that would modify
something in-place. Ensuring that the result has the identity of the
`self` argument is also beneficial for an intuition of effects.
- An outstanding concern, 'Correct argument type', is avoided by not
committing to any specific argument type. This is consistent with cast
which does not require its receiver to be a 'raw address'.
Hopefully the usage examples in `sync/rc.rs` serve as sufficient examples of the style to convince the reader of the readability improvements of this style, when compared to the previous order of arguments.
I want to take the opportunity to motivate inclusion of this method _separate_ from metadata API, separate from `feature(ptr_metadata)`. It does _not_ involve the `Pointee` trait in any form. This may be regarded as a very, very light form that does not commit to any details of the pointee trait, or its associated metadata. There are several use cases for which this is already sufficient and no further inspection of metadata is necessary.
- Storing the coercion of `*mut T` into `*mut dyn Trait` as a way to dynamically cast some an arbitrary instance of the same type to a dyn trait instance. In particular, one can have a field of type `Option<*mut dyn io::Seek>` to memorize if a particular writer is seekable. Then a method `fn(self: &T) -> Option<&dyn Seek>` can be provided, which does _not_ involve the static trait bound `T: Seek`. This makes it possible to create an API that is capable of utilizing seekable streams and non-seekable streams (instead of a possible less efficient manner such as more buffering) through the same entry-point.
- Enabling more generic forms of unsizing for no-`std` smart pointers. Using the stable APIs only few concrete cases are available. One can unsize arrays to `[T]` by `ptr::slice_from_raw_parts` but unsizing a custom smart pointer to, e.g., `dyn Iterator`, `dyn Future`, `dyn Debug`, can't easily be done generically. Exposing `with_metadata_of` would allow smart pointers to offer their own `unsafe` escape hatch with similar parameters where the caller provides the unsized metadata. This is particularly interesting for embedded where `dyn`-trait usage can drastically reduce code size.
Allow comparing `Vec`s with different allocators using `==`
See https://stackoverflow.com/q/71021633/7884305.
I did not changed the `PartialOrd` impl too because it was not generic already (didn't support `Vec<T> <=> Vec<U> where T: PartialOrd<U>`).
Does it needs tests?
I don't think this will hurt type inference much because the default allocator is usually not inferred (`new()` specifies it directly, and even with other allocators, you pass the allocator to `new_in()` so the compiler usually knows the type).
I think this requires FCP since the impls are already stable.
Fix typo in `String::try_reserve_exact` docs
Copying the pattern from `Vec::try_reserve_exact` and `String::try_reserve`,
it looks like this doc comment is intending to refer to the currently-being-documented
function.
Copying the pattern from `Vec::try_reserve_exact` and `String::try_reserve`,
it looks like this doc comment is intending to refer to the currently-being-documented
function.
Hermit now properly separates kernel from userspace.
Applications for hermit can now use Rust's default alloc_error_handler instead of calling the kernel's __rg_oom.
add module-level documentation for vec's in-place iteration
As requested in the last libs team meeting and during previous reviews.
Feel free to point out any gaps you encounter, after all non-obvious things may with hindsight seem obvious to me.
r? `@yaahc`
CC `@steffahn`
By reversing the arguments we achieve several clarifications:
- The function closely resembles `cast` but with an argument to
initialized the metadata. This is easier to teach and answers an long
outstanding question that had restricted cast to `Sized` targets
initially. See multiples reviews of
<https://github.com/rust-lang/rust/pull/47631>
- The 'object identity', in the form or provenance, is now preserved
from the call receiver to the result. This helps explain the method as
a builder-style, instead of some kind of setter that would modify
something in-place. Ensuring that the result has the identity of the
`self` argument is also beneficial for an intuition of effects.
- An outstanding concern, 'Correct argument type', is avoided by not
committing to any specific argument type. This is consistent with cast
which does not require its receiver to be a raw address.
BTreeMap::entry: Avoid allocating if no insertion
This PR allows the `VacantEntry` to borrow from an empty tree with no root, and to lazily allocate a new root node when the user calls `.insert(value)`.
Made the fields of VecDeque's Iter private by creating a Iter::new(...) function to create a new instance of Iter and migrating usage to use Iter::new(...).
Improve doc wording for retain on some collections
I found the documentation wording on the various retain methods on many collections to be unusual.
I tried to invert the relation by switching `such that` with `for which` .
Use modern formatting for format! macros
This updates the standard library's documentation to use the new format_args syntax.
The documentation is worthwhile to update as it should be more idiomatic
(particularly for features like this, which are nice for users to get acquainted
with). The general codebase is likely more hassle than benefit to update: it'll
hurt git blame, and generally updates can be done by folks updating the code if
(and when) that makes things more readable with the new format.
A few places in the compiler and library code are updated (mostly just due to
already having been done when this commit was first authored).
`eprintln!("{}", e)` becomes `eprintln!("{e}")`, but `eprintln!("{}", e.kind())` remains untouched.
This updates the standard library's documentation to use the new syntax. The
documentation is worthwhile to update as it should be more idiomatic
(particularly for features like this, which are nice for users to get acquainted
with). The general codebase is likely more hassle than benefit to update: it'll
hurt git blame, and generally updates can be done by folks updating the code if
(and when) that makes things more readable with the new format.
A few places in the compiler and library code are updated (mostly just due to
already having been done when this commit was first authored).
fix typo in btree/vec doc: Self -> self
this pr fixes#92345
the documentation refers to the object the method is called for, not the type, so it should be using the lower case self.
Fix a layout possible miscalculation in `alloc::RawVec`
A layout miscalculation could happen in `RawVec` when used with a type whose size isn't a multiple of its alignment. I don't know if such type can exist in Rust, but the Layout API provides ways to manipulate such types. Anyway, it is better to calculate memory size in a consistent way.
Add documentation to more `From::from` implementations.
For users looking at documentation through IDE popups, this gives them relevant information rather than the generic trait documentation wording “Performs the conversion”. For users reading the documentation for a specific type for any reason, this informs them when the conversion may allocate or copy significant memory versus when it is always a move or cheap copy.
Notes on specific cases:
* The new documentation for `From<T> for T` explains that it is not a conversion at all.
* Also documented `impl<T, U> Into<U> for T where U: From<T>`, the other central blanket implementation of conversion.
* The new documentation for construction of maps and sets from arrays of keys mentions the handling of duplicates. Future work could be to do this for *all* code paths that convert an iterable to a map or set.
* I did not add documentation to conversions of a specific error type to a more general error type.
* I did not add documentation to unstable code.
This change was prepared by searching for the text "From<... for" and so may have missed some cases that for whatever reason did not match. I also looked for `Into` impls but did not find any worth documenting by the above criteria.
Replace iterator-based construction of collections by `Into<T>`
Just a few quality of life improvements in the doc examples. I also removed some `Vec`s in favor of arrays.
Stabilize arc_new_cyclic
This stabilizes feature `arc_new_cyclic` as the implementation has been merged for one year and there is no unresolved questions. The FCP is not started yet.
Closes#75861 .
``@rustbot`` label +T-libs-api
Improve `Arc` and `Rc` documentation
This makes two changes (I can split the PR if necessary, but the changes are pretty small):
1. A bunch of trait implementations claimed to be zero cost; however, they use the `Arc<T>: From<Box<T>>` impl which is definitely not free, especially for large dynamically sized `T`.
2. The code in deferred initialization examples unnecessarily used excessive amounts of `unsafe`. This has been reduced.
doc: guarantee call order for sort_by_cached_key
`slice::sort_by_cached_key` takes a caching function `f: impl FnMut(&T) -> K`, which means that the order that calls to the caching function are made is user-visible. This adds a clause to the documentation to promise the current behavior, which is that `f` is called on all elements of the slice from left to right, unless the slice has len < 2 in which case `f` is not called.
For example, this can be used to ensure that the following code is a correct way to involve the index of the element in the sort key:
```rust
let mut index = 0;
slice.sort_by_cached_key(|x| (my_key(index, x), index += 1).0);
```
Clarify explicitly that BTree{Map,Set} are ordered.
One of the main reasons one would want to use a BTree{Map,Set} rather than a Hash{Map,Set} is because they maintain their keys in sorted order; but this was never explicitly stated in the top-level docs (it was only indirectly alluded to there, and stated explicitly in the docs for `iter`, `values`, etc.)
This PR states the ordering guarantee more prominently.
Add diagnostic items for macros
For use in Clippy, it adds diagnostic items to all the stable public macros
Clippy has lints that look for almost all of these (currently by name or path), but there are a few that aren't currently part of any lint, I could remove those if it's preferred to add them as needed rather than ahead of time
Yield means something else in the context of generators, which are
sufficiently close to iterators that it's better to avoid the
terminology collision here.
Partially stabilize `maybe_uninit_extra`
This covers:
```rust
impl<T> MaybeUninit<T> {
pub unsafe fn assume_init_read(&self) -> T { ... }
pub unsafe fn assume_init_drop(&mut self) { ... }
}
```
It does not cover the const-ness of `write` under `const_maybe_uninit_write` nor the const-ness of `assume_init_read` (this commit adds `const_maybe_uninit_assume_init_read` for that).
FCP: https://github.com/rust-lang/rust/issues/63567#issuecomment-958590287.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
This covers:
impl<T> MaybeUninit<T> {
pub unsafe fn assume_init_read(&self) -> T { ... }
pub unsafe fn assume_init_drop(&mut self) { ... }
}
It does not cover the const-ness of `write` under
`const_maybe_uninit_write` nor the const-ness of
`assume_init_read` (this commit adds
`const_maybe_uninit_assume_init_read` for that).
FCP: https://github.com/rust-lang/rust/issues/63567#issuecomment-958590287.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Replace usages of vec![].into_iter with [].into_iter
`[].into_iter` is idiomatic over `vec![].into_iter` because its simpler and faster (unless the vec is optimized away in which case it would be the same)
So we should change all the implementation, documentation and tests to use it.
I skipped:
* `src/tools` - Those are copied in from upstream
* `src/test/ui` - Hard to tell if `vec![].into_iter` was used intentionally or not here and not much benefit to changing it.
* any case where `vec![].into_iter` was used because we specifically needed a `Vec::IntoIter<T>`
* any case where it looked like we were intentionally using `vec![].into_iter` to test it.
Mak DefId to AccessLevel map in resolve for export
hir_id to accesslevel in resolve and applied in privacy
using local def id
removing tracing probes
making function not recursive and adding comments
Move most of Exported/Public res to rustc_resolve
moving public/export res to resolve
fix missing stability attributes in core, std and alloc
move code to access_levels.rs
return for some kinds instead of going through them
Export correctness, macro changes, comments
add comment for import binding
add comment for import binding
renmae to access level visitor, remove comments, move fn as closure, remove new_key
fmt
fix rebase
fix rebase
fmt
fmt
fix: move macro def to rustc_resolve
fix: reachable AccessLevel for enum variants
fmt
fix: missing stability attributes for other architectures
allow unreachable pub in rustfmt
fix: missing impl access level + renaming export to reexport
Missing impl access level was found thanks to a test in clippy
Fix a minor mistake in `String::try_reserve_exact` examples
The examples of `String::try_reserve_exact` didn't actually use `try_reserve_exact`, which was probably a minor mistake, and this PR fixed it.
Drop guards in slice sorting derive src pointers from &mut T, which is invalidated by interior mutation in comparison
I tried to run https://github.com/rust-lang/miri-test-libstd on `alloc` with `-Zmiri-track-raw-pointers`, and got a failure on the test `slice::panic_safe`. The test failure has nothing to do with panic safety, it's from how the test tests for panic safety.
I minimized the test failure into this very silly program:
```rust
use std::cell::Cell;
use std::cmp::Ordering;
#[derive(Clone)]
struct Evil(Cell<usize>);
fn main() {
let mut input = vec![Evil(Cell::new(0)); 3];
// Hits the bug pattern via CopyOnDrop in core
input.sort_unstable_by(|a, _b| {
a.0.set(0);
Ordering::Less
});
// Hits the bug pattern via InsertionHole in alloc
input.sort_by(|_a, b| {
b.0.set(0);
Ordering::Less
});
}
```
To fix this, I'm just removing the mutability/uniqueness where it wasn't required.
Implement split_at_spare_mut without Deref to a slice so that the spare slice is valid
~I'm not sure I understand what's going on here correctly. And I'm pretty sure this safety comment needs to be changed. I'm just referring to the same thing that `as_mut_ptr_range` does.~ (Thanks `@RalfJung` for the guidance and clearing things up)
I tried to run https://github.com/rust-lang/miri-test-libstd on alloc with -Zmiri-track-raw-pointers, and got a failure on the test `vec::test_extend_from_within`.
I minimized the test failure into this program:
```rust
#![feature(vec_split_at_spare)]
fn main() {
Vec::<i32>::with_capacity(1).split_at_spare_mut();
}
```
The problem is that the existing implementation is actually getting a pointer range where both pointers are derived from the initialized region of the Vec's allocation, but we need the second one to be valid for the region between len and capacity. (thanks Ralf for clearing this up)
RawVec: don't recompute capacity after allocating.
Currently it sets the capacity to `ptr.len() / mem::size_of::<T>()`
after any buffer allocation/reallocation. This would be useful if
allocators ever returned a `NonNull<[u8]>` with a size larger than
requested. But this never happens, so it's not useful.
Removing this slightly reduces the size of generated LLVM IR, and
slightly speeds up the hot path of `RawVec` growth.
r? `@ghost`
Currently it sets the capacity to `ptr.len() / mem::size_of::<T>()`
after any buffer allocation/reallocation. This would be useful if
allocators ever returned a `NonNull<[u8]>` with a size larger than
requested. But this never happens, so it's not useful.
Removing this slightly reduces the size of generated LLVM IR, and
slightly speeds up the hot path of `RawVec` growth.
The previous implementation used slice::as_mut_ptr_range to derive the
pointer for the spare capacity slice. This is invalid, because that
pointer is derived from the initialized region, so it does not have
provenance over the uninitialized region.
Update example code for Vec::splice to change the length
The current example for `Vec::splice` illustrates the replacement of a section of length 2 with a new section of length 2. This isn't a particularly interesting case for splice, and makes it look a bit like a shorthand for the kind of manipulations that could be done with a mutable slice.
In order to provide a stronger example, this updates the example to use different lengths for the source and destination regions, and uses a slice from the middle of the vector to illustrate that this does not necessarily have to be at the beginning or the end.
Resolves#92067
The src pointers in CopyOnDrop and InsertionHole used to be *mut T, and
were derived via automatic conversion from &mut T. According to Stacked
Borrows 2.1, this means that those pointers become invalidated by
interior mutation in the comparison function.
But there's no need for mutability in this code path. Thus, we can
change the drop guards to use *const and derive those from &T.
add BinaryHeap::try_reserve and BinaryHeap::try_reserve_exact
`try_reserve` of many collections were stablized in https://github.com/rust-lang/rust/pull/87993 in 1.57.0. Add `try_reserve` for the rest collections such as `BinaryHeap` should be not controversial.
Use spare_capacity_mut instead of invalid unchecked indexing when joining str
This is a fix for https://github.com/rust-lang/rust/issues/91574
I think in general I'd prefer to see this code implemented with raw pointers or `MaybeUninit::write_slice`, but there's existing code in here based on copying from slice to slice, so converting everything from `&[T]` to `&[MaybeUninit<T>]` is less disruptive.
BTree: improve public descriptions and comments
BTreeSet has always used the term "value" next to and meaning the same thing as "elements" (in the mathematical sense but also used for key-value pairs in BTreeMap), while in the BTreeMap sense these "values" are known as "keys" and definitely not "values". Today I had enough of that.
r? `@Mark-Simulacrum`
Btree: assert more API compatibility
Introducing a member such as `BTreeSet::min()` would silently break compatibility if no code calls the existing `BTreeSet::min(set)`. `BTreeSet` is the only btree class silently bringing in stable members, apart from many occurrences of `#[derive(Debug)]` on iterators.
r? `@Mark-Simulacrum`
Rollup of 11 pull requests
Successful merges:
- #91668 (Remove the match on `ErrorKind::Other`)
- #91678 (Add tests fixed by #90023)
- #91679 (Move core/stream/stream/mod.rs to core/stream/stream.rs)
- #91681 (fix typo in `intrinsics::raw_eq` docs)
- #91686 (Fix `Vec::reserve_exact` documentation)
- #91697 (Delete Utf8Lossy::from_str)
- #91706 (Add unstable book entries for parts of asm that are not being stabilized)
- #91709 (Replace iterator-based set construction by *Set::From<[T; N]>)
- #91716 (Improve x.py logging and defaults a bit more)
- #91747 (Add pierwill to .mailmap)
- #91755 (Fix since attribute for const_linked_list_new feature)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
Replace iterator-based set construction by *Set::From<[T; N]>
This uses the array-based construction for `BtreeSet`s and `HashSet`s instead of first creating an iterator. I could also replace the `let mut a = Set::new(); a.insert(...);` fragments if desired.
Fix `Vec::reserve_exact` documentation
The documentation previously said the new capacity cannot overflow `usize`, but in fact it cannot exceed `isize::MAX`.
Update documentation to use `from()` to initialize `HashMap`s and `BTreeMap`s
As of Rust 1.56, `HashMap` and `BTreeMap` both have associated `from()` functions. I think using these in the documentation cleans things up a bit. It allows us to remove some of the `mut`s and avoids the Initialize-Then-Modify anti-pattern.
replace vec::Drain drop loops with drop_in_place
The `Drain::drop` implementation came up in https://github.com/rust-lang/rust/pull/82185#issuecomment-789584796 as potentially interfering with other optimization work due its widespread use somewhere in `println!`
`@rustbot` label T-libs-impl
Suggest try_reserve in try_reserve_exact
During developing #91529 , I found that `try_reserve_exact` suggests `reserve` for further insertions. I think it's a mistake by copy&paste, `try_reserve` is better here.
Implement VecDeque::retain_mut
Part of https://github.com/rust-lang/rust/issues/90829.
In https://github.com/rust-lang/rust/pull/90772, someone suggested that `retain_mut` should also be implemented on `VecDeque`. I think that it follows the same logic (coherency). So first: is it ok? Second: should I create a new feature for it or can we put it into the same one?
r? `@joshtriplett`
For users looking at documentation through IDE popups, this gives them
relevant information rather than the generic trait documentation wording
“Performs the conversion”. For users reading the documentation for a
specific type for any reason, this informs them when the conversion may
allocate or copy significant memory versus when it is always a move or
cheap copy.
Notes on specific cases:
* The new documentation for `From<T> for T` explains that it is not a
conversion at all.
* Also documented `impl<T, U> Into<U> for T where U: From<T>`, the other
central blanket implementation of conversion.
* I did not add documentation to conversions of a specific error type to
a more general error type.
* I did not add documentation to unstable code.
This change was prepared by searching for the text "From<... for" and so
may have missed some cases that for whatever reason did not match. I
also looked for `Into` impls but did not find any worth documenting by
the above criteria.
