BTreeMap: address namespace conflicts
Fix an annoyance popping up whenever synchronizing the test cases with a version capable of miri-track-raw-pointers.
r? `@Mark-Simulacrum`
Stabilize refcell_take
Tracking Issue: #71395
``@KodrAus`` nominated this for FCP, so here's a PR!
I've never made a stabilization PR, so please mention if there's anything I can improve, thanks.
Stabilize alloc::Layout const functions
Stabilizes #67521. In particular the following stable methods are stabilized as `const fn`:
* `size`
* `align`
* `from_size_align`
Stabilizing `size` and `align` should not be controversial as they are simple (usize and NonZeroUsize) fields and I don't think there's any reason to make them not const compatible in the future. That being true, the other methods are trivially `const`. The only other issue being returning a `Result` from a `const fn` but this has been made more usable by recent stabilizations.
Split each iterator adapter and source into individual modules
This PR creates individual modules for each iterator adapter and iterator source.
This is done to enhance the readability of corresponding modules (`adapters/mod.rs` and `sources.rs`) which were hard to navigate and read because of lots of repeated lines (e.g.: `adapters/mod.rs` was 3k lines long). This is also in line with some adapters which already had their own modules (`Flatten`, `FlatMap`, `Chain`, `Zip`, `Fuse`).
This PR also makes `Take`s adapter fields private (I have no idea why they were `pub(super)` before).
r? ``@LukasKalbertodt``
Add f{32,64}::is_subnormal
The docs recommend that you use dedicated methods instead of calling `classify` directly, although there isn't actually a way of checking if a number is subnormal without calling classify. There are dedicated methods for all other forms, excluding `is_zero` (which is just `== 0.0` anyway).
Stabilize clamp
Tracking issue: https://github.com/rust-lang/rust/issues/44095
Clamp has been merged and unstable for about a year and a half now. How do we feel about stabilizing this?
More consistently use spaces after commas in lists in docs
This PR changes instances of lists that didn't use spaces after commas, like `vec![1,2,3]`, to `vec![1, 2, 3]` to be more consistent with idiomatic Rust style (the way these were looks strange to me, especially because there are often lists that *do* use spaces after the commas later in the same code block 😬).
I noticed one of these in an example in the stdlib docs and went looking for more, but as far as I can see, I'm only changing those spots in user-facing documentation or rustc output, and the changes make no semantic difference.
clarify rules for ZST Boxes
LLVM's rules around `getelementptr inbounds` with offset 0 are a bit annoying, and as a consequence we have no choice but say that a `Box<()>` pointing to previously allocated memory that has since been freed is UB. Clarify the docs to reflect this.
This is based on conversations on the LLVM mailing list.
* Here's my initial mail: https://lists.llvm.org/pipermail/llvm-dev/2019-February/130452.html
* The first email of the March part of that thread: https://lists.llvm.org/pipermail/llvm-dev/2019-March/130831.html
* First email of the April part: https://lists.llvm.org/pipermail/llvm-dev/2019-April/131693.html
The conclusion for me at least was that `getelementptr inbounds` with offset 0 is *not* the identity function, but can sometimes return `poison` even when the input is a regular pointer -- specifically, it returns `poison` when this pointer points into something that LLVM "knows has been deallocated", i.e., a former LLVM-managed allocation. It is however the identity function on pointers obtained by casting integers.
Note that there [are formal proposals](https://people.mpi-sws.org/~jung/twinsem/twinsem.pdf) for LLVM semantics where `getelementptr inbounds` with offset 0 isn't quite the identity function but never returns `poison` (it affects the provenance of the pointer but in a way that doesn't matter if this pointer is never used for memory accesses), and indeed this is likely necessary to consistently describe LLVM semantics. But with the informal LLVM LangRef that we have right now, and with LLVM devs insisting otherwise, it seems unwise to rely on this.
std: Update the bactrace crate submodule
This commit updates the `library/backtrace` submodule which primarily
pulls in support for split-debuginfo on macOS, avoiding the need for
`dsymutil` to get run to get line numbers and filenames in backtraces.
rustc_expand: Mark inner `#![test]` attributes as soft-unstable
Custom inner attributes are feature gated (https://github.com/rust-lang/rust/issues/54726) except for attributes having name `test` literally, which are not gated for historical reasons.
`#![test]` is an inner proc macro attribute, so it has all the issues described in https://github.com/rust-lang/rust/issues/54726 too.
This PR gates it with the `soft_unstable` lint.
This commit updates the `library/backtrace` submodule which primarily
pulls in support for split-debuginfo on macOS, avoiding the need for
`dsymutil` to get run to get line numbers and filenames in backtraces.
BTreeMap: replace Root with NodeRef<Owned, ...>
`NodeRef<marker::Owned, …>` already exists as a representation of root nodes, and it makes more sense to alias `Root` to that than to reuse the space-efficient `BoxedNode` that is oblivious to height, where height is required.
r? `@Mark-Simulacrum`
unix/weak: pass arguments to syscall at the given type
Given that we know the type the argument should have, it seems a bit strange not to use that information.
r? `@m-ou-se` `@cuviper`
Add lint for panic!("{}")
This adds a lint that warns about `panic!("{}")`.
`panic!(msg)` invocations with a single argument use their argument as panic payload literally, without using it as a format string. The same holds for `assert!(expr, msg)`.
This lints checks if `msg` is a string literal (after expansion), and warns in case it contained braces. It suggests to insert `"{}", ` to use the message literally, or to add arguments to use it as a format string.
This lint is also a good starting point for adding warnings about `panic!(not_a_string)` later, once [`panic_any()`](https://github.com/rust-lang/rust/pull/74622) becomes a stable alternative.
Rollup of 11 pull requests
Successful merges:
- #79119 (Clarify availability of atomic operations)
- #79123 (Add u128 and i128 integer tests)
- #79177 (Test drop order for (destructuring) assignments)
- #79181 (rustdoc: add [src] links to methods on a trait's page)
- #79183 (Make compiletest testing use the local sysroot)
- #79185 (expand/resolve: Pre-requisites to "Turn `#[derive]` into a regular macro attribute")
- #79193 (Revert #78969 "Normalize function type during validation")
- #79194 (Make as{_mut,}_slice on array::IntoIter public)
- #79204 (Add jyn514 email alias to mailmap)
- #79212 (Move `rustc_ty` -> `rustc_ty_utils`)
- #79217 (Add the "memcpy" doc alias to slice::copy_from_slice)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
Make as{_mut,}_slice on array::IntoIter public
The functions are useful in cases where you want to move data out of the IntoIter in bulk, by transmute_copy'ing the slice and then forgetting the IntoIter.
In the compiler, this is useful for providing a sped up IntoIter implementation. One can alternatively provide a separate allocate_array function but one can avoid duplicating some logic by passing everything through the generic iterator using interface.
As per suggestion in https://github.com/rust-lang/rust/pull/78569/files#r526506964
Clarify availability of atomic operations
This was noticed while we were updating the embedded rust book: https://github.com/rust-embedded/book/pull/273/files
These targets do natively have atomic load/stores, but do not support CAS operations.
libary: Forward compiler-builtins "asm" and "mangled-names" feature
In principle this is a followup of rust-lang/rust#78472. In the previous PR was the support of the test crate missing.
Now users will be able to do:
```
cargo build -Zbuild-std=core -Zbuild-std-features=compiler-builtins-asm
```
and correctly get the assembly implemenations for `memcpy` and friends.
Remove semicolon from internal `err` macro
This macro is used in expression position (a match arm), and only
compiles because of #33953
Regardless of what happens with that issue, this makes the
usage of the macro less confusing at the call site.
This macro is used in expression position (a match arm), and only
compiles because of #33953
Regardless of what happens with that issue, this makes the
usage of the macro less confusing at the call site.
Fix typo in `std::io::Write` docs
These referred to a “`Write`er”—extra *e*. Presumably a copy-paste
holdover from “`Read`er”.
Test Plan:
Running ``git grep '`\?[Ww]rite`\?er'`` no longer finds any results.
wchargin-branch: io-write-docs
add trailing_zeros and leading_zeros to non zero types
as a way towards being able to use the optimized intrinsics ctlz_nonzero and cttz_nonzero from stable.
have not crated any tracking issue if this is not a solution that is wanted
Tighten the bounds on atomic Ordering in std::sys::unix::weak::Weak
This moves reading this from multiple SeqCst reads to Relaxed read + Acquire fence if we are actually going to use the data.
Would love to avoid the Acquire fence, but doing so would need Ordering::Consume, which neither Rust, nor LLVM supports (a shame, since this fence is hardly free on ARM, which is what I was hoping to improve).
r? ``@Amanieu`` (Sorry for always picking you, but I know a lot of people wouldn't feel comfortable reviewing atomic ordering changes)
linux: try to use libc getrandom to allow interposition
We'll try to use a weak `getrandom` symbol first, because that allows
things like `LD_PRELOAD` interposition. For example, perf measurements
might want to disable randomness to get reproducible results. If the
weak symbol is not found, we fall back to a raw `SYS_getrandom` call.
Updated the list of white-listed target features for x86
This PR both adds in-source documentation on what to look out for when adding a new (X86) feature set and [adds all that are detectable at run-time in Rust stable as of 1.27.0](https://github.com/rust-lang/stdarch/blob/master/crates/std_detect/src/detect/arch/x86.rs).
This should only enable the use of the corresponding LLVM intrinsics.
Actual intrinsics need to be added separately in rust-lang/stdarch.
It also re-orders the run-time-detect test statements to be more consistent
with the actual list of intrinsics whitelisted and removes underscores not present
in the actual names (which might be mistaken as being part of the name)
The reference for LLVM's feature names used is [this file](https://github.com/llvm/llvm-project/blob/master/llvm/include/llvm/Support/X86TargetParser.def).
This PR was motivated as the compiler end's part for allowing #67329 to be adressed over on rust-lang/stdarch
These referred to a “`Write`er”—extra *e*. Presumably a copy-paste
holdover from “`Read`er”.
Test Plan:
Running ``git grep '`\?[Ww]rite`\?er'`` no longer finds any results.
wchargin-branch: io-write-docs
Rollup of 9 pull requests
Successful merges:
- #77939 (Ensure that the source code display is working with DOS backline)
- #78138 (Upgrade dlmalloc to version 0.2)
- #78967 (Make codegen tests compatible with extra inlining)
- #79027 (Limit storage duration of inlined always live locals)
- #79077 (document that __rust_alloc is also magic to our LLVM fork)
- #79088 (clarify `span_label` documentation)
- #79097 (Code block invalid html tag lint)
- #79105 (std: Fix test `symlink_hard_link` on Windows)
- #79107 (build-manifest: strip newline from rustc version)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
std: Fix test `symlink_hard_link` on Windows
The test was introduced in https://github.com/rust-lang/rust/pull/78026 and fails depending on Windows version and admin rights.
Other similar tests check for symlink creation permissions before doing anything, this PR performs the same check for `symlink_hard_link` as well.
Upgrade dlmalloc to version 0.2
In preparation of adding dynamic memory management support for SGXv2-enabled platforms, the dlmalloc crate has been refactored. More specifically, support has been added to implement platform specification outside of the dlmalloc crate. (see https://github.com/alexcrichton/dlmalloc-rs/pull/15)
This PR upgrades dlmalloc to version 0.2 for the `wasm` and `sgx` targets.
As the dlmalloc changes have received a positive review, but have not been merged yet, this PR contains a commit to prevent tidy from aborting CI prematurely.
cc: `@jethrogb`
Make the libstd build script smaller
Of all sysroot crates currently only compiler_builtins, miniz_oxide and std require a build script. compiler_builtins uses to conditionally enable certain features and possibly compile a C version ([source](63ccaf11f0/build.rs)), miniz_oxide only uses it to detect if liballoc is supported as the MSRV is 1.34.0 instead of the 1.36.0 which stabilized liballoc ([source](28514ec09f/miniz_oxide/build.rs)). std now only uses it to enable `freebsd12` when the `RUST_STD_FREEBSD_12_ABI` env var is set, to determine if `restricted-std` should be set, to set the `STD_ENV_ARCH` env var identical to `CARGO_CFG_TARGET_ARCH`, and to unconditionally enable `backtrace_in_libstd`.
If all build scripts were to be removed, it would be possible for rustc to completely compile it's own sysroot. It currently requires a rustc version that already has an available libstd to compile the build scripts. If rustc can completely compile it's own sysroot, rustbuild could be simplified to not forcefully use the bootstrap compiler for build scripts.
`@rustbot` modify labels: +T-compiler +libs-impl
We'll try to use a weak `getrandom` symbol first, because that allows
things like `LD_PRELOAD` interposition. For example, perf measurements
might want to disable randomness to get reproducible results. If the
weak symbol is not found, we fall back to a raw `SYS_getrandom` call.
Simplify output capturing
This is a sequence of incremental improvements to the unstable/internal `set_panic` and `set_print` mechanism used by the `test` crate:
1. Remove the `LocalOutput` trait and use `Arc<Mutex<dyn Write>>` instead of `Box<dyn LocalOutput>`. In practice, all implementations of `LocalOutput` were just `Arc<Mutex<..>>`. This simplifies some logic and removes all custom `Sink` implementations such as `library/test/src/helpers/sink.rs`. Also removes a layer of indirection, as the outermost `Box` is now gone. It also means that locking now happens per `write_fmt`, not per individual `write` within. (So `"{} {}\n"` now results in one `lock()`, not four or more.)
2. Since in all cases the `dyn Write`s were just `Vec<u8>`s, replace the type with `Arc<Mutex<Vec<u8>>>`. This simplifies things more, as error handling and flushing can be removed now. This also removes the hack needed in the default panic handler to make this work with `::realstd`, as (unlike `Write`) `Vec<u8>` is from `alloc`, not `std`.
3. Replace the `RefCell`s by regular `Cell`s. The `RefCell`s were mostly used as `mem::replace(&mut *cell.borrow_mut(), something)`, which is just `Cell::replace`. This removes an unecessary bookkeeping and makes the code a bit easier to read.
4. Merge `set_panic` and `set_print` into a single `set_output_capture`. Neither the test crate nor rustc (the only users of this feature) have a use for using these separately. Merging them simplifies things even more. This uses a new function name and feature name, to make it clearer this is internal and not supposed to be used by other crates.
Might be easier to review per commit.
Rename/Deprecate LayoutErr in favor of LayoutError
Implements rust-lang/wg-allocators#73.
This patch renames LayoutErr to LayoutError, and uses a type alias to support users using the old name.
The new name will be instantly stable in release 1.49 (current nightly), the type alias will become deprecated in release 1.51 (so that when the current nightly is 1.51, 1.49 will be stable).
This is the only error type in `std` that ends in `Err` rather than `Error`, if this PR lands all stdlib error types will end in `Error` 🥰
Test structural matching for all range types
As of #70166 all range types (`core::ops::Range` etc.) can be structurally matched upon, and by extension used in const generics. In reference to the fact that this is a publicly observable property of these types, and thus falls under the Rust stability guarantees of the standard library, a regression test was added in #70283.
This regression test was implemented by me by testing for the ability to use the range types within const generics, but that is not the actual property the std guarantees now (const generics is still unstable). This PR addresses that situation by adding extra tests for the range types that directly test whether they can be structurally matched upon.
Note: also adds the otherwise unrelated test `test_range_to_inclusive` for completeness with the other range unit tests
Implement `Index` and `IndexMut` for arrays
Adds implementations of `Index` and `IndexMut` for arrays that simply forward to the slice indexing implementation in order to fix the following problem:
If you implement `Index<MyIndexType>` for an array, you lose all the other indexing functionality that used to be available to the array via its implicit coercion to a slice. An example of what I'm talking about:
```rust
use std::ops::Index;
pub enum MyIndexType {
_0, _1, _2, _3, _4, _5, _6, _7,
}
impl<T> Index<MyIndexType> for [T; 8] {
type Output = T;
fn index(&self, index: MyIndexType) -> &T {
unsafe { self.get_unchecked(index as usize) }
}
}
fn main() {
let array = [11u8; 8];
println!("{:?}", array[MyIndexType::_0]); // OK
println!("{:?}", array[0usize]); // error[E0277]
// ^^^^^^^^^^^^^ `[u8; 8]` cannot be indexed by `usize`
}
```
Now users will be able to do:
```
cargo build -Zbuild-std=core -Zbuild-std-features=compiler-builtins-asm
```
and correctly get the assembly implemenations for `memcpy` and friends.
BTreeMap: fix pointer provenance rules in underfullness
Continuing on #78480, and for readability, and possibly for performance: avoid aliasing when handling underfull nodes, and consolidate the code doing that. In particular:
- Avoid the rather explicit aliasing for internal nodes in `remove_kv_tracking`.
- Climb down to the root to handle underfull nodes using a reborrowed handle, rather than one copied with `ptr::read`, before resuming on the leaf level.
- Integrate the code tracking leaf edge position into the functions performing changes, rather than bolting it on.
r? `@Mark-Simulacrum`
Implement BTreeMap::retain and BTreeSet::retain
Adds new methods `BTreeMap::retain` and `BTreeSet::retain`. These are implemented on top of `drain_filter` (#70530).
The API of these methods is identical to `HashMap::retain` and `HashSet::retain`, which were implemented in #39560 and stabilized in #36648. The docs and tests are also copied from HashMap/HashSet.
The new methods are unstable, behind the `btree_retain` feature gate, with tracking issue #79025. See also rust-lang/rfcs#1338.
Fix an intrinsic invocation on threaded wasm
This looks like it was forgotten to get updated in #74482 and wasm with
threads isn't built on CI so we didn't catch this by accident.
refactor: removing alloc::collections::vec_deque ignore-tidy-filelength
This PR removes the need for ignore-tidy-filelength for alloc::collections::vec_deque which is part of the issue https://github.com/rust-lang/rust/issues/60302
It is probably easiest to review this PR by looking at it commit by commit rather than looking at the overall diff.
specialize io::copy to use copy_file_range, splice or sendfile
Fixes#74426.
Also covers #60689 but only as an optimization instead of an official API.
The specialization only covers std-owned structs so it should avoid the problems with #71091
Currently linux-only but it should be generalizable to other unix systems that have sendfile/sosplice and similar.
There is a bit of optimization potential around the syscall count. Right now it may end up doing more syscalls than the naive copy loop when doing short (<8KiB) copies between file descriptors.
The test case executes the following:
```
[pid 103776] statx(3, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_ALL, {stx_mask=STATX_ALL|STATX_MNT_ID, stx_attributes=0, stx_mode=S_IFREG|0644, stx_size=17, ...}) = 0
[pid 103776] write(4, "wxyz", 4) = 4
[pid 103776] write(4, "iklmn", 5) = 5
[pid 103776] copy_file_range(3, NULL, 4, NULL, 5, 0) = 5
```
0-1 `stat` calls to identify the source file type. 0 if the type can be inferred from the struct from which the FD was extracted
𝖬 `write` to drain the `BufReader`/`BufWriter` wrappers. only happen when buffers are present. 𝖬 ≾ number of wrappers present. If there is a write buffer it may absorb the read buffer contents first so only result in a single write. Vectored writes would also be an option but that would require more invasive changes to `BufWriter`.
𝖭 `copy_file_range`/`splice`/`sendfile` until file size, EOF or the byte limit from `Take` is reached. This should generally be *much* more efficient than the read-write loop and also have other benefits such as DMA offload or extent sharing.
## Benchmarks
```
OLD
test io::tests::bench_file_to_file_copy ... bench: 21,002 ns/iter (+/- 750) = 6240 MB/s [ext4]
test io::tests::bench_file_to_file_copy ... bench: 35,704 ns/iter (+/- 1,108) = 3671 MB/s [btrfs]
test io::tests::bench_file_to_socket_copy ... bench: 57,002 ns/iter (+/- 4,205) = 2299 MB/s
test io::tests::bench_socket_pipe_socket_copy ... bench: 142,640 ns/iter (+/- 77,851) = 918 MB/s
NEW
test io::tests::bench_file_to_file_copy ... bench: 14,745 ns/iter (+/- 519) = 8889 MB/s [ext4]
test io::tests::bench_file_to_file_copy ... bench: 6,128 ns/iter (+/- 227) = 21389 MB/s [btrfs]
test io::tests::bench_file_to_socket_copy ... bench: 13,767 ns/iter (+/- 3,767) = 9520 MB/s
test io::tests::bench_socket_pipe_socket_copy ... bench: 26,471 ns/iter (+/- 6,412) = 4951 MB/s
```
Previously EOVERFLOW handling was only applied for io::copy specialization
but not for fs::copy sharing the same code.
Additionally we lower the chunk size to 1GB since we have a user report
that older kernels may return EINVAL when passing 0x8000_0000
but smaller values succeed.
