Test and fix `size_hint` for slice’s [r]split* iterators
Adds extensive test (of `size_hint`) for all the _[r]split*_ iterators.
Fixes `size_hint` upper bound for _split_inclusive*_ iterators which was one higher than necessary for non-empty slices.
Fixes `size_hint` lower bound for _[r]splitn*_ iterators when _n == 0_, which was one too high.
**Lower bound being one too high was a logic error, violating the correctness condition of `size_hint`.**
_Edit:_ I’ve opened an issue for that bug, so this PR fixes#87978
Specialize `Vec::clone_from` for `Copy` types
This should improve performance and reduce code size.
This also improves `clone_from` for `String`, `OsString` and `PathBuf`.
WASI previously used `u32` as its `RawFd` type, since its "file descriptors"
are unsigned table indices, and there's no fundamental reason why WASI can't
have more than 2^31 handles.
However, this creates myriad little incompability problems with code
that also supports Unix platforms, where `RawFd` is `c_int`. While WASI
isn't a Unix, it often shares code with Unix, and this difference made
such shared code inconvenient. #87329 is the most recent example of such
code.
So, switch WASI to use `c_int`, which is `i32`. This will mean that code
intending to support WASI should ideally avoid assuming that negative file
descriptors are invalid, even though POSIX itself says that file descriptors
are never negative.
This is a breaking change, but `RawFd` is considerd an experimental
feature in [the documentation].
[the documentation]: https://doc.rust-lang.org/stable/std/os/wasi/io/type.RawFd.html
Implement `black_box` using intrinsic
Introduce `black_box` intrinsic, as suggested in https://github.com/rust-lang/rust/pull/87590#discussion_r680468700.
This is still codegenned as empty inline assembly for LLVM. For MIR interpretation and cranelift it's treated as identity.
cc `@Amanieu` as this is related to inline assembly
cc `@bjorn3` for rustc_codegen_cranelift changes
cc `@RalfJung` as this affects MIRI
r? `@nagisa` I suppose
Adds extensive test for all the [r]split* iterators.
Fixes size_hint upper bound for split_inclusive* iterators which was one higher than necessary for non-empty slices.
Fixes size_hint lower bound for [r]splitn* iterators when n==0, which was one too high.
The new implementation allows some `memcpy`s to be optimized away,
so the uninit value in ui/sanitize/memory.rs is constructed directly
onto the return place. Therefore the sanitizer now says that the
value is allocated by `main` rather than `random`.
STD support for the ESP-IDF framework
Dear all,
This PR is implementing libStd support for the [ESP-IDF](https://github.com/espressif/esp-idf) newlib-based framework, which is the open source SDK provided by Espressif for their MCU family (esp32, esp32s2, esp32c3 and all other forthcoming ones).
Note that this PR has a [sibling PR](https://github.com/rust-lang/libc/pull/2310) against the libc crate, which implements proper declarations for all ESP-IDF APIs which are necessary for libStd support.
# Implementation approach
The ESP-IDF framework - despite being bare metal - offers a relatively complete POSIX API based on newlib. `pthread`, BSD sockets, file descriptors, and even a small file-system VFS layer. Perhaps the only significant exception is the lack of support for processes, which is to be expected of course on bare metal.
Therefore, the libStd support is implemented as a set of (hopefully small) changes to the `sys/unix` family of modules, in the form of conditional-compilation branches based either on `target_os = "espidf"` or in a couple of cases - based on `target_env = "newlib"` (the latter was already there actually and is not part of this patch).
The PR also contains two new targets:
- `riscv32imc-esp-espidf`
- `riscv32imac-esp-espidf`
... which are essentially copies of `riscv32imc-unknown-none-elf` and `riscv32imac-unknown-none-elf`, but enriched with proper `linker`, `linker_flavor`, `families`, `os`, `env` etc. specifications so that (a) the proper conditional compilation branches in libStd are selected when compiling with these targets and (b) the correct linker is used.
Since support for atomics is a precondition for libStd, the `riscv32imc-esp-espidf` target additionally is configured in such a way, so as to emit libcalls to the `__sync*` & `__atomic*` GCC functions, which are already implemented in the ESP-IDF framework. If this modification is not acceptable, we can also live with only the `riscv32imac-esp-espidf` target as well. While the RiscV chips of Espressif lack native atomics support, the relevant instructions are transparently emulated in the ESP-IDF framework using invalid instruction trap. This modification was implemented specifically with Rust support in mind.
# Target maintainers
In case this PR eventually gets merged, you can list myself as a Target Maintainer.
More importantly, Espressif (the chip vendor) is now actively involved and [embracing](https://github.com/espressif/rust-esp32-example/blob/main/docs/rust-on-xtensa.md) all [Rust-related efforts](https://github.com/esp-rs) which were originally a community effort. In light of that, I suppose `@MabezDev` - who initiated the Rust-on-Espressif efforts back in time and who now works for Espressif won't object to being listed as a maintainer as well.
**EDIT:** I was hinted (thanks, `@Urgau)` that answering the Tier 3 policy explicitly might be helpful. Answers below.
# Tier 3 Target Policy - answers
> A proposed target or target-specific patch that substantially changes code shared with other targets (not just target-specific code) must be reviewed and approved by the appropriate team for that shared code before acceptance.
Hopefully, the changes introduced by the ESP-IDF libStd support are rather on the small side. They are completely contained within the `sys/unix` set of modules (that is, aside from the obviously necessary one-liners in the `unwind` crate and in `build.rs`).
> A tier 3 target must have a designated developer or developers (the "target maintainers") on record to be CCed when issues arise regarding the target. (The mechanism to track and CC such developers may evolve over time.)
`@ivmarkov`
`@MabezDev`
> Targets must use naming consistent with any existing targets; for instance, a target for the same CPU or OS as an existing Rust target should use the same name for that CPU or OS. Targets should normally use the same names and naming conventions as used elsewhere in the broader ecosystem beyond Rust (such as in other toolchains), unless they have a very good reason to diverge. Changing the name of a target can be highly disruptive, especially once the target reaches a higher tier, so getting the name right is important even for a tier 3 target.
The two introduced targets follow as much as possible the naming conventions of the other targets. I.e. taking the bare-metal `riscv32imac_unknown_none_elf` as a base:
* The name of the new target was derived by replacing `none` with `espidf` to designate the `target_os`.
* `_elf` was removed, as the non-bare metal targets seem not to have it
* `-newlib` was deliberately NOT added at the end, as I believe the chance of having two simultaneously active separate targets for the ESP-IDF framework with different C libraries (say, newlib vs musl) is way too small
* Finally, we replaced the middle `unknown` with `esp` which is kind of the name of the whole chipset MCU family (and abbreviation from Espressif which is too long). It will stay `esp` for all RiscV32-based MCUs of the company, as they all use the riscv32imc instruction set. By necessity however (disambiguation), it will be `esp32` or `esp32s2` or `esp32s3` for the Xtensa-based MCUs as all of these have their own variation of the Xtensa architecture. (The Xtensa targets are not part of this PR, even though they would use 1:1 the same LibStd implementation provided here, as they depend on the upstreaming of the Xtensa architecture support in LLVM; this upstreaming this is currently in progress.)
There was also a preceding discussion on the topic [here](https://github.com/espressif/rust-esp32-example/issues/14).
> Target names should not introduce undue confusion or ambiguity unless absolutely necessary to maintain ecosystem compatibility. For example, if the name of the target makes people extremely likely to form incorrect beliefs about what it targets, the name should be changed or augmented to disambiguate it.
We are explicitly putting an `-espidf` suffix to designate that the target is *specifically* for Rust + ESP-IDF
> Tier 3 targets may have unusual requirements to build or use, but must not create legal issues or impose onerous legal terms for the Rust project or for Rust developers or users.
Agreed.
> The target must not introduce license incompatibilities.
To the best of our knowledge, it doesn't.
> Anything added to the Rust repository must be under the standard Rust license (MIT OR Apache-2.0).
MIT + Apache 2.0
> The target must not cause the Rust tools or libraries built for any other host (even when supporting cross-compilation to the target) to depend on any new dependency less permissive than the Rust licensing policy. This applies whether the dependency is a Rust crate that would require adding new license exceptions (as specified by the tidy tool in the rust-lang/rust repository), or whether the dependency is a native library or binary. In other words, the introduction of the target must not cause a user installing or running a version of Rust or the Rust tools to be subject to any new license requirements.
Requirements are not changed for any other target.
> If the target supports building host tools (such as rustc or cargo), those host tools must not depend on proprietary (non-FOSS) libraries, other than ordinary runtime libraries supplied by the platform and commonly used by other binaries built for the target. For instance, rustc built for the target may depend on a common proprietary C runtime library or console output library, but must not depend on a proprietary code generation library or code optimization library. Rust's license permits such combinations, but the Rust project has no interest in maintaining such combinations within the scope of Rust itself, even at tier 3.
The targets are for bare-metal environment which is not hosting build tools or a compiler.
> Targets should not require proprietary (non-FOSS) components to link a functional binary or library.
The linker used by the targets is the GCC linker from the GCC toolchain cross-compiled for riscv. GNU GPL.
> "onerous" here is an intentionally subjective term. At a minimum, "onerous" legal/licensing terms include but are not limited to: non-disclosure requirements, non-compete requirements, contributor license agreements (CLAs) or equivalent, "non-commercial"/"research-only"/etc terms, requirements conditional on the employer or employment of any particular Rust developers, revocable terms, any requirements that create liability for the Rust project or its developers or users, or any requirements that adversely affect the livelihood or prospects of the Rust project or its developers or users.
> Neither this policy nor any decisions made regarding targets shall create any binding agreement or estoppel by any party. If any member of an approving Rust team serves as one of the maintainers of a target, or has any legal or employment requirement (explicit or implicit) that might affect their decisions regarding a target, they must recuse themselves from any approval decisions regarding the target's tier status, though they may otherwise participate in discussions.
> This requirement does not prevent part or all of this policy from being cited in an explicit contract or work agreement (e.g. to implement or maintain support for a target). This requirement exists to ensure that a developer or team responsible for reviewing and approving a target does not face any legal threats or obligations that would prevent them from freely exercising their judgment in such approval, even if such judgment involves subjective matters or goes beyond the letter of these requirements.
Agreed.
> Tier 3 targets should attempt to implement as much of the standard libraries as possible and appropriate (core for most targets, alloc for targets that can support dynamic memory allocation, std for targets with an operating system or equivalent layer of system-provided functionality), but may leave some code unimplemented (either unavailable or stubbed out as appropriate), whether because the target makes it impossible to implement or challenging to implement. The authors of pull requests are not obligated to avoid calling any portions of the standard library on the basis of a tier 3 target not implementing those portions.
The targets implement libStd almost in its entirety, except for the missing support for process, as this is a bare metal platform. The process `sys\unix` module is currently stubbed to return "not implemented" errors.
> The target must provide documentation for the Rust community explaining how to build for the target, using cross-compilation if possible. If the target supports running tests (even if they do not pass), the documentation must explain how to run tests for the target, using emulation if possible or dedicated hardware if necessary.
Target does not (yet) support running tests. We would gladly provide all documentation how to build for the target (where?). It is currently hosted in this [README.md](https://github.com/ivmarkov/rust-esp32-std-hello) file, but will likely be moved to the [esp-rs](https://github.com/esp-rs) organization. Since the build for the target is driven by cargo and [all other tooling is downloaded automatically during the build](https://github.com/esp-rs/esp-idf-sys/blob/master/build.rs), there is no need for extensive documentation.
> Tier 3 targets must not impose burden on the authors of pull requests, or other developers in the community, to maintain the target. In particular, do not post comments (automated or manual) on a PR that derail or suggest a block on the PR based on a tier 3 target. Do not send automated messages or notifications (via any medium, including via `@)` to a PR author or others involved with a PR regarding a tier 3 target, unless they have opted into such messages.
Agreed.
> Backlinks such as those generated by the issue/PR tracker when linking to an issue or PR are not considered a violation of this policy, within reason. However, such messages (even on a separate repository) must not generate notifications to anyone involved with a PR who has not requested such notifications.
Agreed.
> Patches adding or updating tier 3 targets must not break any existing tier 2 or tier 1 target, and must not knowingly break another tier 3 target without approval of either the compiler team or the maintainers of the other tier 3 target.
To the best of our knowledge, we believe we are not breaking any other target (be it tier 1, 2 or 3).
> In particular, this may come up when working on closely related targets, such as variations of the same architecture with different features. Avoid introducing unconditional uses of features that another variation of the target may not have; use conditional compilation or runtime detection, as appropriate, to let each target run code supported by that target.
To the best of our knowledge, we have not introduced any unconditional use of a feature that affects any other target.
> If a tier 3 target stops meeting these requirements, or the target maintainers no longer have interest or time, or the target shows no signs of activity and has not built for some time, or removing the target would improve the quality of the Rust codebase, we may post a PR to remove it; any such PR will be CCed to the target maintainers (and potentially other people who have previously worked on the target), to check potential interest in improving the situation.
Agreed.
Implement Extend<(A, B)> for (Extend<A>, Extend<B>)
I oriented myself at the implementation of `Iterator::unzip` and also rewrote the impl in terms of `(A, B)::extend` after that.
Since (A, B) now also implements Extend we could also mention in the documentation of unzip that it can do "nested unzipping" (you could unzip `Iterator<Item=(A, (B, C))>` into `(Vec<A>, (Vec<B>, Vec<C>))` for example) but I'm not sure of that so I'm asking here 🙂
(P.S. I saw a couple of people asking if there is an unzip3 but there isn't. So this could be a way to get equivalent functionality)
removed references to parent/child from std::thread documentation
- also clarifies how thread.join and detaching of threads works
- the previous prose implied that there is a relationship between a
spawning thread and the thread being spawned, and that "child" threads
couldn't outlive their "parents" unless detached, which is incorrect.
Added the `Option::unzip()` method
* Adds the `Option::unzip()` method to turn an `Option<(T, U)>` into `(Option<T>, Option<U>)` under the `unzip_option` feature
* Adds tests for both `Option::unzip()` and `Option::zip()`, I noticed that `.zip()` didn't have any
* Adds `#[inline]` to a few of `Option`'s methods that were missing it
Constify implementations of `(Try)From` for int types
I believe this to be one of the (many?) things blocking const (Range) iterators.
~~If this is to be merged maybe that should wait until `#![feature(const_trait_impl)]` no longer needs `#![allow(incomplete_features)]`?~~ - Done
Replace read_to_string with read_line in Stdin example
The current example results in infinitely reading from stdin, which can confuse newcomers trying to read from stdin.
(`@razmag` encountered this while learning the language from the docs)
While going through the "The Rust Programming Language" book (Klabnik & Nichols), the TCP server example directs us to use TcpListener::incoming. I was curious how I could pass this value to a function (before reading ahead in the book), so I looked up the docs to determine the signature.
When I opened the docs, I found https://doc.rust-lang.org/std/net/struct.TcpListener.html#method.incoming, which didn't mention TcpStream anywhere in the example.
Eventually, I clicked on https://doc.rust-lang.org/std/net/struct.TcpListener.html#method.accept in the docs (after clicking a few other locations first), and was able to surmise that the value contained TcpStream.
## Opportunity
While this type is mentioned several times in this doc, I feel that someone should be able to fully use the results of the TcpListner::incoming iterator based solely on the docs of just this method.
## Implementation
I took the code from the top-level TcpListener https://doc.rust-lang.org/std/net/struct.TcpListener.html#method.incoming and blended it with the existing docs for TcpListener::incoming https://doc.rust-lang.org/std/net/struct.TcpListener.html#method.incoming.
It does make the example a little longer, and it also introduces a little duplication. It also gives the reader the type signatures they need to move on to the next step.
## Additional considerations
I noticed that in this doc, `handle_connection` and `handle_client` are both used to accept a TcpStream in the docs on this page. I want to standardize on one function name convention, so readers don't accidentally think two different concepts are being referenced. I didn't want to cram do too much in one PR, I can update this PR to make that change, or I could send another PR (if you would like).
First attempted contribution to Rust (and I'm also still very new, hence reading through the rust book for the first time)! Would you please let me know what you think?
Stabilize Vec<T>::shrink_to
This PR stabilizes `shrink_to` feature and closes the corresponding issue. The second point was addressed already, and no `panic!` should occur.
Closes#56431.
Avoid using the `copy_nonoverlapping` wrapper through `mem::replace`.
This is a much simpler way to achieve the pre-#86003 behavior of `mem::replace` not needing dynamically-sized `memcpy`s (at least before inlining), than re-doing #81238 (which needs #86699 or something similar).
I didn't notice it until recently, but `ptr::write` already explicitly avoided using the wrapper, while `ptr::read` just called the wrapper (and was the reason for us observing any behavior change from #86003 in Rust-GPU).
<hr/>
The codegen test I've added fails without the change to `core::ptr::read` like this (ignore the `v0` mangling, I was using a worktree with it turned on by default, for this):
```llvm
13: ; core::intrinsics::copy_nonoverlapping::<u8>
14: ; Function Attrs: inlinehint nonlazybind uwtable
15: define internal void `@_RINvNtCscK5tvALCJol_4core10intrinsics19copy_nonoverlappinghECsaS4X3EinRE8_25mem_replace_direct_memcpy(i8*` %src, i8* %dst, i64 %count) unnamed_addr #0 {
16: start:
17: %0 = mul i64 %count, 1
18: call void `@llvm.memcpy.p0i8.p0i8.i64(i8*` align 1 %dst, i8* align 1 %src, i64 %0, i1 false)
not:17 !~~~~~~~~~~~~~~~~~~~~~ error: no match expected
19: ret void
20: }
```
With the `core::ptr::read` change, `core::intrinsics::copy_nonoverlapping` doesn't get instantiated and the test passes.
<hr/>
r? `@m-ou-se` cc `@nagisa` (codegen test) `@oli-obk` / `@RalfJung` (miri diagnostic changes)
Change proc_macro::Diagnostics docs
With Rust 1.54 attributes can invoke function-like macros. This allows functions generated by macros to have non-generic documentation. This pull request changes the documentation of `proc_macro::Diagnostics` to be more specific and include a link to the level.
## Example
Before:
```markdown
Adds a new child diagnostic message to `self` with the level
identified by this method’s name with the given `message`.
```
After:
```markdown
Adds a new child diagnostic message to self with the [`Level::Error`]
level, and the given `message`.
```
impl Default, Copy, Clone for std::io::Sink and Empty
The omission of `Sink: Default` is causing me a slight inconvenience in a test harness. There seems little reason for this and `Empty` not to be `Clone` and `Copy` too.
I have made all three of these insta-stable, because:
AIUI `Copy` can only be derived, and I was not able to find any examples of how to unstably derive it. I think it is probably not possible.
I hunted through the git history for precedent and found
> 79b8ad84c8
> Implement `Copy` for `IoSlice`
> https://github.com/rust-lang/rust/pull/69403
which was also insta-stable.
Fix intra doc link in hidden doc of Iterator::__iterator_get_unchecked
Recently, I edited the import list of the `core::iter::traits::iterator` module (in #85874). This results in a broken intra doc link in a hidden documentation with the effect that `RUSTDOCFLAGS='--document-private-items --document-hidden-items' x doc library/std` fails. (This can be worked around by adding `-Arustdoc::broken-intra-doc-links`; still, it’s a broken link so let’s fix it.)
``@rustbot`` label C-cleanup, T-libs
Document that fs::read_dir skips . and ..
Hi,
I think this is worth noting in the docs since it differs from POSIX `readdir`. I didn’t put it under platform-specific notes because it seems to be consistent across platforms, and changing this behavior in the future could cause pretty nasty bugs.
Thanks!
- also clarifies how thread.join and detaching of threads works
- the previous prose implied that there is a relationship between a
spawning thread and the thread being spawned, and that "child" threads
couldn't outlive their parents unless detached, which is incorrect.
Hide allocator details from TryReserveError
I think there's [no need for TryReserveError to carry detailed information](https://github.com/rust-lang/rust/issues/48043#issuecomment-825139280), but I wouldn't want that issue to delay stabilization of the `try_reserve` feature.
So I'm proposing to stabilize `try_reserve` with a `TryReserveError` as an opaque structure, and if needed, expose error details later.
This PR moves the `enum` to an unstable inner `TryReserveErrorKind` that lives under a separate feature flag. `TryReserveErrorKind` could possibly be left as an implementation detail forever, and the `TryReserveError` get methods such as `allocation_size() -> Option<usize>` or `layout() -> Option<Layout>` instead, or the details could be dropped completely to make try-reserve errors just a unit struct, and thus smaller and cheaper.
Rollup of 9 pull requests
Successful merges:
- #87561 (thread set_name haiku implementation.)
- #87715 (Add long error explanation for E0625)
- #87727 (explicit_generic_args_with_impl_trait: fix min expected number of generics)
- #87742 (Validate FFI-safety warnings on naked functions)
- #87756 (Add back -Zno-profiler-runtime)
- #87759 (Re-use std::sealed::Sealed in os/linux/process.)
- #87760 (Promote `aarch64-apple-ios-sim` to Tier 2)
- #87770 (permit drop impls with generic constants in where clauses)
- #87780 (alloc: Use intra doc links for the reserve function)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
alloc: Use intra doc links for the reserve function
The sentence exists to highlight the existence of a
performance footgun of repeated calls of the
reserve_exact function.
Re-use std::sealed::Sealed in os/linux/process.
This uses `std::sealed::Sealed` in `std::os::linux::process` instead of defining new `Sealed` traits there.
Currently there is no API that allows fallible zero-allocation of a Vec.
Vec.try_reserve is not appropriate for this job since it doesn't know
whether it should zero or arbitrary uninitialized memory is fine.
Since Box currently holds most of the zeroing/uninit/slice allocation APIs
it's the best place to add yet another entry into this feature matrix.
rustc: Fill out remaining parts of C-unwind ABI
This commit intends to fill out some of the remaining pieces of the
C-unwind ABI. This has a number of other changes with it though to move
this design space forward a bit. Notably contained within here is:
* On `panic=unwind`, the `extern "C"` ABI is now considered as "may
unwind". This fixes a longstanding soundness issue where if you
`panic!()` in an `extern "C"` function defined in Rust that's actually
UB because the LLVM representation for the function has the `nounwind`
attribute, but then you unwind.
* Whether or not a function unwinds now mainly considers the ABI of the
function instead of first checking the panic strategy. This fixes a
miscompile of `extern "C-unwind"` with `panic=abort` because that ABI
can still unwind.
* The aborting stub for non-unwinding ABIs with `panic=unwind` has been
reimplemented. Previously this was done as a small tweak during MIR
generation, but this has been moved to a separate and dedicated MIR
pass. This new pass will, for appropriate functions and function
calls, insert a `cleanup` landing pad for any function call that may
unwind within a function that is itself not allowed to unwind. Note
that this subtly changes some behavior from before where previously on
an unwind which was caught-to-abort it would run active destructors in
the function, and now it simply immediately aborts the process.
* The `#[unwind]` attribute has been removed and all users in tests and
such are now using `C-unwind` and `#![feature(c_unwind)]`.
I think this is largely the last piece of the RFC to implement.
Unfortunately I believe this is still not stabilizable as-is because
activating the feature gate changes the behavior of the existing `extern
"C"` ABI in a way that has no replacement. My thinking for how to enable
this is that we add support for the `C-unwind` ABI on stable Rust first,
and then after it hits stable we change the behavior of the `C` ABI.
That way anyone straddling stable/beta/nightly can switch to `C-unwind`
safely.
#[inline] slice::Iter::advance_by
https://github.com/rust-lang/rust/pull/87387#issuecomment-891942661 was marked as a regression. One of the methods in the PR was missing an inline annotation unlike all the other methods on slice iterators.
Let's see if that makes a difference.
Make wrapping_neg() use wrapping_sub(), #[inline(always)]
This is a follow-up change to the fix for #75598. It simplifies the implementation of wrapping_neg() for all integer types by just calling 0.wrapping_sub(self) and always inlines it. This leads to much less assembly code being emitted for opt-level≤1 and thus much better performance for debug-compiled code.
Background is [this discussion on the internals forum](https://internals.rust-lang.org/t/why-does-rust-generate-10x-as-much-unoptimized-assembly-as-gcc/14930).
Remove the aarch64 `crypto` target_feature
The subfeatures `aes` or `sha2` should be used instead.
This can't yet be done for ARM targets as some LLVM intrinsics still require `crypto`.
Also update the runtime feature detection tests in `library/std` to mirror the updates in `stdarch`. This also helps https://github.com/rust-lang/rust/issues/86941
r? ``@Amanieu``
Remove space after negative sign in Literal to_string
Negative proc macro literal tokens used to be printed with a space between the minus sign and the magnitude. That's because `impl ToString for Literal` used to convert the Literal into a TokenStream, which splits the minus sign into a separate Punct token.
```rust
Literal::isize_unsuffixed(-10).to_string() // "- 10"
```
This PR updates the ToString impl to directly use `rustc_ast::token::Lit`'s ToString, which matches the way Rust negative numbers are idiomatically written without a space.
```rust
Literal::isize_unsuffixed(-10).to_string() // "-10"
```
Add `core::stream::from_iter`
_Tracking issue: https://github.com/rust-lang/rust/issues/81798_
This_ PR implements `std::stream::from_iter`, as outlined in the _"Converting an Iterator to a Stream"_ section of the [Stream RFC](https://github.com/nellshamrell/rfcs/blob/add-async-stream-rfc/text/0000-async-stream.md#converting-an-iterator-to-a-stream). This function enables converting an `Iterator` to a `Stream` by wrapping each item in the iterator with a `Poll::Ready` instance.
r? `@tmandry`
cc/ `@rust-lang/libs` `@rust-lang/wg-async-foundations`
## Example
Being able to convert from an iterator into a stream is useful when refactoring from iterative loops into a more functional adapter-based style. This is fairly common when using more complex `filter` / `map` / `find` chains. In its basic form this conversion looks like this:
**before**
```rust
let mut output = vec![];
for item in my_vec {
let out = do_io(item).await?;
output.push(out);
}
```
**after**
```rust
use std::stream;
let output = stream::from_iter(my_vec.iter())
.map(async |item| do_io(item).await)
.collect()?;
```
Having a way to convert an `Iterator` to a `Stream` is essential in enabling this flow.
## Implementation Notes
This PR makes use of `unsafe {}` to pin an item. Currently we're having conversations on the libs stream in Zulip how to bring `pin-project` in as a dependency to `core` so we can omit the `unsafe {}`.
This PR also includes a documentation block which references `Stream::next` which currently doesn't exist in the stdlib (originally included in the RFC and PR, but later omitted because of an unresolved issue). `stream::from_iter` can't stabilize before `Stream` does, and there's still a chance we may stabilize `Stream` with a `next` method. So this PR includes documentation referencing that method, which we can remove as part of stabilization if by any chance we don't have `Stream::next`.
## Alternatives Considered
### `impl IntoStream for T: IntoIterator`
An obvious question would be whether we could make it so every iterator can automatically be converted into a stream by calling `into_stream` on it. The answer is: "perhaps, but it could cause type issues". Types like `std::collections` may want to opt to create manual implementations for `IntoStream` and `IntoIter`, which wouldn't be possible if it was implemented through a catch-all trait.
Possibly an alternative such as `impl IntoStream for T: Iterator` could work, but it feels somewhat restrictive. In the end, converting an iterator to a stream is likely to be a bit of a niche case. And even then, **adding a standalone function to convert an `Iterator` into a `Stream` would not be mutually exclusive with a blanket implementation**.
### Naming
The exact name can be debated in the period before stabilization. But I've chosen `stream::from_iter` rather than `stream::iter` because we are _creating a stream from an iterator_ rather than _iterating a stream_. We also expect to add a stream counterpart to `iter::from_fn` later on (blocked on async closures), and having `stream::from_fn` and `stream::from_iter` would feel like a consistent pair. It also has prior art in `async_std::stream::from_iter`.
## Future Directions
### Stream conversions for collections
This is a building block towards implementing `stream/stream_mut/into_stream` methods for `std::collections`, `std::vec`, and more. This would allow even quicker refactorings from using loops to using iterator adapters by omitting the import altogether:
**before**
```rust
use std::stream;
let output = stream::from_iter(my_vec.iter())
.map(async |item| do_io(item).await)
.collect()?;
```
**after**
```rust
let output = my_vec
.stream()
.map(async |item| do_io(item).await)
.collect()?;
```
This commit intends to fill out some of the remaining pieces of the
C-unwind ABI. This has a number of other changes with it though to move
this design space forward a bit. Notably contained within here is:
* On `panic=unwind`, the `extern "C"` ABI is now considered as "may
unwind". This fixes a longstanding soundness issue where if you
`panic!()` in an `extern "C"` function defined in Rust that's actually
UB because the LLVM representation for the function has the `nounwind`
attribute, but then you unwind.
* Whether or not a function unwinds now mainly considers the ABI of the
function instead of first checking the panic strategy. This fixes a
miscompile of `extern "C-unwind"` with `panic=abort` because that ABI
can still unwind.
* The aborting stub for non-unwinding ABIs with `panic=unwind` has been
reimplemented. Previously this was done as a small tweak during MIR
generation, but this has been moved to a separate and dedicated MIR
pass. This new pass will, for appropriate functions and function
calls, insert a `cleanup` landing pad for any function call that may
unwind within a function that is itself not allowed to unwind. Note
that this subtly changes some behavior from before where previously on
an unwind which was caught-to-abort it would run active destructors in
the function, and now it simply immediately aborts the process.
* The `#[unwind]` attribute has been removed and all users in tests and
such are now using `C-unwind` and `#![feature(c_unwind)]`.
I think this is largely the last piece of the RFC to implement.
Unfortunately I believe this is still not stabilizable as-is because
activating the feature gate changes the behavior of the existing `extern
"C"` ABI in a way that has no replacement. My thinking for how to enable
this is that we add support for the `C-unwind` ABI on stable Rust first,
and then after it hits stable we change the behavior of the `C` ABI.
That way anyone straddling stable/beta/nightly can switch to `C-unwind`
safely.
Add convenience method for handling ipv4-mapped addresses by canonicalizing them
This simplifies checking common properties in an address-family-agnostic
way since #86335 commits to not checking IPv4 semantics
of IPv4-mapped addresses in the `Ipv6Addr` property methods.
Commit to not supporting IPv4-in-IPv6 addresses
Stabilization of the `ip` feature has for a long time been blocked on the question of whether Rust should support handling "IPv4-in-IPv6" addresses: should the various `Ipv6Address` property methods take IPv4-mapped or IPv4-compatible addresses into account. See also the IPv4-in-IPv6 Address Support issue #85609 and #69772 which originally asked the question.
# Overview
In the recent PR #85655 I proposed changing `is_loopback` to take IPv4-mapped addresses into account, so `::ffff:127.0.0.1` would be recognized as a looback address. However, due to the points that came up in that PR, I alternatively propose the following: Keeping the current behaviour and commit to not assigning any special meaning for IPv4-in-IPv6 addresses, other than what the standards prescribe. This would apply to the stable method `is_loopback`, but also to currently unstable methods like `is_global` and `is_documentation` and any future methods. This is implemented in this PR as a change in documentation, specifically the following section:
> Both types of addresses are not assigned any special meaning by this implementation, other than what the relevant standards prescribe. This means that an address like `::ffff:127.0.0.1`, while representing an IPv4 loopback address, is not itself an IPv6 loopback address; only `::1` is. To handle these so called "IPv4-in-IPv6" addresses, they have to first be converted to their canonical IPv4 address.
# Discussion
In the discussion for or against supporting IPv4-in-IPv6 addresses the question what would be least surprising for users of other languages has come up several times. At first it seemed most big other languages supported IPv4-in-IPv6 addresses (or at least considered `::ffff:127.0.0.1` a loopback address). However after further investigation it appears that supporting IPv4-in-IPv6 addresses comes down to how a language represents addresses. .Net and Go do not have a separate type for IPv4 or IPv6 addresses, and do consider `::ffff:127.0.0.1` a loopback address. Java and Python, which do have separate types, do not consider `::ffff:127.0.0.1` a loopback address. Seeing as Rust has the separate `Ipv6Addr` type, it would make sense to also not support IPv4-in-IPv6 addresses. Note that this focuses on IPv4-mapped addresses, no other language handles IPv4-compatible addresses.
Another issue that was raised is how useful supporting these IPv4-in-IPv6 addresses would be in practice. Again with the example of `::ffff:127.0.0.1`, considering it a loopback address isn't too useful as to use it with most of the socket APIs it has to be converted to an IPv4 address anyway. From that perspective it would be better to instead provide better ways for doing this conversion like stabilizing `to_ipv4_mapped` or introducing a `to_canonical` method.
A point in favour of not supporting IPv4-in-IPv6 addresses is that that is the behaviour Rust has always had, and that supporting it would require changing already stable functions like `is_loopback`. This also keeps the documentation of these functions simpler, as we only have to refer to the relevant definitions in the IPv6 specification.
# Decision
To make progress on the `ip` feature, a decision needs to be made on whether or not to support IPv4-in-IPv6 addresses.
There are several options:
- Keep the current implementation and commit to never supporting IPv4-in-IPv6 addresses (accept this PR).
- Support IPv4-in-IPv6 addresses in some/all `IPv6Addr` methods (accept PR #85655).
- Keep the current implementation and but not commit to anything yet (reject both this PR and PR #85655), this entire issue will however come up again in the stabilization of several methods under the `ip` feature.
There are more options, like supporting IPv4-in-IPv6 addresses in `IpAddr` methods instead, but to my knowledge those haven't been seriously argued for by anyone.
There is currently an FCP ongoing on PR #85655. I would ask the libs team for an alternative FCP on this PR as well, which if completed means the rejection of PR #85655, and the decision to commit to not supporting IPv4-in-IPv6 addresses.
If anyone feels there is not enough evidence yet to make the decision for or against supporting IPv4-in-IPv6 addresses, let me know and I'll do whatever I can to resolve it.
This simplifies checking common properties in an address-family-agnostic
way since since #86335 commits to not checking IPv4 semantics
of IPv4-mapped addresses in the `Ipv6Addr` property methods.
Add missing "allocated object" doc link to `<*mut T>::add`
The portion of the documentation expecting the link was already there, but it was rendered as "[allocated object]". The added reference is just copied from the documentation for `<*const T>::add`.
rfc3052 followup: Remove authors field from Cargo manifests
Since RFC 3052 soft deprecated the authors field, hiding it from
crates.io, docs.rs, and making Cargo not add it by default, and it is
not generally up to date/useful information for contributors, we may as well
remove it from crates in this repo.
Recommend `swap_remove` in `Vec::remove` docs
I was able to increase the performance (by 20%!) of my project by changing a `Vec::remove` call to `Vec::swap_remove` in a hot function. I think we should explicitly put a note in the Vec::remove docs to guide people in the right direction so they don't make a similar oversight.
Add documentation for `Ipv6MulticastScope`
Adds basic documentation to the unstable `Ipv6MulticastScope`, as well as marking it `#[non_exhaustive]` because future IETF RFCs may introduce additional scopes. The documentation mentions this in a section "Stability Guarantees":
> /// Not all possible values for a multicast scope have been assigned.
/// Future RFCs may introduce new scopes, which will be added as variants to this enum;
/// because of this the enum is marked as `#[non_exhaustive]`.
Partially stabilize `const_slice_first_last`
This stabilizes the non-`mut` methods of `const_slice_first_last` as `const`. These methods are trivial to implement and have no blockers that I am aware of.
`@rustbot` label +A-const-fn +S-waiting-on-review +T-libs-api
Move `os_str_bytes` to `sys::unix`
Followup to #84967, with `OsStrExt` and `OsStringExt` moved out of `sys_common`, there is no reason anymore for `os_str_bytes` to live in `sys_common` and not in sys. This pr moves it to the location `sys::unix::os_str` and reuses the code on other platforms via `#[path]` (as is common in `sys`) instead of importing.
Remove `Ipv4Addr::is_ietf_protocol_assignment`
This PR removes the unstable method `Ipv4Addr::is_ietf_protocol_assignment`, as I suggested in https://github.com/rust-lang/rust/issues/85612#issuecomment-847863404. The method was added in #60145, as far as I can tell primarily for the implementation of `Ipv4Addr::is_global` (addresses reserved for IETF protocol assignment are not globally reachable unless otherwise specified).
The method was added in 2019, but I haven't been able to find any open-source code using this method so far. I'm also having a hard time coming up with a usecase for specifically this method; knowing that an address is reserved for future protocols doesn't allow you to do much with it, especially since now some of those addresses are indeed assigned to a protocol and have their own behaviour (and might even be defined to be globally reachable, so if that is what you care about it is always more accurate to call `!is_global()`, instead of `is_ietf_protocol_assignment()`).
Because of these reasons, I propose removing the method (or alternatively make it a private helper for `is_global`) and also not introduce `Ipv6Addr::is_ietf_protocol_assignment` and `IpAddr::is_ietf_protocol_assignment` in the future.
Change environment variable getters to error recoverably
This PR changes the standard library environment variable getter functions to error recoverably (i.e. not panic) when given an invalid value.
On some platforms, it is invalid for environment variable names to contain `'\0'` or `'='`, or for their values to contain `'\0'`. Currently, the standard library panics when manipulating environment variables with names or values that violate these invariants. However, this behavior doesn't make a lot of sense, at least in the case of getters. If the environment variable is missing, the standard library just returns an error value, rather than panicking. It doesn't make sense to treat the case where the variable is invalid any differently from that. See the [internals thread](https://internals.rust-lang.org/t/why-should-std-var-panic/14847) for discussion. Thus, this PR changes the functions to error recoverably in this case as well.
If desired, I could change the functions that manipulate environment variables in other ways as well. I didn't do that here because it wasn't entirely clear what to change them to. Should they error silently or do something else? If someone tells me how to change them, I'm happy to implement the changes.
This fixes#86082, an ICE that arises from the current behavior. It also adds a regression test to make sure the ICE does not occur again in the future.
`@rustbot` label +T-libs
r? `@joshtriplett`
BTree: lazily locate leaves in rangeless iterators
BTree iterators always locate both the first and last leaf edge and often only need either one, i.e., whenever they are traversed in a single direction, like in for-loops and in the common use of `iter().next()` or `iter().next_back()` to retrieve the first or last key/value-pair (#62924). It's fairly easy to avoid because the iterators with this disadvantage already are quite separate from other iterators.
r? `@Mark-Simulacrum`
Bump bootstrap compiler to 1.55
Changing the cfgs for stdarch is missing, but my understanding is that we don't need to do it as part of this PR?
r? `@Mark-Simulacrum`
Add Linux-specific pidfd process extensions (take 2)
Continuation of #77168.
I addressed the following concerns from the original PR:
- make `CommandExt` and `ChildExt` sealed traits
- wrap file descriptors in `PidFd` struct representing ownership over the fd
- add `take_pidfd` to take the fd out of `Child`
- close fd when dropped
Tracking Issue: #82971
Move UnwindSafe, RefUnwindSafe, AssertUnwindSafe to core
They were previously only available in std::panic, not core::panic.
- https://doc.rust-lang.org/1.51.0/std/panic/trait.UnwindSafe.html
- https://doc.rust-lang.org/1.51.0/std/panic/trait.RefUnwindSafe.html
- https://doc.rust-lang.org/1.51.0/std/panic/struct.AssertUnwindSafe.html
Where this is relevant: trait objects! Inside a `#![no_std]` library it's otherwise impossible to have a struct holding a trait object, and at the same time can be used from downstream std crates in a way that doesn't interfere with catch_unwind.
```rust
// common library
#![no_std]
pub struct Thing {
pub(crate) x: &'static (dyn SomeTrait + Send + Sync),
}
pub(crate) trait SomeTrait {...}
```
```rust
// downstream application
fn main() {
let thing: library::Thing = ...;
let _ = std::panic::catch_unwind(|| { let _ = thing; }); // does not work :(
}
```
See a4131708e2/src/gradient.rs (L7-L15) for a real life example of needing to work around this problem. In particular that workaround would not even be viable if implementors of the trait were provided externally by a caller, as the `feature = "std"` would become non-additive in that case.
What happens without the UnwindSafe constraints:
```rust
fn main() {
let gradient = colorous::VIRIDIS;
let _ = std::panic::catch_unwind(|| { let _ = gradient; });
}
```
```console
error[E0277]: the type `(dyn colorous::gradient::EvalGradient + Send + Sync + 'static)` may contain interior mutability and a reference may not be safely transferrable across a catch_unwind boundary
--> src/main.rs:3:13
|
3 | let _ = std::panic::catch_unwind(|| { let _ = gradient; });
| ^^^^^^^^^^^^^^^^^^^^^^^^ `(dyn colorous::gradient::EvalGradient + Send + Sync + 'static)` may contain interior mutability and a reference may not be safely transferrable across a catch_unwind boundary
|
::: .rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panic.rs:430:40
|
430 | pub fn catch_unwind<F: FnOnce() -> R + UnwindSafe, R>(f: F) -> Result<R> {
| ---------- required by this bound in `catch_unwind`
|
= help: within `Gradient`, the trait `RefUnwindSafe` is not implemented for `(dyn colorous::gradient::EvalGradient + Send + Sync + 'static)`
= note: required because it appears within the type `&'static (dyn colorous::gradient::EvalGradient + Send + Sync + 'static)`
= note: required because it appears within the type `Gradient`
= note: required because of the requirements on the impl of `UnwindSafe` for `&Gradient`
= note: required because it appears within the type `[closure@src/main.rs:3:38: 3:62]`
```
Implement advance_by, advance_back_by for slice::{Iter, IterMut}
Part of #77404.
Picking up where #77633 was closed.
I have addressed https://github.com/rust-lang/rust/pull/77633#issuecomment-771842599 by restoring `nth` and `nth_back`. So according to that comment this should already be r=m-ou-se, but it has been sitting for a while.
Track caller of Vec::remove()
`vec.remove(invalid)` doesn't print a helpful source position:
> thread 'main' panicked at 'removal index (is 99) should be < len (is 1)', **library/alloc/src/vec/mod.rs:1379:13**
Add docs about performance and `Iterator::map` to `[T; N]::map`
This suboptimal code gen for some usages of array::map got a bit of
attention by multiple people throughout the community. Some cases:
- https://github.com/rust-lang/rust/issues/75243#issuecomment-866051086
- https://github.com/rust-lang/rust/issues/75243#issuecomment-874732134
- https://www.reddit.com/r/rust/comments/oeqqf7/unexpected_high_stack_usage/
My *guess* is that this gets the attention it gets because in JavaScript
(and potentially other languages), a `map` function on arrays is very
commonly used since in those languages, arrays basically take the role
of Rust's iterator. I considered explicitly naming JavaScript in the
first paragraph I added, but I couldn't find precedence of mentioning
other languages in standard library doc, so I didn't add it.
When array::map was stabilized, we still wanted to add docs, but that
somehow did not happen in time. So here we are. Not sure if this sounds
crazy but maybe it is worth considering beta backporting this? Only if
it's not a lot of work, of course! But yeah, stabilized array::map is
already in beta and if this problem is really as big as it sometimes seems,
might be worth having the docs in place when 1.55 is released.
CC ``@CryZe``
r? ``@m-ou-se`` (since you were involved in that discussion and the stabilization)
Make `SEMICOLON_IN_EXPRESSIONS_FROM_MACROS` warn by default
This PR makes the `SEMICOLON_IN_EXPRESSIONS_FROM_MACROS` lint warn by default.
To avoid showing a large number of un-actionable warnings to users, we only enable the lint for macros defined in the same crate. This ensures that users will be able to fix the warning by simply removing a semicolon.
In the future, I'd like to enable this lint unconditionally, and eventually make it into a hard error in a future edition. This PR is a step towards that goal.
[backtraces]: look for the `begin` symbol only after seeing `end`
On `x86_64-pc-windows-msvc`, we often get backtraces which look like
this:
```
10: 0x7ff77e0e9be5 - std::panicking::rust_panic_with_hook
11: 0x7ff77e0e11b4 - std::sys_common::backtrace::__rust_begin_short_backtrace::h5769736bdb11136c
12: 0x7ff77e0e116f - std::sys_common::backtrace::__rust_end_short_backtrace::h61c7ecb1b55338ae
13: 0x7ff77e0f89dd - std::panicking::begin_panic::h8e60ef9f82a41805
14: 0x7ff77e0e108c - d
15: 0x7ff77e0e1069 - c
16: 0x7ff77e0e1059 - b
17: 0x7ff77e0e1049 - a
18: 0x7ff77e0e1039 - core::ptr::drop_in_place<std::rt::lang_start<()>::{{closure}}>::h1bfcd14d5e15ba81
19: 0x7ff77e0e1186 - std::sys_common::backtrace::__rust_begin_short_backtrace::h5769736bdb11136c
20: 0x7ff77e0e100c - std::rt::lang_start::{{closure}}::ha054184bbf9921e3
```
Notice that `__rust_begin_short_backtrace` appears on frame 11 before
`__rust_end_short_backtrace` on frame 12. This is because in typical
release binaries without debug symbols, dbghelp.dll, which we use to walk
and symbolize the stack, does not know where CGU internal functions
start or end and so the closure invoked by `__rust_end_short_backtrace`
is incorrectly described as `__rust_begin_short_backtrace` because it
happens to be near that symbol.
While that can obviously change, this has been happening quite
consistently since #75048. Since this is a very small change to the std
and the change makes sense by itself, I think this is worth doing.
This doesn't completely resolve the situation for release binaries on
Windows, since without debug symbols, the stack printed can still show
incorrect symbol names (this is why the test uses `#[no_mangle]`) but it
does slightly improve the situation in that you see the same backtrace
you would see with `RUST_BACKTRACE=full` or in a debugger (without the
uninteresting bits at the top and bottom).
Fixes part of #87481
