Support static linking with glibc and target-feature=+crt-static
With this change, it's possible to build on a linux-gnu target and pass
RUSTFLAGS='-C target-feature=+crt-static' or the equivalent via a
`.cargo/config.toml` file, and get a statically linked executable.
Update to libc 0.2.78, which adds support for static linking with glibc.
Add `crt_static_respected` to the `linux_base` target spec.
Update `android_base` and `linux_musl_base` accordingly. Avoid enabling
crt_static_respected on Android platforms, since that hasn't been
tested.
Closes https://github.com/rust-lang/rust/issues/65447.
Implement advance_by, advance_back_by for iter::Chain
Part of #77404.
This PR does two things:
- implement `Chain::advance[_back]_by` in terms of `advance[_back]_by` on `self.a` and `advance[_back]_by` on `self.b`
- change `Chain::nth[_back]` to use `advance[_back]_by` on `self.a` and `nth[_back]` on `self.b`
This ensures that `Chain::nth` can take advantage of an efficient `nth` implementation on the second iterator, in case it doesn't implement `advance_by`.
cc `@scottmcm` in case you want to review this
Replace some once(x).chain(once(y)) with [x, y] IntoIter
Now that we have by-value array iterators that are [already used](25c8c53dd9/compiler/rustc_hir/src/def.rs (L305-L307))...
For example,
```diff
- once(self.type_ns).chain(once(self.value_ns)).chain(once(self.macro_ns)).filter_map(|it| it)
+ IntoIter::new([self.type_ns, self.value_ns, self.macro_ns]).filter_map(|it| it)
```
BTreeMap: admit the existence of leaf edges in comments
The btree code is ambiguous about leaf edges (i.e., edges within leaf nodes). Iteration relies on them heavily, but some of the comments suggest there are no leaf edges (extracted from #77025)
r? @Mark-Simulacrum
Use more intra-doc-links in `core::fmt`
This is a follow-up to #75819, which encountered some broken links due to #75176, so this PR contains the links that are blocked on #75176.
r? @jyn514
Allows getting the slices directly, rather than just through an iterator as in `array_chunks(_mut)`. The constructors for those iterators are then written in terms of these methods, so the iterator constructors no longer have any `unsafe` of their own.
Hint the maximum length permitted by invariant of slices
One of the safety invariants of references, and in particular of references to slices, is that they may not cover more than `isize::MAX` bytes. The unsafe `from_raw_parts` constructors of slices explicitly requires the caller to guarantee this fact. Violating it would also be UB with regards to the semantics of generated llvm code.
This effectively bounds the length of a (non-ZST) slice from above by a compile time constant. But when the length is loaded from a function argument it appears llvm is not aware of this requirement. The additional value range assertions allow some further elision of code branches, including overflow checks, especially in the presence of artithmetic on the indices.
This may have a performance impact, adding more code to a common method but allowing more optimization. I'm not quite sure, is the Rust side of const-prop strong enough to elide the irrelevant match branches?
Fixes: #67186
Uses assume to check the length against a constant upper bound. The
inlined result then informs the optimizer of the sound value range.
This was tried with unreachable_unchecked before which introduces a
branch. This has the advantage of not being executed in sound code but
complicates basic blocks. It resulted in ~2% increased compile time in
some worst cases.
Add a codegen test for the assumption, testing the issue from #67186
Rollup of 8 pull requests
Successful merges:
- #77072 (Minor `hash_map` doc adjustments + item attribute orderings)
- #77368 (Backport LLVM apfloat commit to rustc_apfloat)
- #77445 (BTreeMap: complete the compile-time test_variance test case)
- #77504 (Support vectors with fewer than 8 elements for simd_select_bitmask)
- #77513 (Change DocFragments from enum variant fields to structs with a nested enum)
- #77518 (Only use Fira Sans for the first `td` in item lists)
- #77521 (Move target feature whitelist from cg_llvm to cg_ssa)
- #77525 (Enable RenameReturnPlace MIR optimization on mir-opt-level >= 2)
Failed merges:
r? `@ghost`
BTreeMap: complete the compile-time test_variance test case
Some of the items added to the new `test_sync` belonged in the old `test_variance` as well. And fixed inconsistent paths to nearby modules.
r? @Mark-Simulacrum
Minor `hash_map` doc adjustments + item attribute orderings
This PR is really a couple visual changes glued together:
1. Some of the doc comments for items in `std::collections::hash_map` referenced the names of types without escaping their formatting (e.g. using "VacantEntry" instead of "`VacantEntry`") - the ones I could find were changed to the latter
2. The vast majority of pre-item attributes seem to place doc comments as the first attribute (instead of things like `#[feature(...)]`), so the few that had the other order were changed.
3. Also ordering related: the general trend seems to be that `#[feature]` attributes follow `#[inline]`, so I swapped the two lines in places where that ordering was reversed. This is primarily a change based on stylistic continuity and aesthetics - I'm not sure how important that actually is / should be.
I figured this would be pretty uncontroversial, but some of these might have been intentional for reasons I don't know about - if so, I'd be happy to remove the relevant changes. Of these, the final set of changes is probably the most unnecessary, so it also might be better to leave those out (in favor of reducing code churn).
Implement as_ne_bytes() for integers and floats
This is related to issue #64464.
I am pretty sure that these functions are actually const-ify-able, and technically as_bits() can also be implemented for floats, but I might need some comments on both.
Implement Make `handle_alloc_error` default to panic (for no_std + liballoc)
Related: https://github.com/rust-lang/rust/issues/66741
Guarded with `#![feature(default_alloc_error_handler)]` a default
`alloc_error_handler` is called, if a custom allocator is used and no
other custom `#[alloc_error_handler]` is defined.
Unbox mutexes and condvars on some platforms
Both mutexes and condition variables contained a Box containing the actual os-specific object. This was done because moving these objects may cause undefined behaviour on some platforms.
However, this is not needed on Windows[1], Wasm[2], cloudabi[2], and 'unsupported'[3], were the box was only needlessly making them less efficient.
This change gets rid of the box on those platforms.
On those platforms, `Condvar` can no longer verify it is only used with one `Mutex`, as mutexes no longer have a stable address. This was addressed and considered acceptable in #76932.
[1]\: https://docs.microsoft.com/en-us/windows/win32/api/synchapi/nf-synchapi-initializesrwlock
[2]\: These are just a single atomic integer together with futex wait/wake calls/instructions.
[3]\: The `unsupported` platform doesn't support multiple threads at all.
Use less divisions in display u128/i128
This PR is an absolute mess, and I need to test if it improves the speed of fmt::Display for u128/i128, but I think it's correct.
It hopefully is more efficient by cutting u128 into at most 2 u64s, and also chunks by 1e16 instead of just 1e4.
Also I specialized the implementations for uints to always be non-false because it bothered me that it was checked at all
Do not merge until I benchmark it and also clean up the god awful mess of spaghetti.
Based on prior work in #44583
cc: `@Dylan-DPC`
Due to work on `itoa` and suggestion in original issue:
r? `@dtolnay`
Allow Weak::as_ptr and friends for unsized T
Relaxes `impl<T> Weak<T>` to `impl<T: ?Sized> Weak<T>` for the methods `rc::Weak::as_ptr`, `into_raw`, and `from_raw`.
Follow-up to #73845, which did most of the impl work to make these functions work for `T: ?Sized`.
We still have to adjust the implementation of `Weak::from_raw` here, however, because I missed a use of `ptr.is_null()` previously. This check was necessary when `into`/`from_raw` were first implemented, as `into_raw` returned `ptr::null()` for dangling weak. However, we now just (wrapping) offset dangling weaks' pointers the same as nondangling weak, so the null check is no longer necessary (or even hit). (I can submit just 17a928f as a separate PR if desired.)
As a nice side effect, moves the `fn is_dangling` definition closer to `Weak::new`, which creates the dangling weak.
This technically stabilizes that "something like `align_of_val_raw`" is possible to do. However, I believe the part of the functionality required by these methods here -- specifically, getting the alignment of a pointee from a pointer where it may be dangling iff the pointee is `Sized` -- is uncontroversial enough to stabilize these methods without a way to implement them on stable Rust.
r? `@RalfJung,` who reviewed #73845.
ATTN: This changes (relaxes) the (input) generic bounds on stable fn!
Remove --cfg dox from rustdoc.rs
This was added in https://github.com/rust-lang/rust/pull/53076 because
several dependencies were using `cfg(dox)` instead of `cfg(rustdoc)` (now `cfg(doc)`).
I ran `rg 'cfg\(dox\)'` on the source tree with no matches, so I think
this is now safe to remove.
r? `@Mark-Simulacrum`
cc `@QuietMisdreavus` :)
Only use LOCAL_{STDOUT,STDERR} when set_{print/panic} is used.
The thread local `LOCAL_STDOUT` and `LOCAL_STDERR` are only used by the `test` crate to capture output from tests when running them in the same process in differen threads. However, every program will check these variables on every print, even outside of testing.
This involves allocating a thread local key, and registering a thread local destructor. This can be somewhat expensive.
This change keeps a global flag (`LOCAL_STREAMS`) which will be set to `true` when either of these local streams is used. (So, effectively only in test and benchmark runs.) When this flag is off, these thread locals are not even looked at and therefore will not be initialized on the first output on every thread, which also means no thread local destructors will be registered.
---
Together with https://github.com/rust-lang/rust/pull/77154, this should make output a little bit more efficient.
Fix Debug implementations of some of the HashMap and BTreeMap iterator types
HashMap's `ValuesMut`, BTreeMaps `ValuesMut`, IntoValues and `IntoKeys` structs were printing both keys and values on their Debug implementations. But they are iterators over either keys or values. Irrelevant values should not be visible. With this PR, they only show relevant fields.
This fixes#75297.
[Here's an example code.](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=0c79356ed860e347a0c1a205616f93b7) This prints this on nightly:
```
ValuesMut { inner: IterMut { range: [(1, "hello"), (2, "goodbye")], length: 2 } }
IntoKeys { inner: [(1, "hello"), (2, "goodbye")] }
IntoValues { inner: [(1, "hello"), (2, "goodbye")] }
[(2, "goodbye"), (1, "hello")]
```
After the patch this example prints these instead:
```
["hello", "goodbye"]
["hello", "goodbye"]
[1, 2]
["hello", "goodbye"]
```
I didn't add test cases for them, since I couldn't see any tests for Debug implementations anywhere. But please let me know if I should add it to a specific place.
r? @dtolnay
Use posix_spawn on musl targets
The posix_spawn had been available in a form suitable for use in a
Command implementation since musl 0.9.12. Use it in a preference to a
fork when possible, to benefit from CLONE_VM|CLONE_VFORK used there.
Previously `Command::spawn` would fall back to the non-posix_spawn based
implementation if the `PATH` environment variable was possibly changed.
On systems with a modern (g)libc `posix_spawn()` can be significantly
faster. If program is a path itself the `PATH` environment variable is
not used for the lookup and it should be safe to use the
`posix_spawnp()` method. [1]
We found this, because we have a cli application that effectively runs a
lot of subprocesses. It would sometimes noticeably hang while printing
output. Profiling showed that the process was spending the majority of
time in the kernel's `copy_page_range` function while spawning
subprocesses. During this time the process is completely blocked from
running, explaining why users were reporting the cli app hanging.
Through this we discovered that `std::process::Command` has a fast and
slow path for process execution. The fast path is backed by
`posix_spawnp()` and the slow path by fork/exec syscalls being called
explicitly. Using fork for process creation is supposed to be fast, but
it slows down as your process uses more memory. It's not because the
kernel copies the actual memory from the parent, but it does need to
copy the references to it (see `copy_page_range` above!). We ended up
using the slow path, because the command spawn implementation in falls
back to the slow path if it suspects the PATH environment variable was
changed.
Here is a smallish program demonstrating the slowdown before this code
change:
```
use std::process::Command;
use std::time::Instant;
fn main() {
let mut args = std::env::args().skip(1);
if let Some(size) = args.next() {
// Allocate some memory
let _xs: Vec<_> = std::iter::repeat(0)
.take(size.parse().expect("valid number"))
.collect();
let mut command = Command::new("/bin/sh");
command
.arg("-c")
.arg("echo hello");
if args.next().is_some() {
println!("Overriding PATH");
command.env("PATH", std::env::var("PATH").expect("PATH env var"));
}
let now = Instant::now();
let child = command
.spawn()
.expect("failed to execute process");
println!("Spawn took: {:?}", now.elapsed());
let output = child.wait_with_output().expect("failed to wait on process");
println!("Output: {:?}", output);
} else {
eprintln!("Usage: prog [size]");
std::process::exit(1);
}
()
}
```
Running it and passing different amounts of elements to use to allocate
memory shows that the time taken for `spawn()` can differ quite
significantly. In latter case the `posix_spawnp()` implementation is 30x
faster:
```
$ cargo run --release 10000000
...
Spawn took: 324.275µs
hello
$ cargo run --release 10000000 changepath
...
Overriding PATH
Spawn took: 2.346809ms
hello
$ cargo run --release 100000000
...
Spawn took: 387.842µs
hello
$ cargo run --release 100000000 changepath
...
Overriding PATH
Spawn took: 13.434677ms
hello
```
[1]: 5f72f9800b/posix/execvpe.c (L81)
Add accessors to Command.
This adds some accessor methods to `Command` to provide a way to access the values set when building the `Command`. An example where this can be useful is to display the command to be executed. This is roughly based on the [`ProcessBuilder`](13b73cdaf7/src/cargo/util/process_builder.rs (L105-L134)) in Cargo.
Possible concerns about the API:
- Values with NULs on Unix will be returned as `"<string-with-nul>"`. I don't think it is practical to avoid this, since otherwise a whole separate copy of all the values would need to be kept in `Command`.
- Does not handle `arg0` on Unix. This can be awkward to support in `get_args` and is rarely used. I figure if someone really wants it, it can be added to `CommandExt` as a separate method.
- Does not offer a way to detect `env_clear`. I'm uncertain if it would be useful for anyone.
- Does not offer a way to get an environment variable by name (`get_env`). I figure this can be added later if anyone really wants it. I think the motivation for this is weak, though. Also, the API could be a little awkward (return a `Option<Option<&OsStr>>`?).
- `get_envs` could skip "cleared" entries and just return `&OsStr` values instead of `Option<&OsStr>`. I'm on the fence here. My use case is to display a shell command, and I only intend it to be roughly equivalent to the actual execution, and I probably won't display `None` entries. I erred on the side of providing extra information, but I suspect many situations will just filter out the `None`s.
- Could implement more iterator stuff (like `DoubleEndedIterator`).
I have not implemented new std items before, so I'm uncertain if the existing issue should be reused, or if a new tracking issue is needed.
cc #44434
Related: https://github.com/rust-lang/rust/issues/66741
Guarded with `#![feature(default_alloc_error_handler)]` a default
`alloc_error_handler` is called, if a custom allocator is used and no
other custom `#[alloc_error_handler]` is defined.
The panic message does not contain the size anymore, because it would
pull in the fmt machinery, which would blow up the code size
significantly.
Fix is_absolute on WASI
WASI does not match `cfg(unix)`, but its paths are Unix-like (`/some/path`) and don't have Windows-like prefixes.
Without this change, `is_absolute` for any paths, including `/some/path`, was returning `false`on a WASI target, which is obviously not true and undesirable.
Split sys_common::Mutex in StaticMutex and MovableMutex.
The (unsafe) `Mutex` from `sys_common` had a rather complicated interface. You were supposed to call `init()` manually, unless you could guarantee it was neither moved nor used reentrantly.
Calling `destroy()` was also optional, although it was unclear if 1) resources might be leaked or not, and 2) if `destroy()` should only be called when `init()` was called.
This allowed for a number of interesting (confusing?) different ways to use this `Mutex`, all captured in a single type.
In practice, this type was only ever used in two ways:
1. As a static variable. In this case, neither `init()` nor `destroy()` are called. The variable is never moved, and it is never used reentrantly. It is only ever locked using the `LockGuard`, never with `raw_lock`.
2. As a `Box`ed variable. In this case, both `init()` and `destroy()` are called, it will be moved and possibly used reentrantly.
No other combinations are used anywhere in `std`.
This change simplifies things by splitting this `Mutex` type into two types matching the two use cases: `StaticMutex` and `MovableMutex`.
The interface of both new types is now both safer and simpler. The first one does not call nor expose `init`/`destroy`, and the second one calls those automatically in its `new()` and `Drop` functions. Also, the locking functions of `MovableMutex` are no longer unsafe.
---
This will also make it easier to conditionally box mutexes later, by moving that decision into sys/sys_common. Some of the mutex implementations (at least those of Wasm and 'sys/unsupported') are safe to move, so wouldn't need a box. ~~(But that's blocked on #76932 for now.)~~ (See #77380.)
Stabilize slice_ptr_range.
This has been unstable for almost a year now. Time to stabilize?
Closes#65807.
@rustbot modify labels: +T-libs +A-raw-pointers +A-slice +needs-fcp
Improve std::sys::windows::compat
Improves the compat_fn macro in sys::windows, which is used for conditionally loading APIs that might not be available.
- The module (dll) name can now be any string, not just an ident. (Not all Windows api modules are valid Rust identifiers. E.g. `WaitOnAddress` comes from `API-MS-Win-Core-Synch-l1-2-0.dll`.)
- Adds `FuncName::is_available()` for checking if a function is really available without having to do a duplicate lookup.
- Add comment explaining the lack of locking.
- Use `$_:block` to simplify the macro_rules.
- Apply `allow(unused_variables)` only to the fallback instead of everything.
---
The second point (`is_available()`) simplifies code that needs to pick an implementation depening on what is available, like `sys/windows/mutex.rs`. Before this change, it'd do its own lookup and keep its own `AtomicUsize` to track the result. Now it can just use `c::AcquireSRWLockExclusive::is_available()` directly.
This will also be useful when park/unpark/CondVar/etc. get improved implementations (e.g. from parking_lot or something else), as the best APIs for those are not available before Windows 8.
This commit entroduces `core::str::SplitInclusive::as_str` method similar to
`core::str::Split::as_str`, but under different gate -
"str_split_inclusive_as_str" (this is done so because `SplitInclusive` is
itself unstable).
This commit introduses 2 methods under "str_split_as_str" gate with common
signature of `&Split<'a, _> -> &'a str'`. Both of them work like
`Chars::as_str` - return unyield part of the inner string.
Refactor memchr to allow optimization
Closes#75659
The implementation already uses naive search if the slice if short enough, but the case is complicated enough to not be optimized away. This PR refactors memchr so that it exists early when the slice is short enough.
Codegen-wise, as shown in #75659, memchr was not inlined previously so the only way I could find to test this is to check if there is no memchr call. Let me know if there is a more robust solution here.
Make RawFd implement the RawFd traits
This PR makes `RawFd` implement `AsRawFd`, `IntoRawFd` and `FromRawFd`, so it can be passed to interfaces that use one of those traits as a bound.
- Module name can now be any string, not just an ident.
(Not all Windows api modules are valid Rust identifiers.)
- Adds c::FuncName::is_available() for checking if a function is really
available without having to do a duplicate lookup.
- Add comment explaining the lack of locking.
- Use `$_:block` to simplify the macro_rules.
- Apply allow(unused_variables) only to the fallback instead of
everything.
Use futex-based thread::park/unpark on Linux.
This moves the parking/unparking logic out of `thread/mod.rs` into a module named `thread_parker` in `sys_common`. The current implementation is moved to `sys_common/thread_parker/generic.rs` and the new implementation using futexes is added in `sys_common/thread_parker/futex.rs`.
Add Iterator::advance_by and DoubleEndedIterator::advance_back_by
This PR adds the iterator method
```rust
fn advance_by(&mut self, n: usize) -> Result<(), usize>
```
that advances the iterator by `n` elements, returning `Ok(())` if this succeeds or `Err(len)` if the length of the iterator was less than `n`.
Currently `Iterator::nth` is the method to override for efficiently advancing an iterator by multiple elements at once. `advance_by` is superior for this purpose because
- it's simpler to implement: instead of advancing the iterator and producing the next element you only need to advance the iterator
- it composes better: iterators like `Chain` and `FlatMap` can implement `advance_by` in terms of `advance_by` on their inner iterators, but they cannot implement `nth` in terms of `nth` on their inner iterators (see #60395)
- the default implementation of `nth` can trivially be implemented in terms of `advance_by` and `next`, which this PR also does
This PR also adds `DoubleEndedIterator::advance_back_by` for all the same reasons.
I'll make a tracking issue if it's decided this is worth merging. Also let me know if anything can be improved, this went through several iterations so there might very well still be room for improvement (especially in the doc comments). I've written overrides of these methods for most iterators that already override `nth`/`nth_back`, but those still need tests so I'll add them in a later PR.
cc @cuviper @scottmcm @Amanieu
The posix_spawn had been available in a form suitable for use in a
Command implementation since musl 0.9.12. Use it in a preference to a
fork when possible, to benefit from CLONE_VM|CLONE_VFORK used there.
Split core/str/mod.rs to smaller files
Note for reviewer:
* I split to multiple commits for easier reviewing, but I could git squash them all to one if requested.
* Recommend pulling this change locally and using advanced git diff viewer or this command:
```bash
git show --reverse --color-moved=dimmed-zebra --color-moved-ws=ignore-all-space master..
```
---
I split `core/str/mod.rs` to these modules:
* `converts`: Contains helper functions to convert from bytes to str.
* `error`: For error structs like Utf8Error.
* `iter`: For iterators of many str methods.
* `traits`: For indexing operations and build in traits on str.
* `validations`: For functions validating utf8 --- This name is awkward, maybe utf8.rs is better.
Use `rtassert!` instead of `assert!` from the child process after fork() in std::sys::unix::process::Command::spawn()
As discussed in #73894, `assert!` panics on failure, which is not signal-safe, and `rtassert!` is a suitable replacement.
Fixes#73894.
r? @Amanieu @cuviper @joshtriplett
library: Forward compiler-builtins "mem" feature
This fixes https://github.com/rust-lang/wg-cargo-std-aware/issues/53
Now users will be able to do:
```
cargo build -Zbuild-std=core -Zbuild-std-features=compiler-builtins-mem
```
and correctly get the Rust implemenations for `memcpy` and friends.
Signed-off-by: Joe Richey <joerichey@google.com>
WASI does not match `cfg(unix)`, but its paths are Unix-like (`/some/path`) and don't have Windows-like prefixes.
Without this change, `is_absolute` for paths like `/some/path` was returning `false`on a WASI target, which is obviously not true and undesirable.
Resolve `crate` in intra-doc links properly across crates
Closes https://github.com/rust-lang/rust/issues/77193; see https://github.com/rust-lang/rust/issues/77193#issuecomment-699065946 for an explanation of what's going on here.
~~This also fixes the BTreeMap docs that have been broken for a while; see the description on the second commit for why and how.~~ Nope, see the second commit for why the link had to be changed.
r? `@Manishearth`
cc `@dylni`
`@dylni` note that this doesn't solve your original problem - now _both_ `with_code` and `crate::with_code` will be broken links. However this will fix a lot of other broken links (in particular I think https://docs.rs/sqlx/0.4.0-beta.1/sqlx/query/struct.Query.html is because of this bug). I'll open another issue for resolving additional docs in the new scope.
Add zero padding
Add benchmarks for fmt u128
This tests both when there is the max amount of work(all characters used)
And least amount of work(1 character used)
Add doc alias for iterator fold
fold is known in python and javascript as reduce,
not sure about inject but it was written in doc there.
This was my first confusion when coming into rust, I somehow cannot find where is reduce, sometimes I still forget that it is known as `fold`.
Remove `#[rustc_allow_const_fn_ptr]` and add `#![feature(const_fn_fn_ptr_basics)]`
`rustc_allow_const_fn_ptr` was a hack to work around the lack of an escape hatch for the "min `const fn`" checks in const-stable functions. Now that we have co-opted `allow_internal_unstable` for this purpose, we no longer need a bespoke attribute.
Now this functionality is gated under `const_fn_fn_ptr_basics` (how concise!), and `#[allow_internal_unstable(const_fn_fn_ptr_basics)]` replaces `#[rustc_allow_const_fn_ptr]`. `const_fn_fn_ptr_basics` allows function pointer types to appear in the arguments and locals of a `const fn` as well as function pointer casts to be performed inside a `const fn`. Both of these were allowed in constants and statics already. Notably, this does **not** allow users to invoke function pointers in a const context. Presumably, we will use a nicer name for that (`const_fn_ptr`?).
r? @oli-obk
UI to unit test for those using Cell/RefCell/UnsafeCell
Helps with #76268.
I'm working on all files using `Cell` and moving them to unit tests when possible.
r? @matklad
Add missing definitions required by the sparc-unknown-linux-gnu target
This PR adds a few missing definitions required by sparc-unknown-linux-target which were discovered during build tests.
This fixes https://github.com/rust-lang/wg-cargo-std-aware/issues/53
Now users will be able to do:
```
cargo build -Zbuild-std=core -Zbuild-std-features=compiler-builtins-mem
```
and correctly get the Rust implemenations for `memcpy` and friends.
Signed-off-by: Joe Richey <joerichey@google.com>
The syscalls returning a new file descriptors generally use
lowest-numbered file descriptor not currently opened, without any
exceptions for those corresponding to the standard streams.
Previously when any of standard streams has been closed before starting
the application, operations on std::io::{stderr,stdin,stdout} objects
were likely to operate on other logically unrelated file resources
opened afterwards.
Avoid the issue by reopening the standard streams when they are closed.
This was a hack to work around the lack of an escape hatch for the "min
`const fn`" checks in const-stable functions. Now that we have co-opted
`allow_internal_unstable` for this purpose, we no longer need the
bespoke attribute.
Previously, `BTreeMap` tried to link to `crate::collections`, intending
for the link to go to `std/collections/index.html`. But `BTreeMap` is
defined in `alloc`, so after the fix in the previous commit, the links
instead went to `alloc/collections/index.html`, which has almost no
information.
This changes it to link to `index.html`, which only works when viewing
from `std::collections::BTreeMap`, the most common place to visit the
docs. Fixing it to work from anywhere would require the docs for
`std::collections` to be duplicated in `alloc::collections`, which in
turn would require HashMap to be `alloc` for intra-doc links to work
(https://github.com/rust-lang/rust/issues/74481).
update stdarch submodule
This commit update the src/stdarch submodule, we primarily want to include [https://github.com/rust-lang/stdarch/pull/918](url) which provides prefetch hints for aarch64. This PR could deliver ~20% performance gain on our aarch64 server in Filecoin. Wish this could be used as soon as possible.
Thanks.
The thread local LOCAL_STDOUT and LOCAL_STDERR are only used by the test
crate to capture output from tests when running them in the same process
in differen threads. However, every program will check these variables
on every print, even outside of testing.
This involves allocating a thread local key, and registering a thread
local destructor. This can be somewhat expensive.
This change keeps a global flag (LOCAL_STREAMS) which will be set to
true when either of these local streams is used. (So, effectively only
in test and benchmark runs.) When this flag is off, these thread locals
are not even looked at and therefore will not be initialized on the
first output on every thread, which also means no thread local
destructors will be registered.
The (unsafe) Mutex from sys_common had a rather complicated interface.
You were supposed to call init() manually, unless you could guarantee it
was neither moved nor used reentrantly.
Calling `destroy()` was also optional, although it was unclear if 1)
resources might be leaked or not, and 2) if destroy() should only be
called when `init()` was called.
This allowed for a number of interesting (confusing?) different ways to
use this Mutex, all captured in a single type.
In practice, this type was only ever used in two ways:
1. As a static variable. In this case, neither init() nor destroy() are
called. The variable is never moved, and it is never used
reentrantly. It is only ever locked using the LockGuard, never with
raw_lock.
2. As a Boxed variable. In this case, both init() and destroy() are
called, it will be moved and possibly used reentrantly.
No other combinations are used anywhere in `std`.
This change simplifies things by splitting this Mutex type into
two types matching the two use cases: StaticMutex and MovableMutex.
The interface of both new types is now both safer and simpler. The first
one does not call nor expose init/destroy, and the second one calls
those automatically in its new() and Drop functions. Also, the locking
functions of MovableMutex are no longer unsafe.
Remove std::io::lazy::Lazy in favour of SyncOnceCell
The (internal) std::io::lazy::Lazy was used to lazily initialize the stdout and stdin buffers (and mutexes). It uses atexit() to register a destructor to flush the streams on exit, and mark the streams as 'closed'. Using the stream afterwards would result in a panic.
Stdout uses a LineWriter which contains a BufWriter that will flush the buffer on drop. This one is important to be executed during shutdown, to make sure no buffered output is lost. It also forbids access to stdout afterwards, since the buffer is already flushed and gone.
Stdin uses a BufReader, which does not implement Drop. It simply forgets any previously read data that was not read from the buffer yet. This means that in the case of stdin, the atexit() function's only effect is making stdin inaccessible to the program, such that later accesses result in a panic. This is uncessary, as it'd have been safe to access stdin during shutdown of the program.
---
This change removes the entire io::lazy module in favour of SyncOnceCell. SyncOnceCell's fast path is much faster (a single atomic operation) than locking a sys_common::Mutex on every access like Lazy did.
However, SyncOnceCell does not use atexit() to drop the contained object during shutdown.
As noted above, this is not a problem for stdin. It simply means stdin is now usable during shutdown.
The atexit() call for stdout is moved to the stdio module. Unlike the now-removed Lazy struct, SyncOnceCell does not have a 'gone and unusable' state that panics. Instead of adding this again, this simply replaces the buffer with one with zero capacity. This effectively flushes the old buffer *and* makes any writes afterwards pass through directly without touching a buffer, making print!() available during shutdown without panicking.
---
In addition, because the contents of the SyncOnceCell are no longer dropped, we can now use `&'static` instead of `Arc` in `Stdout` and `Stdin`. This also saves two levels of indirection in `stdin()` and `stdout()`, since Lazy effectively stored a `Box<Arc<T>>`, and SyncOnceCell stores the `T` directly.
Rust vec bench import specific rand::RngCore
Using `RngCore` import for side effects is clearer than `*` which may bring it unnecessary more stuff than needed, it is also more explicit doing so.
@pickfire change `LEN = 16384` (and pos) and `once` instead of `[0].iter()` after this.
@rustbot modify labels: +C-cleanup +A-testsuite
Add `#![feature(const_fn_floating_point_arithmetic)]`
cc #76618
This is a template for splitting up `const_fn` into granular feature gates. I think this will make it easier, both for us and for users, to track stabilization of each individual feature. We don't *have* to do this, however. We could also keep stabilizing things out from under `const_fn`.
cc @rust-lang/wg-const-eval
r? @oli-obk
Explicitly document the size guarantees that Option makes.
Triggered by a discussion on wg-unsafe-code-guidelines about which layouts of `Option<T>` one can guarantee are optimised to a single pointer.
CC @RalfJung
Std/thread: deny unsafe op in unsafe fn
Partial fix of #73904.
This encloses `unsafe` operations in `unsafe fn` in `libstd/thread`.
`@rustbot` modify labels: F-unsafe-block-in-unsafe-fn
It's possible for method resolution to pick this method over a lower
priority stable method, causing compilation errors. Since this method
is permanently unstable, give it a name that is very unlikely to be used
in user code.
Make [].as_[mut_]ptr_range() (unstably) const.
Gated behind `const_ptr_offset`, as suggested by https://github.com/rust-lang/rust/issues/65807#issuecomment-697229404
This also marks `[].as_mut_ptr()` as const, because it's used by `as_mut_ptr_range`. I gated it behind the same feature, because I figured it's not worth adding a separate tracking issue for const `as_mut_ptr`.
BtreeMap: refactoring around edges
Parts chipped off a more daring effort, that the btree benchmarks judge to be performance-neutral.
r? @Mark-Simulacrum
Relax promises about condition variable.
For quite a while now, there have been plans to at some point use parking_lot or some other more efficient implementation of mutexes and condition variables. Right now, Mutex and CondVar both Box the 'real' mutex/condvar inside, to give it a stable address. This was done because implementations like pthread and Windows critical sections may not be moved. More efficient implementations based on futexes, WaitOnAddress, Windows SRW locks, parking_lot, etc. may be moved (while not borrowed), so wouldn't need boxing.
However, not boxing them (which would be great goal to achieve), breaks a promise std currently makes about CondVar. CondVar promises to panic when used with different mutexes, to ensure consistent behaviour on all platforms. To this check, a mutex is considered 'the same' if the address of the 'real mutex' in the Box is the same. This address doesn't change when moving a `std::mutex::Mutex` object, effectively giving it an identity that survives moves of the Mutex object. If we ever switch to a non-boxed version, they no longer carry such an identity, and this check can no longer be made.
Four options:
1. Always box mutexes.
2. Add a `MutexId` similar to `ThreadId`. Making mutexes bigger, and making it hard to ever have a `const fn new` for them.
3. Making the requirement of CondVar stricter: panic if the Mutex object itself moved.
4. Making the promise of CondVar weaker: don't promise to panic.
1, 2, and 3 seem like bad options. This PR updates the documentation for 4.
Use `Self` in docs when possible
Fixes#76542.
I used `rg '\s*//[!/]\s+fn [\w_]+\(&?self, ' .` in `library/` to find instances, I found some with that and some by manually checking.
@rustbot modify labels: C-enhancement T-doc
add array::from_ref
mirrors the methods in `std::slice` with the same name.
I guess this method previously didn't exist as there was close to no reason to create an array of size `1`.
This will change due to const generics in the near future.
Make delegation methods of `std::net::IpAddr` unstably const
Make the following methods of `std::net::IpAddr` unstable const under the `const_ip` feature:
- `is_unspecified`
- `is_loopback`
- `is_global`
- `is_multicast`
Also adds a test for these methods in a const context.
Possible because these methods delegate to the inner `Ipv4Addr` or `Ipv6Addr`, which were made const ([PR#76205](https://github.com/rust-lang/rust/pull/76142) and [PR#76206](https://github.com/rust-lang/rust/pull/76206)), and the recent stabilization of const control flow.
Part of #76205
r? @ecstatic-morse
The (internal) std::io::lazy::Lazy was used to lazily initialize the
stdout and stdin buffers (and mutexes). It uses atexit() to register a
destructor to flush the streams on exit, and mark the streams as
'closed'. Using the stream afterwards would result in a panic.
Stdout uses a LineWriter which contains a BufWriter that will flush the
buffer on drop. This one is important to be executed during shutdown,
to make sure no buffered output is lost. It also forbids access to
stdout afterwards, since the buffer is already flushed and gone.
Stdin uses a BufReader, which does not implement Drop. It simply forgets
any previously read data that was not read from the buffer yet. This
means that in the case of stdin, the atexit() function's only effect is
making stdin inaccessible to the program, such that later accesses
result in a panic. This is uncessary, as it'd have been safe to access
stdin during shutdown of the program.
---
This change removes the entire io::lazy module in favour of
SyncOnceCell. SyncOnceCell's fast path is much faster (a single atomic
operation) than locking a sys_common::Mutex on every access like Lazy
did.
However, SyncOnceCell does not use atexit() to drop the contained object
during shutdown.
As noted above, this is not a problem for stdin. It simply means stdin
is now usable during shutdown.
The atexit() call for stdout is moved to the stdio module. Unlike the
now-removed Lazy struct, SyncOnceCell does not have a 'gone and
unusable' state that panics. Instead of adding this again, this simply
replaces the buffer with one with zero capacity. This effectively
flushes the old buffer *and* makes any writes afterwards pass through
directly without touching a buffer, making print!() available during
shutdown without panicking.
revert const_type_id stabilization
This reverts #72488, which is currently on beta and scheduled to stabilize in `1.47.0`, based on https://github.com/rust-lang/rust/pull/75923#issuecomment-696676511
It turns out we might not be quite ready to stabilize `TypeId` in const contexts before having a chance to rework its internals. Since `TypeId` is a bit of an oddity we want to be careful about how those internals are currently being relied on while making changes. That will be easier to do without having to also consider compile-time contexts.
r? `@eddyb`
Rollup of 9 pull requests
Successful merges:
- #76898 (Record `tcx.def_span` instead of `item.span` in crate metadata)
- #76939 (emit errors during AbstractConst building)
- #76965 (Add cfg(target_has_atomic_equal_alignment) and use it for Atomic::from_mut.)
- #76993 (Changing the alloc() to accept &self instead of &mut self)
- #76994 (fix small typo in docs and comments)
- #77017 (Add missing examples on Vec iter types)
- #77042 (Improve documentation for ToSocketAddrs)
- #77047 (Miri: more informative deallocation error messages)
- #77055 (Add #[track_caller] to more panicking Cell functions)
Failed merges:
r? `@ghost`
Make the following methods of `std::net::IpAddr` unstable const under the `const_ip` feature:
- `is_unspecified`
- `is_loopback`
- `is_global`
- `is_multicast`
Also adds a test for these methods in a const context.
Possible because these methods delegate to the inner `Ipv4Addr` or `Ipv6Addr`, which were made const, and the recent stabilization of const control flow.
Part of #76205
Add #[track_caller] to more panicking Cell functions
Continuation of #74526
Adds the #[track_caller] attribute to almost all panicking Cell
functions. The ones that borrow two Cells in their function
body are spared, because the panic location helps pinpoint
which of the two borrows failed. You'd need to have
full debuginfo and backtraces enabled together with column
info in order to be able to discern the cases.
Column info in debuginfo is only available on non-Windows platforms.
Add cfg(target_has_atomic_equal_alignment) and use it for Atomic::from_mut.
Fixes some platform-specific problems with #74532 by using the actual alignment of the types instead of hardcoding a few `target_arch`s.
r? @RalfJung
Continuation of #74526
Adds the #[track_caller] attribute to almost all panicking Cell
functions. The ones that borrow two Cells in their function
body are spared, because the panic location helps pinpoint
which of the two borrows failed. You'd need to have
full debuginfo and backtraces enabled together with column
info in order to be able to discern the cases.
Column info is only available on non-Windows platforms.
Function to convert OpenOptions to c_int
Fixes: #74943
The creation_mode and access_mode function were already available in the OpenOptions struct, but currently private. I've added a new free functions to unix/fs.rs which takes the OpenOptions, and returns the c_int to be used as parameter for the `open` call.
Make some methods of `Pin` unstable const
Make the following methods unstable const under the `const_pin` feature:
- `new`
- `new_unchecked`
- `into_inner`
- `into_inner_unchecked`
- `get_ref`
- `into_ref`
- `get_mut`
- `get_unchecked_mut`
Of these, `into_inner` and `into_inner_unchecked` require the unstable `const_precise_live_drops`.
Also adds tests for these methods in a const context.
Tracking issue: #76654
r? @ecstatic-morse
Don't recommend ManuallyDrop to customize drop order
See
https://internals.rust-lang.org/t/need-for-controlling-drop-order-of-fields/12914/21
for the discussion.
TL;DR: ManuallyDrop is unsafe and footguny, but you can just ask the compiler to do all the work for you by re-ordering declarations.
Specifically, the original example from the docs is much better written as
```rust
struct Peach;
struct Banana;
struct Melon;
struct FruitBox {
melon: Melon,
// XXX: mind the relative drop order of the fields below
peach: Peach,
banana: Banana,
}
```
Remove duplicated library links between std and libc
The libc crate is already responsible for linking in the appropriate
libraries, and std doing the same thing results in duplicated library
names on the linker command line. Removing this duplication slightly
reduces linker time, and makes it simpler to adjust the set or order of
linked libraries in one place (such as to add static linking support).
Revert adding Atomic::from_mut.
This reverts #74532, which made too many assumptions about platforms, breaking some things.
Will need to be added later with a better way of gating on proper alignment, without hardcoding cfg(target_arch)s.
---
To be merged if fixing from_mut (#76965) takes too long.
r? @ghost
Add non-`unsafe` `.get_mut()` for `Unsafecell`
- Tracking issue: https://github.com/rust-lang/rust/issues/76943
As discussed in: https://internals.rust-lang.org/t/add-non-unsafe-get-mut-for-unsafecell/12407
- ### [Rendered documentation](https://modest-dubinsky-1f9f47.netlify.app/core/cell/struct.unsafecell)
This PR tries to move the sound `&mut UnsafeCell<T> -> &mut T` projection that all the "downstream" constructions were already relying on, up to the root abstraction, where it rightfully belongs, and officially blessing it.
- this **helps reduce the amount of `unsafe` snippets out there** (_c.f._, the second commit of this PR: 09503fd1b3)
The fact that this getter is now expose for `UnsafeCell<T>` itself, will also help convey the idea that **`UnsafeCell` is not magical _w.r.t._ `&mut` accesses**, contrary to what some people incorrectly think.
- Even the standard library itself at some point had such a confusion, _c.f._ this comment where there is a mention of multi-threaded (and thus _shared_) access despite dealing with exclusive references over unique ownership: 59fb88d061/library/core/src/cell.rs (L498-L499)
r? @RalfJung
