Cleanup cloudabi mutexes and condvars
This gets rid of lots of unnecessary unsafety.
All the AtomicU32s were wrapped in UnsafeCell or UnsafeCell<MaybeUninit>, and raw pointers were used to get to the AtomicU32 inside. This change cleans that up by using AtomicU32 directly.
Also replaces a UnsafeCell<u32> by a safer Cell<u32>.
@rustbot modify labels: +C-cleanup
Static mutex is static
StaticMutex is only ever used with as a static (as the name already suggests). So it doesn't have to be generic over a lifetime, but can simply assume 'static.
This 'static lifetime guarantees the object is never moved, so this is no longer a manually checked requirement for unsafe calls to lock().
@rustbot modify labels: +T-libs +A-concurrency +C-cleanup
For backtrace, use StaticMutex instead of a raw sys Mutex.
The code used the very unsafe `sys::mutex::Mutex` directly, and built its own unlock-on-drop wrapper around it. The StaticMutex wrapper already provides that and is easier to use safely.
@rustbot modify labels: +T-libs +C-cleanup
Use futex-based thread-parker for Wasm32.
This uses the existing `sys_common/thread_parker/futex.rs` futex-based thread parker (that was already used for Linux) for wasm32 as well (if the wasm32 atomics target feature is enabled, which is not the case by default).
Wasm32 provides the basic futex operations as instructions: https://webassembly.github.io/threads/syntax/instructions.html
These are now exposed from `sys::futex::{futex_wait, futex_wake}`, just like on Linux. So, `thread_parker/futex.rs` stays completely unmodified.
Refactor io/buffered.rs into submodules
This pull request splits `BufWriter`, `BufReader`, `LineWriter`, and `LineWriterShim` (along with their associated tests) into separate submodules. It contains no functional changes. This change is being made in anticipation of adding another type of buffered writer which can be switched between line- and block-buffering mode.
Part of a series of pull requests resolving #60673.
StaticMutex is only ever used with as a static (as the name already
suggests). So it doesn't have to be generic over a lifetime, but can
simply assume 'static.
This 'static lifetime guarantees the object is never moved, so this is
no longer a manually checked requirement for unsafe calls to lock().
The comment said it's UB to call lock() while it is locked. That'd be
quite a useless Mutex. :) It was supposed to say 'locked by the same
thread', not just 'locked'.
warning: the operation is ineffective. Consider reducing it to
`self.segments()[0]`
--> library/std/src/net/ip.rs:1265:9
|
1265 | (self.segments()[0] & 0xffff) == 0xfe80
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: `#[warn(clippy::identity_op)]` on by default
= help: for further information visit
https://rust-lang.github.io/rust-clippy/master/index.html#identity_op
warning: the operation is ineffective. Consider reducing it to
`self.segments()[1]`
--> library/std/src/net/ip.rs:1266:16
|
1266 | && (self.segments()[1] & 0xffff) == 0
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= help: for further information visit
https://rust-lang.github.io/rust-clippy/master/index.html#identity_op
warning: the operation is ineffective. Consider reducing it to
`self.segments()[2]`
--> library/std/src/net/ip.rs:1267:16
|
1267 | && (self.segments()[2] & 0xffff) == 0
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= help: for further information visit
https://rust-lang.github.io/rust-clippy/master/index.html#identity_op
warning: the operation is ineffective. Consider reducing it to
`self.segments()[3]`
--> library/std/src/net/ip.rs:1268:16
|
1268 | && (self.segments()[3] & 0xffff) == 0
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= help: for further information visit
https://rust-lang.github.io/rust-clippy/master/index.html#identity_op
Signed-off-by: wcampbell <wcampbell1995@gmail.com>
warning: struct update has no effect, all the fields in the struct have
already been specified
--> library/std/src/net/addr.rs:367:19
|
367 | ..unsafe { mem::zeroed() }
| ^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: `#[warn(clippy::needless_update)]` on by default
= help: for further information visit
https://rust-lang.github.io/rust-clippy/master/index.html#needless_update
Remove unsafety from sys/unsupported and add deny(unsafe_op_in_unsafe_fn).
Replacing `UnsafeCell`s by a `Cell`s simplifies things and makes the mutex and rwlock implementations safe. Other than that, only unsafety in strlen() contained unsafe code.
@rustbot modify labels: +F-unsafe-block-in-unsafe-fn +C-cleanup
Implement `AsRawFd` for `StdinLock` etc. on WASI.
WASI implements `AsRawFd` for `Stdin`, `Stdout`, and `Stderr`, so
implement it for `StdinLock`, `StdoutLock`, and `StderrLock` as well.
r? @alexcrichton
Avoid SeqCst or static mut in mach_timebase_info and QueryPerformanceFrequency caches
This patch went through a couple iterations but the end result is replacing a pattern where an `AtomicUsize` (updated with many SeqCst ops) guards a `static mut` with a single `AtomicU64` that is known to use 0 as a value indicating that it is not initialized.
The code in both places exists to cache values used in the conversion of Instants to Durations on macOS, iOS, and Windows.
I have no numbers to prove that this improves performance (It seems a little futile to benchmark something like this), but it's much simpler, safer, and in practice we'd expect it to be faster everywhere where Relaxed operations on AtomicU64 are cheaper than SeqCst operations on AtomicUsize, which is a lot of places.
Anyway, it also removes a bunch of unsafe code and greatly simplifies the logic, so IMO that alone would be worth it unless it was a regression.
If you want to take a look at the assembly output though, see https://godbolt.org/z/rbr6vn for x86_64, https://godbolt.org/z/cqcbqv for aarch64 (Note that this just the output of the mac side, but i'd expect the windows part to be the same and don't feel like doing another godbolt for it). There are several versions of this function in the godbolt:
- `info_new`: version in the current patch
- `info_less_new`: version in initial PR
- `info_original`: version currently in the tree
- `info_orig_but_better_orderings`: a version that just tries to change the original code's orderings from SeqCst to the (probably) minimal orderings required for soundness/correctness.
The biggest concern I have here is if we can use AtomicU64, or if there are targets that dont have it that this code supports. AFAICT: no. (If that changes in the future, it's easy enough to do something different for them)
r? `@Amanieu` because he caught a couple issues last time I tried to do a patch reducing orderings 😅
---
<details>
<summary>I rewrote this whole message so the original is inside here</summary>
I happened to notice the code we use for caching the result of mach_timebase_info uses SeqCst exclusively.
However, thinking a little more, it's actually pretty easy to avoid the static mut by packing the timebase info into an AtomicU64.
This entirely avoids needing to do the compare_exchange. The AtomicU64 can be read/written using Relaxed ops, which on current macos/ios platforms (x86_64/aarch64) have no overhead compared to direct loads/stores. This simplifies the code and makes it a lot safer too.
I have no numbers to prove that this improves performance (It seems a little futile to benchmark something like this), although it should do that on both targets it applies to.
That said, it also removes a bunch of unsafe code and simplifies the logic (arguably at least — there are only two states now, initialized or not), so I think it's a net win even without concrete numbers.
If you want to take a look at the assembly output though, see below. It has the new version, the original, and a version of the original with lower Orderings (which is still worse than the version in this PR)
- godbolt.org/z/obfqf9 x86_64-apple-darwin
- godbolt.org/z/Wz5cWc aarch64-unknown-linux-gnu (godbolt can't do aarch64-apple-ios but that doesn't matter here)
A different (and more efficient) option than this would be to just use the AtomicU64 and use the knowledge that after initialization the denominator should be nonzero... That felt like it's relying on too many things I'm not confident in, so I didn't want to do that.
</details>
rust-lang/rust#77147 simplifies things by splitting this Mutex type
into two types matching the two use cases: StaticMutex and MovableMutex.
To support the behavior of StaticMutex, we move part of the mutex
implementation into libstd.
doc: disambiguate stat in MetadataExt::as_raw_stat
A few architectures in `os::linux::raw` import `libc::stat`, rather than
defining that type directly. However, that also imports the _function_
called `stat`, which makes this doc link ambiguous:
error: `crate::os::linux::raw::stat` is both a struct and a function
--> library/std/src/os/linux/fs.rs:21:19
|
21 | /// [`stat`]: crate::os::linux::raw::stat
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^ ambiguous link
|
= note: `-D broken-intra-doc-links` implied by `-D warnings`
help: to link to the struct, prefix with the item type
|
21 | /// [`stat`]: struct@crate::os::linux::raw::stat
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
help: to link to the function, add parentheses
|
21 | /// [`stat`]: crate::os::linux::raw::stat()
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
We want the `struct`, so it's now prefixed accordingly.
Link to documentation-specific guidelines.
Changed contribution information URL because it's not obvious how to get from the current URL to the documentation-specific content.
The current URL points to this "Getting Started" page, which contains nothing specific about documentation[*] and instead launches into how to *build* `rustc` which is not a strict prerequisite for contributing documentation fixes:
* https://rustc-dev-guide.rust-lang.org/getting-started.html
[*] The most specific content is a "Writing documentation" bullet point which is not itself a link to anything (I guess a patch for that might be helpful too).
### Why?
Making this change will make it easier for people who wish to make small "drive by" documentation fixes (and read contribution guidelines ;) ) which I find are often how I start contributing to a project. (Exhibit A: https://github.com/rust-lang/rust/pull/77050 :) )
### Background
My impression is the change of content linked is an unintentional change due to a couple of other changes:
* Originally, the link pointed to `contributing.md` which started with a "table of contents" linking to each section. But the content in `contributing.md` was removed and replaced with a link to the "Getting Started" section here:
* 3f6928f1f6 (diff-6a3371457528722a734f3c51d9238c13L1)
But the changed link doesn't actually point to the equivalent content, which is now located here:
* https://rustc-dev-guide.rust-lang.org/contributing.html
(If the "Guide to Rustc Development" is now considered the canonical location of "How to Contribute" content it might be a good idea to merge some of the "Contributing" Introduction section into the "Getting Started" section.)
* This was then compounded by changing the link from `contributing.md` to `contributing.html` here:
* https://github.com/rust-lang/rust/pull/74037/files#diff-242481015141f373dcb178e93cffa850L88
In order to even find the new location of the previous `contributing.md` content I ended up needing to do a GitHub search of the `rust-lang` org for the phrase "Documentation improvements are very welcome". :D
Fix error checking in posix_spawn implementation of Command
* Check for errors returned from posix_spawn*_init functions
* Check for non-zero return value from posix_spawn functions
A few architectures in `os::linux::raw` import `libc::stat`, rather than
defining that type directly. However, that also imports the _function_
called `stat`, which makes this doc link ambiguous:
error: `crate::os::linux::raw::stat` is both a struct and a function
--> library/std/src/os/linux/fs.rs:21:19
|
21 | /// [`stat`]: crate::os::linux::raw::stat
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^ ambiguous link
|
= note: `-D broken-intra-doc-links` implied by `-D warnings`
help: to link to the struct, prefix with the item type
|
21 | /// [`stat`]: struct@crate::os::linux::raw::stat
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
help: to link to the function, add parentheses
|
21 | /// [`stat`]: crate::os::linux::raw::stat()
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
We want the `struct`, so it's now prefixed accordingly.
`DirEntry` contains a `ReadDir` handle, which used to just be a wrapper
on `Arc<InnerReadDir>`. Commit af75314ecd added `end_of_stream: bool`
which is not needed by `DirEntry`, but adds 8 bytes after padding. We
can let `DirEntry` have an `Arc<InnerReadDir>` directly to avoid that.
The posix_spawnattr_init & posix_spawn_file_actions_init might fail,
but their return code is not checked.
Check for non-zero return code and destroy only succesfully initialized
objects.
The cvt function compares the argument with -1 and when equal returns a new
io::Error constructed from errno. It is used together posix_spawn_* functions.
This is incorrect. Those functions do not set errno. Instead they return
non-zero error code directly.
Check for non-zero return code and use it to construct a new io::Error.
(docs): make mutex error comment consistent with codebase
Although exceptionally minor, I found this stands out from other error reporting language used in doc comments. With the existence of the `failure` crate, I suppose this could be slightly ambiguous. In any case, this change brings the particular comment into a consistent state with other mentions of returning errors.
Support static linking with glibc and target-feature=+crt-static
With this change, it's possible to build on a linux-gnu target and pass
RUSTFLAGS='-C target-feature=+crt-static' or the equivalent via a
`.cargo/config.toml` file, and get a statically linked executable.
Update to libc 0.2.78, which adds support for static linking with glibc.
Add `crt_static_respected` to the `linux_base` target spec.
Update `android_base` and `linux_musl_base` accordingly. Avoid enabling
crt_static_respected on Android platforms, since that hasn't been
tested.
Closes https://github.com/rust-lang/rust/issues/65447.
Minor `hash_map` doc adjustments + item attribute orderings
This PR is really a couple visual changes glued together:
1. Some of the doc comments for items in `std::collections::hash_map` referenced the names of types without escaping their formatting (e.g. using "VacantEntry" instead of "`VacantEntry`") - the ones I could find were changed to the latter
2. The vast majority of pre-item attributes seem to place doc comments as the first attribute (instead of things like `#[feature(...)]`), so the few that had the other order were changed.
3. Also ordering related: the general trend seems to be that `#[feature]` attributes follow `#[inline]`, so I swapped the two lines in places where that ordering was reversed. This is primarily a change based on stylistic continuity and aesthetics - I'm not sure how important that actually is / should be.
I figured this would be pretty uncontroversial, but some of these might have been intentional for reasons I don't know about - if so, I'd be happy to remove the relevant changes. Of these, the final set of changes is probably the most unnecessary, so it also might be better to leave those out (in favor of reducing code churn).
Unbox mutexes and condvars on some platforms
Both mutexes and condition variables contained a Box containing the actual os-specific object. This was done because moving these objects may cause undefined behaviour on some platforms.
However, this is not needed on Windows[1], Wasm[2], cloudabi[2], and 'unsupported'[3], were the box was only needlessly making them less efficient.
This change gets rid of the box on those platforms.
On those platforms, `Condvar` can no longer verify it is only used with one `Mutex`, as mutexes no longer have a stable address. This was addressed and considered acceptable in #76932.
[1]\: https://docs.microsoft.com/en-us/windows/win32/api/synchapi/nf-synchapi-initializesrwlock
[2]\: These are just a single atomic integer together with futex wait/wake calls/instructions.
[3]\: The `unsupported` platform doesn't support multiple threads at all.
Only use LOCAL_{STDOUT,STDERR} when set_{print/panic} is used.
The thread local `LOCAL_STDOUT` and `LOCAL_STDERR` are only used by the `test` crate to capture output from tests when running them in the same process in differen threads. However, every program will check these variables on every print, even outside of testing.
This involves allocating a thread local key, and registering a thread local destructor. This can be somewhat expensive.
This change keeps a global flag (`LOCAL_STREAMS`) which will be set to `true` when either of these local streams is used. (So, effectively only in test and benchmark runs.) When this flag is off, these thread locals are not even looked at and therefore will not be initialized on the first output on every thread, which also means no thread local destructors will be registered.
---
Together with https://github.com/rust-lang/rust/pull/77154, this should make output a little bit more efficient.
Fix Debug implementations of some of the HashMap and BTreeMap iterator types
HashMap's `ValuesMut`, BTreeMaps `ValuesMut`, IntoValues and `IntoKeys` structs were printing both keys and values on their Debug implementations. But they are iterators over either keys or values. Irrelevant values should not be visible. With this PR, they only show relevant fields.
This fixes#75297.
[Here's an example code.](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=0c79356ed860e347a0c1a205616f93b7) This prints this on nightly:
```
ValuesMut { inner: IterMut { range: [(1, "hello"), (2, "goodbye")], length: 2 } }
IntoKeys { inner: [(1, "hello"), (2, "goodbye")] }
IntoValues { inner: [(1, "hello"), (2, "goodbye")] }
[(2, "goodbye"), (1, "hello")]
```
After the patch this example prints these instead:
```
["hello", "goodbye"]
["hello", "goodbye"]
[1, 2]
["hello", "goodbye"]
```
I didn't add test cases for them, since I couldn't see any tests for Debug implementations anywhere. But please let me know if I should add it to a specific place.
r? @dtolnay
Use posix_spawn on musl targets
The posix_spawn had been available in a form suitable for use in a
Command implementation since musl 0.9.12. Use it in a preference to a
fork when possible, to benefit from CLONE_VM|CLONE_VFORK used there.
Previously `Command::spawn` would fall back to the non-posix_spawn based
implementation if the `PATH` environment variable was possibly changed.
On systems with a modern (g)libc `posix_spawn()` can be significantly
faster. If program is a path itself the `PATH` environment variable is
not used for the lookup and it should be safe to use the
`posix_spawnp()` method. [1]
We found this, because we have a cli application that effectively runs a
lot of subprocesses. It would sometimes noticeably hang while printing
output. Profiling showed that the process was spending the majority of
time in the kernel's `copy_page_range` function while spawning
subprocesses. During this time the process is completely blocked from
running, explaining why users were reporting the cli app hanging.
Through this we discovered that `std::process::Command` has a fast and
slow path for process execution. The fast path is backed by
`posix_spawnp()` and the slow path by fork/exec syscalls being called
explicitly. Using fork for process creation is supposed to be fast, but
it slows down as your process uses more memory. It's not because the
kernel copies the actual memory from the parent, but it does need to
copy the references to it (see `copy_page_range` above!). We ended up
using the slow path, because the command spawn implementation in falls
back to the slow path if it suspects the PATH environment variable was
changed.
Here is a smallish program demonstrating the slowdown before this code
change:
```
use std::process::Command;
use std::time::Instant;
fn main() {
let mut args = std::env::args().skip(1);
if let Some(size) = args.next() {
// Allocate some memory
let _xs: Vec<_> = std::iter::repeat(0)
.take(size.parse().expect("valid number"))
.collect();
let mut command = Command::new("/bin/sh");
command
.arg("-c")
.arg("echo hello");
if args.next().is_some() {
println!("Overriding PATH");
command.env("PATH", std::env::var("PATH").expect("PATH env var"));
}
let now = Instant::now();
let child = command
.spawn()
.expect("failed to execute process");
println!("Spawn took: {:?}", now.elapsed());
let output = child.wait_with_output().expect("failed to wait on process");
println!("Output: {:?}", output);
} else {
eprintln!("Usage: prog [size]");
std::process::exit(1);
}
()
}
```
Running it and passing different amounts of elements to use to allocate
memory shows that the time taken for `spawn()` can differ quite
significantly. In latter case the `posix_spawnp()` implementation is 30x
faster:
```
$ cargo run --release 10000000
...
Spawn took: 324.275µs
hello
$ cargo run --release 10000000 changepath
...
Overriding PATH
Spawn took: 2.346809ms
hello
$ cargo run --release 100000000
...
Spawn took: 387.842µs
hello
$ cargo run --release 100000000 changepath
...
Overriding PATH
Spawn took: 13.434677ms
hello
```
[1]: 5f72f9800b/posix/execvpe.c (L81)
Add accessors to Command.
This adds some accessor methods to `Command` to provide a way to access the values set when building the `Command`. An example where this can be useful is to display the command to be executed. This is roughly based on the [`ProcessBuilder`](13b73cdaf7/src/cargo/util/process_builder.rs (L105-L134)) in Cargo.
Possible concerns about the API:
- Values with NULs on Unix will be returned as `"<string-with-nul>"`. I don't think it is practical to avoid this, since otherwise a whole separate copy of all the values would need to be kept in `Command`.
- Does not handle `arg0` on Unix. This can be awkward to support in `get_args` and is rarely used. I figure if someone really wants it, it can be added to `CommandExt` as a separate method.
- Does not offer a way to detect `env_clear`. I'm uncertain if it would be useful for anyone.
- Does not offer a way to get an environment variable by name (`get_env`). I figure this can be added later if anyone really wants it. I think the motivation for this is weak, though. Also, the API could be a little awkward (return a `Option<Option<&OsStr>>`?).
- `get_envs` could skip "cleared" entries and just return `&OsStr` values instead of `Option<&OsStr>`. I'm on the fence here. My use case is to display a shell command, and I only intend it to be roughly equivalent to the actual execution, and I probably won't display `None` entries. I erred on the side of providing extra information, but I suspect many situations will just filter out the `None`s.
- Could implement more iterator stuff (like `DoubleEndedIterator`).
I have not implemented new std items before, so I'm uncertain if the existing issue should be reused, or if a new tracking issue is needed.
cc #44434
Fix is_absolute on WASI
WASI does not match `cfg(unix)`, but its paths are Unix-like (`/some/path`) and don't have Windows-like prefixes.
Without this change, `is_absolute` for any paths, including `/some/path`, was returning `false`on a WASI target, which is obviously not true and undesirable.
Split sys_common::Mutex in StaticMutex and MovableMutex.
The (unsafe) `Mutex` from `sys_common` had a rather complicated interface. You were supposed to call `init()` manually, unless you could guarantee it was neither moved nor used reentrantly.
Calling `destroy()` was also optional, although it was unclear if 1) resources might be leaked or not, and 2) if `destroy()` should only be called when `init()` was called.
This allowed for a number of interesting (confusing?) different ways to use this `Mutex`, all captured in a single type.
In practice, this type was only ever used in two ways:
1. As a static variable. In this case, neither `init()` nor `destroy()` are called. The variable is never moved, and it is never used reentrantly. It is only ever locked using the `LockGuard`, never with `raw_lock`.
2. As a `Box`ed variable. In this case, both `init()` and `destroy()` are called, it will be moved and possibly used reentrantly.
No other combinations are used anywhere in `std`.
This change simplifies things by splitting this `Mutex` type into two types matching the two use cases: `StaticMutex` and `MovableMutex`.
The interface of both new types is now both safer and simpler. The first one does not call nor expose `init`/`destroy`, and the second one calls those automatically in its `new()` and `Drop` functions. Also, the locking functions of `MovableMutex` are no longer unsafe.
---
This will also make it easier to conditionally box mutexes later, by moving that decision into sys/sys_common. Some of the mutex implementations (at least those of Wasm and 'sys/unsupported') are safe to move, so wouldn't need a box. ~~(But that's blocked on #76932 for now.)~~ (See #77380.)
Improve std::sys::windows::compat
Improves the compat_fn macro in sys::windows, which is used for conditionally loading APIs that might not be available.
- The module (dll) name can now be any string, not just an ident. (Not all Windows api modules are valid Rust identifiers. E.g. `WaitOnAddress` comes from `API-MS-Win-Core-Synch-l1-2-0.dll`.)
- Adds `FuncName::is_available()` for checking if a function is really available without having to do a duplicate lookup.
- Add comment explaining the lack of locking.
- Use `$_:block` to simplify the macro_rules.
- Apply `allow(unused_variables)` only to the fallback instead of everything.
---
The second point (`is_available()`) simplifies code that needs to pick an implementation depening on what is available, like `sys/windows/mutex.rs`. Before this change, it'd do its own lookup and keep its own `AtomicUsize` to track the result. Now it can just use `c::AcquireSRWLockExclusive::is_available()` directly.
This will also be useful when park/unpark/CondVar/etc. get improved implementations (e.g. from parking_lot or something else), as the best APIs for those are not available before Windows 8.
Make RawFd implement the RawFd traits
This PR makes `RawFd` implement `AsRawFd`, `IntoRawFd` and `FromRawFd`, so it can be passed to interfaces that use one of those traits as a bound.
- Module name can now be any string, not just an ident.
(Not all Windows api modules are valid Rust identifiers.)
- Adds c::FuncName::is_available() for checking if a function is really
available without having to do a duplicate lookup.
- Add comment explaining the lack of locking.
- Use `$_:block` to simplify the macro_rules.
- Apply allow(unused_variables) only to the fallback instead of
everything.
Use futex-based thread::park/unpark on Linux.
This moves the parking/unparking logic out of `thread/mod.rs` into a module named `thread_parker` in `sys_common`. The current implementation is moved to `sys_common/thread_parker/generic.rs` and the new implementation using futexes is added in `sys_common/thread_parker/futex.rs`.
The posix_spawn had been available in a form suitable for use in a
Command implementation since musl 0.9.12. Use it in a preference to a
fork when possible, to benefit from CLONE_VM|CLONE_VFORK used there.
Use `rtassert!` instead of `assert!` from the child process after fork() in std::sys::unix::process::Command::spawn()
As discussed in #73894, `assert!` panics on failure, which is not signal-safe, and `rtassert!` is a suitable replacement.
Fixes#73894.
r? @Amanieu @cuviper @joshtriplett
library: Forward compiler-builtins "mem" feature
This fixes https://github.com/rust-lang/wg-cargo-std-aware/issues/53
Now users will be able to do:
```
cargo build -Zbuild-std=core -Zbuild-std-features=compiler-builtins-mem
```
and correctly get the Rust implemenations for `memcpy` and friends.
Signed-off-by: Joe Richey <joerichey@google.com>
WASI does not match `cfg(unix)`, but its paths are Unix-like (`/some/path`) and don't have Windows-like prefixes.
Without this change, `is_absolute` for paths like `/some/path` was returning `false`on a WASI target, which is obviously not true and undesirable.
Remove `#[rustc_allow_const_fn_ptr]` and add `#![feature(const_fn_fn_ptr_basics)]`
`rustc_allow_const_fn_ptr` was a hack to work around the lack of an escape hatch for the "min `const fn`" checks in const-stable functions. Now that we have co-opted `allow_internal_unstable` for this purpose, we no longer need a bespoke attribute.
Now this functionality is gated under `const_fn_fn_ptr_basics` (how concise!), and `#[allow_internal_unstable(const_fn_fn_ptr_basics)]` replaces `#[rustc_allow_const_fn_ptr]`. `const_fn_fn_ptr_basics` allows function pointer types to appear in the arguments and locals of a `const fn` as well as function pointer casts to be performed inside a `const fn`. Both of these were allowed in constants and statics already. Notably, this does **not** allow users to invoke function pointers in a const context. Presumably, we will use a nicer name for that (`const_fn_ptr`?).
r? @oli-obk
UI to unit test for those using Cell/RefCell/UnsafeCell
Helps with #76268.
I'm working on all files using `Cell` and moving them to unit tests when possible.
r? @matklad
Add missing definitions required by the sparc-unknown-linux-gnu target
This PR adds a few missing definitions required by sparc-unknown-linux-target which were discovered during build tests.
This fixes https://github.com/rust-lang/wg-cargo-std-aware/issues/53
Now users will be able to do:
```
cargo build -Zbuild-std=core -Zbuild-std-features=compiler-builtins-mem
```
and correctly get the Rust implemenations for `memcpy` and friends.
Signed-off-by: Joe Richey <joerichey@google.com>
The syscalls returning a new file descriptors generally use
lowest-numbered file descriptor not currently opened, without any
exceptions for those corresponding to the standard streams.
Previously when any of standard streams has been closed before starting
the application, operations on std::io::{stderr,stdin,stdout} objects
were likely to operate on other logically unrelated file resources
opened afterwards.
Avoid the issue by reopening the standard streams when they are closed.
The thread local LOCAL_STDOUT and LOCAL_STDERR are only used by the test
crate to capture output from tests when running them in the same process
in differen threads. However, every program will check these variables
on every print, even outside of testing.
This involves allocating a thread local key, and registering a thread
local destructor. This can be somewhat expensive.
This change keeps a global flag (LOCAL_STREAMS) which will be set to
true when either of these local streams is used. (So, effectively only
in test and benchmark runs.) When this flag is off, these thread locals
are not even looked at and therefore will not be initialized on the
first output on every thread, which also means no thread local
destructors will be registered.
The (unsafe) Mutex from sys_common had a rather complicated interface.
You were supposed to call init() manually, unless you could guarantee it
was neither moved nor used reentrantly.
Calling `destroy()` was also optional, although it was unclear if 1)
resources might be leaked or not, and 2) if destroy() should only be
called when `init()` was called.
This allowed for a number of interesting (confusing?) different ways to
use this Mutex, all captured in a single type.
In practice, this type was only ever used in two ways:
1. As a static variable. In this case, neither init() nor destroy() are
called. The variable is never moved, and it is never used
reentrantly. It is only ever locked using the LockGuard, never with
raw_lock.
2. As a Boxed variable. In this case, both init() and destroy() are
called, it will be moved and possibly used reentrantly.
No other combinations are used anywhere in `std`.
This change simplifies things by splitting this Mutex type into
two types matching the two use cases: StaticMutex and MovableMutex.
The interface of both new types is now both safer and simpler. The first
one does not call nor expose init/destroy, and the second one calls
those automatically in its new() and Drop functions. Also, the locking
functions of MovableMutex are no longer unsafe.
Remove std::io::lazy::Lazy in favour of SyncOnceCell
The (internal) std::io::lazy::Lazy was used to lazily initialize the stdout and stdin buffers (and mutexes). It uses atexit() to register a destructor to flush the streams on exit, and mark the streams as 'closed'. Using the stream afterwards would result in a panic.
Stdout uses a LineWriter which contains a BufWriter that will flush the buffer on drop. This one is important to be executed during shutdown, to make sure no buffered output is lost. It also forbids access to stdout afterwards, since the buffer is already flushed and gone.
Stdin uses a BufReader, which does not implement Drop. It simply forgets any previously read data that was not read from the buffer yet. This means that in the case of stdin, the atexit() function's only effect is making stdin inaccessible to the program, such that later accesses result in a panic. This is uncessary, as it'd have been safe to access stdin during shutdown of the program.
---
This change removes the entire io::lazy module in favour of SyncOnceCell. SyncOnceCell's fast path is much faster (a single atomic operation) than locking a sys_common::Mutex on every access like Lazy did.
However, SyncOnceCell does not use atexit() to drop the contained object during shutdown.
As noted above, this is not a problem for stdin. It simply means stdin is now usable during shutdown.
The atexit() call for stdout is moved to the stdio module. Unlike the now-removed Lazy struct, SyncOnceCell does not have a 'gone and unusable' state that panics. Instead of adding this again, this simply replaces the buffer with one with zero capacity. This effectively flushes the old buffer *and* makes any writes afterwards pass through directly without touching a buffer, making print!() available during shutdown without panicking.
---
In addition, because the contents of the SyncOnceCell are no longer dropped, we can now use `&'static` instead of `Arc` in `Stdout` and `Stdin`. This also saves two levels of indirection in `stdin()` and `stdout()`, since Lazy effectively stored a `Box<Arc<T>>`, and SyncOnceCell stores the `T` directly.
Add `#![feature(const_fn_floating_point_arithmetic)]`
cc #76618
This is a template for splitting up `const_fn` into granular feature gates. I think this will make it easier, both for us and for users, to track stabilization of each individual feature. We don't *have* to do this, however. We could also keep stabilizing things out from under `const_fn`.
cc @rust-lang/wg-const-eval
r? @oli-obk
Std/thread: deny unsafe op in unsafe fn
Partial fix of #73904.
This encloses `unsafe` operations in `unsafe fn` in `libstd/thread`.
`@rustbot` modify labels: F-unsafe-block-in-unsafe-fn
Relax promises about condition variable.
For quite a while now, there have been plans to at some point use parking_lot or some other more efficient implementation of mutexes and condition variables. Right now, Mutex and CondVar both Box the 'real' mutex/condvar inside, to give it a stable address. This was done because implementations like pthread and Windows critical sections may not be moved. More efficient implementations based on futexes, WaitOnAddress, Windows SRW locks, parking_lot, etc. may be moved (while not borrowed), so wouldn't need boxing.
However, not boxing them (which would be great goal to achieve), breaks a promise std currently makes about CondVar. CondVar promises to panic when used with different mutexes, to ensure consistent behaviour on all platforms. To this check, a mutex is considered 'the same' if the address of the 'real mutex' in the Box is the same. This address doesn't change when moving a `std::mutex::Mutex` object, effectively giving it an identity that survives moves of the Mutex object. If we ever switch to a non-boxed version, they no longer carry such an identity, and this check can no longer be made.
Four options:
1. Always box mutexes.
2. Add a `MutexId` similar to `ThreadId`. Making mutexes bigger, and making it hard to ever have a `const fn new` for them.
3. Making the requirement of CondVar stricter: panic if the Mutex object itself moved.
4. Making the promise of CondVar weaker: don't promise to panic.
1, 2, and 3 seem like bad options. This PR updates the documentation for 4.
Make delegation methods of `std::net::IpAddr` unstably const
Make the following methods of `std::net::IpAddr` unstable const under the `const_ip` feature:
- `is_unspecified`
- `is_loopback`
- `is_global`
- `is_multicast`
Also adds a test for these methods in a const context.
Possible because these methods delegate to the inner `Ipv4Addr` or `Ipv6Addr`, which were made const ([PR#76205](https://github.com/rust-lang/rust/pull/76142) and [PR#76206](https://github.com/rust-lang/rust/pull/76206)), and the recent stabilization of const control flow.
Part of #76205
r? @ecstatic-morse
The (internal) std::io::lazy::Lazy was used to lazily initialize the
stdout and stdin buffers (and mutexes). It uses atexit() to register a
destructor to flush the streams on exit, and mark the streams as
'closed'. Using the stream afterwards would result in a panic.
Stdout uses a LineWriter which contains a BufWriter that will flush the
buffer on drop. This one is important to be executed during shutdown,
to make sure no buffered output is lost. It also forbids access to
stdout afterwards, since the buffer is already flushed and gone.
Stdin uses a BufReader, which does not implement Drop. It simply forgets
any previously read data that was not read from the buffer yet. This
means that in the case of stdin, the atexit() function's only effect is
making stdin inaccessible to the program, such that later accesses
result in a panic. This is uncessary, as it'd have been safe to access
stdin during shutdown of the program.
---
This change removes the entire io::lazy module in favour of
SyncOnceCell. SyncOnceCell's fast path is much faster (a single atomic
operation) than locking a sys_common::Mutex on every access like Lazy
did.
However, SyncOnceCell does not use atexit() to drop the contained object
during shutdown.
As noted above, this is not a problem for stdin. It simply means stdin
is now usable during shutdown.
The atexit() call for stdout is moved to the stdio module. Unlike the
now-removed Lazy struct, SyncOnceCell does not have a 'gone and
unusable' state that panics. Instead of adding this again, this simply
replaces the buffer with one with zero capacity. This effectively
flushes the old buffer *and* makes any writes afterwards pass through
directly without touching a buffer, making print!() available during
shutdown without panicking.
Rollup of 9 pull requests
Successful merges:
- #76898 (Record `tcx.def_span` instead of `item.span` in crate metadata)
- #76939 (emit errors during AbstractConst building)
- #76965 (Add cfg(target_has_atomic_equal_alignment) and use it for Atomic::from_mut.)
- #76993 (Changing the alloc() to accept &self instead of &mut self)
- #76994 (fix small typo in docs and comments)
- #77017 (Add missing examples on Vec iter types)
- #77042 (Improve documentation for ToSocketAddrs)
- #77047 (Miri: more informative deallocation error messages)
- #77055 (Add #[track_caller] to more panicking Cell functions)
Failed merges:
r? `@ghost`
Make the following methods of `std::net::IpAddr` unstable const under the `const_ip` feature:
- `is_unspecified`
- `is_loopback`
- `is_global`
- `is_multicast`
Also adds a test for these methods in a const context.
Possible because these methods delegate to the inner `Ipv4Addr` or `Ipv6Addr`, which were made const, and the recent stabilization of const control flow.
Part of #76205
Function to convert OpenOptions to c_int
Fixes: #74943
The creation_mode and access_mode function were already available in the OpenOptions struct, but currently private. I've added a new free functions to unix/fs.rs which takes the OpenOptions, and returns the c_int to be used as parameter for the `open` call.
Remove duplicated library links between std and libc
The libc crate is already responsible for linking in the appropriate
libraries, and std doing the same thing results in duplicated library
names on the linker command line. Removing this duplication slightly
reduces linker time, and makes it simpler to adjust the set or order of
linked libraries in one place (such as to add static linking support).
