Use posix_spawn() on unix if program is a path
Previously `Command::spawn` would fall back to the non-posix_spawn based
implementation if the `PATH` environment variable was possibly changed.
On systems with a modern (g)libc `posix_spawn()` can be significantly
faster. If program is a path itself the `PATH` environment variable is
not used for the lookup and it should be safe to use the
`posix_spawnp()` method. [1]
We found this, because we have a cli application that effectively runs a
lot of subprocesses. It would sometimes noticeably hang while printing
output. Profiling showed that the process was spending the majority of
time in the kernel's `copy_page_range` function while spawning
subprocesses. During this time the process is completely blocked from
running, explaining why users were reporting the cli app hanging.
Through this we discovered that `std::process::Command` has a fast and
slow path for process execution. The fast path is backed by
`posix_spawnp()` and the slow path by fork/exec syscalls being called
explicitly. Using fork for process creation is supposed to be fast, but
it slows down as your process uses more memory. It's not because the
kernel copies the actual memory from the parent, but it does need to
copy the references to it (see `copy_page_range` above!). We ended up
using the slow path, because the command spawn implementation in falls
back to the slow path if it suspects the PATH environment variable was
changed.
Here is a smallish program demonstrating the slowdown before this code
change:
```
use std::process::Command;
use std::time::Instant;
fn main() {
let mut args = std::env::args().skip(1);
if let Some(size) = args.next() {
// Allocate some memory
let _xs: Vec<_> = std::iter::repeat(0)
.take(size.parse().expect("valid number"))
.collect();
let mut command = Command::new("/bin/sh");
command
.arg("-c")
.arg("echo hello");
if args.next().is_some() {
println!("Overriding PATH");
command.env("PATH", std::env::var("PATH").expect("PATH env var"));
}
let now = Instant::now();
let child = command
.spawn()
.expect("failed to execute process");
println!("Spawn took: {:?}", now.elapsed());
let output = child.wait_with_output().expect("failed to wait on process");
println!("Output: {:?}", output);
} else {
eprintln!("Usage: prog [size]");
std::process::exit(1);
}
()
}
```
Running it and passing different amounts of elements to use to allocate
memory shows that the time taken for `spawn()` can differ quite
significantly. In latter case the `posix_spawnp()` implementation is 30x
faster:
```
$ cargo run --release 10000000
...
Spawn took: 324.275µs
hello
$ cargo run --release 10000000 changepath
...
Overriding PATH
Spawn took: 2.346809ms
hello
$ cargo run --release 100000000
...
Spawn took: 387.842µs
hello
$ cargo run --release 100000000 changepath
...
Overriding PATH
Spawn took: 13.434677ms
hello
```
[1]: 5f72f9800b/posix/execvpe.c (L81)
Set .llvmbc and .llvmcmd sections as allocatable
This marks both sections as allocatable rather than excluded, which matches what
clang does with the equivalent `-fembed-bitcode` flag.
BTreeMap: improve gdb introspection of BTreeMap with ZST keys or values
I accidentally pushed an earlier revision in #77788: it changes the index of tuples for BTreeSet from ""[{}]".format(i) to "key{}".format(i). Which doesn't seem to make the slightest difference on my linux box nor on CI. In fact, gdb doesn't make any distinction between "key{}" and "val{}" for a BTreeMap either, leading to confusing output if you test more. But easy to improve.
r? @Mark-Simulacrum
liballoc: VecDeque: Add binary search functions
I am submitting rust-lang/rfcs#2997 as a PR as suggested by @scottmcm
I haven't yet created a tracking issue - if there's a favorable feedback I'll create one and update the issue links in the unstable attribs.
Permit uninhabited enums to cast into ints
This essentially reverts part of #6204; it is unclear why that [commit](c0f587de34) was introduced, and I suspect no one remembers.
The changed code was only called from casting checks and appears to not affect any callers of that code (other than permitting this one case).
Fixes#75647.
Remove shrink_to_fit from default ToString::to_string implementation.
As suggested by `@scottmcm` on Zulip. shrink_to_fit() seems like the wrong thing to do here in most use cases of to_string(). Would be intereseting to see if it makes any difference in a timer run.
r? `@joshtriplett`
Rollup of 10 pull requests
Successful merges:
- #75209 (Suggest imports of unresolved macros)
- #77547 (stabilize union with 'ManuallyDrop' fields and 'impl Drop for Union')
- #77827 (Don't link to nightly primitives on stable channel)
- #77855 (resolve: further improvements to "try using the enum's variant" diagnostic)
- #77900 (Use fdatasync for File::sync_data on more OSes)
- #77925 (Suggest minimal subset features in `incomplete_features` lint)
- #77971 (Deny broken intra-doc links in linkchecker)
- #77991 (Bump backtrace-rs)
- #77992 (instrument-coverage: try our best to not ICE)
- #78013 (Fix sidebar scroll on mobile devices)
Failed merges:
r? `@ghost`
Fix sidebar scroll on mobile devices
Fixes#77942.
The issue was coming from the appearance/disappearance of the "wrapper" on the mobile devices web browsers, which triggers the "resize" event, calling the `hideSidebar` function is the JS code.
r? @jyn514
instrument-coverage: try our best to not ICE
instrument-coverage was ICEing for me on some code, in particular code
that had devirtualized paths from standard library. Instrument coverage
probably has no bussiness dictating which paths are valid and which
aren't so just feed it everything and whatever and let tooling deal with
other stuff.
For example, with this commit we can generate coverage hitpoints for
these interesting paths:
* `/rustc/.../library/core/lib.rs` – non-devirtualized path for libcore
* `/home/.../src/library/core/lib.rs` – devirtualized version of above
* `<inline asm>`, `<anon>` and many similar synthetic paths
Even if those paths somehow get to the instrumentation pass, I'd much
rather get hits for these weird paths and hope some of them work (as
would be the case for devirtualized path to libcore), rather than have
compilation fail entirely.
Deny broken intra-doc links in linkchecker
Since rustdoc isn't warning about these links, check for them manually.
This also fixes the broken links that popped up from the lint.
Suggest minimal subset features in `incomplete_features` lint
This tells users that we have a minimal subset feature of it and they can fix the lint warning without allowing it.
The wording improvement is helpful :)
Fixes#77913
resolve: further improvements to "try using the enum's variant" diagnostic
Follow-up on https://github.com/rust-lang/rust/pull/77341#issuecomment-702738281.
This PR improves the diagnostic modified in #77341 to suggest not only those variants which do not have fields, but those with fields (by suggesting with placeholders). In addition, the wording of the tuple-variant-only case is improved slightly.
I've not made further changes to the tuple-variant-only case (e.g. to only suggest variants with the correct number of fields) because I don't think I have enough information to do so reliably (e.g. in the case where there is an attempt to construct a tuple variant, I have no information on how many fields were provided; and in the case of pattern matching, I only have a slice of spans and would need to check for things like `..` in those spans, which doesn't seem worth it).
r? @estebank
stabilize union with 'ManuallyDrop' fields and 'impl Drop for Union'
As [discussed by @SimonSapin and @withoutboats](https://github.com/rust-lang/rust/issues/55149#issuecomment-634692020), this PR proposes to stabilize parts of the `untagged_union` feature gate:
* It will be possible to have a union with field type `ManuallyDrop<T>` for any `T`.
* While at it I propose we also stabilize `impl Drop for Union`; to my knowledge, there are no open concerns around this feature.
In the RFC discussion, we also talked about allowing `&mut T` as another non-`Copy` non-dropping type, but that felt to me like an overly specific exception so I figured we'd wait if there is actually any use for such a special case.
Some things remain unstable and still require the `untagged_union` feature gate:
* Union with fields that do not drop, are not `Copy`, and are not `ManuallyDrop<_>`. The reason to not stabilize this is to avoid semver concerns around libraries adding `Drop` implementations later. (This is already not fully semver compatible as, to my knowledge, the borrow checker will exploit the non-dropping nature of any type, but it seems prudent to avoid further increasing the amount of trouble adding an `impl Drop` can cause.)
Due to this, quite a few tests still need the `untagged_union` feature, but I think the ones where I could remove the feature flag provide good test coverage for the stable part.
Cc @rust-lang/lang
instrument-coverage was ICEing for me on some code, in particular code
that had devirtualized paths from standard library. Instrument coverage
probably has no bussiness dictating which paths are valid and which
aren't so just feed it everything and whatever and let tooling deal with
other stuff.
For example, with this commit we can generate coverage hitpoints for
these interesting paths:
* `/rustc/.../library/core/lib.rs` – non-devirtualized path for libcore
* `/home/.../src/library/core/lib.rs` – devirtualized version of above
* `<inline asm>`, `<anon>` and many similar synthetic paths
Even if those paths somehow get to the instrumentation pass, I'd much
rather get hits for these weird paths and hope some of them work (as
would be the case for devirtualized path to libcore), rather than have
compilation fail entirely.
Prevent miscompilation in trivial loop {}
Ideally, we would want to handle a broader set of cases to fully fix the
underlying bug here. That is currently relatively expensive at compile and
runtime, so we don't do that for now.
Performance results indicate this is not a major regression, if at all, so it should be safe to land.
cc #28728
