Support (stat/fstat/lstat)64 on macos
"In order to accommodate advanced capabilities of newer file systems,
the struct stat, struct statfs, and struct dirent data structures
were updated in Mac OSX 10.5."
"TRANSITIONAL DESCRIPTION (NOW DEPRECATED)
The fstat64, lstat64 and stat64 routines are equivalent to their
corresponding non-64-suffixed routine, when 64-bit inodes are in
effect. They were added before there was support for the symbol
variants, and so are now deprecated. Instead of using these, set
the _DARWIN_USE_64_BIT_INODE macro before including header files to
force 64-bit inode support. The stat64 structure used by these deprecated routines is the same
as the stat structure when 64-bit inodes are in effect (see above)."
"HISTORY
An lstat() function call appeared in 4.2BSD. The stat64(),
fstat64(), and lstat64() system calls first appeared in Mac OS X
10.5 (Leopard) and are now deprecated in favor of the corresponding
symbol variants. The fstatat() system call appeared in OS X 10.10"
"In order to accommodate advanced capabilities of newer file systems,
the struct stat, struct statfs, and struct dirent data structures
were updated in Mac OSX 10.5."
"TRANSITIONAL DESCRIPTION (NOW DEPRECATED)
The fstat64, lstat64 and stat64 routines are equivalent to their
corresponding non-64-suffixed routine, when 64-bit inodes are in
effect. They were added before there was support for the symbol
variants, and so are now deprecated. Instead of using these, set
the _DARWIN_USE_64_BIT_INODE macro before including header files to
force 64-bit inode support.
The stat64 structure used by these deprecated routines is the same
as the stat structure when 64-bit inodes are in effect (see above)."
"HISTORY
An lstat() function call appeared in 4.2BSD. The stat64(),
fstat64(), and lstat64() system calls first appeared in Mac OS X
10.5 (Leopard) and are now deprecated in favor of the corresponding
symbol variants. The fstatat() system call appeared in OS X 10.10"
This adds a very simple LRU-like cache which stores the locations of
often-used tags. While the implementation is very simple, the cache hit
rate is incredible at ~99.9% on most programs, and often the element at
position 0 in the cache has a hit rate of 90%. So the sub-optimality of
this cache basicaly vanishes into the noise in a profile.
Additionally, we keep a range which denotes where there might be an item
granting Unique permission in the stack, so that when we invalidate
Uniques we do not need to scan much of the stack, and often scan nothing
at all.
Enable permissive provenance by default
This completes the plan laid out in https://github.com/rust-lang/miri/issues/2133:
- We use permissive provenance with wildcard pointers by default.
- We print a warning on int2ptr casts. `-Zmiri-permissive-provenance` suppresses the warning; `-Zmiri-strict-provenance` turns it into a hard error.
- Raw pointer tagging is now always enabled, so we remove the `-Zmiri-tag-raw-pointers` flag and the code for untagged pointers. (Passing the flag still works, for compatibility -- but we just ignore it, with a warning.)
We also fix an intptrcast issue:
- Only live allocations are considered when computing the AllocId from an address.
So, finally, Miri has a good story for ptr2int2ptr roundtrips *and* no weird false negatives when doing raw pointer stuff with Stacked Borrows. :-) 🎉 Thanks a lot to everyone who helped with this, in particular `@carbotaniuman` who convinced me this is even possible.
Fixes https://github.com/rust-lang/miri/issues/2133
Fixes https://github.com/rust-lang/miri/issues/1866
Fixes https://github.com/rust-lang/miri/issues/1993
Prevent futex_wait from actually waiting if a concurrent waker was executed before us
Fixes#2223
Two SC fences were placed in `futex_wake` (after the caller has changed `addr`), and in `futex_wait` (before we read `addr`). This guarantees that `futex_wait` sees the value written to `addr` before the last `futex_wake` call, should one exists, and avoid going into sleep with no one else to wake us up.
ada7b72a87/src/concurrency/weak_memory.rs (L324-L326)
Earlier I proposed to use `fetch_add(0)` to read the latest value in MO, though this isn't the proper way to do it and breaks aliasing: syscall caller may pass in a `*const` from a `&` and Miri complains about write to a `SharedReadOnly` location, causing this test to fail.
ada7b72a87/tests/pass/concurrency/linux-futex.rs (L56-L68)
make Miri's scheduler proper round-robin
When thread N blocks or yields, we activate thread N+1 next, rather than always activating thread 0. This should guarantee that as long as all threads regularly yield, each thread eventually takes a step again.
Fixes the "multiple loops that yield playing ping-pong" part of https://github.com/rust-lang/miri/issues/1388.
`@cbeuw` I hope this doesn't screw up the scheduler-dependent tests you are adding in your PR.
run Clippy on CI
and fix some things it complains about. Also use `rustup-toolchain` script on CI (reduces code duplication, and good thing to make sure it keeps working, since we recommend it in the docs).
I left `ui_test` out for now; I'll leave those nits to `@oli-obk.` ;)
Save a created event for zero-size reborrows
Currently, we don't save a created event for zero-sized reborrows. Attempting to use something from a zero-sized reborrow is surprisingly common, for example on `minimal-lexical==0.2.1` we previously just emit this:
```
Undefined Behavior: attempting a write access using <187021> at alloc72933[0x0], but that tag does not exist in the borrow stack for this location
--> /root/rust/library/core/src/ptr/mod.rs:1287:9
|
1287 | copy_nonoverlapping(&src as *const T, dst, 1);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| attempting a write access using <187021> at alloc72933[0x0], but that tag does not exist in the borrow stack for this location
| this error occurs as part of an access at alloc72933[0x0..0x8]
|
= help: this indicates a potential bug in the program: it performed an invalid operation, but the rules it violated are still experimental
= help: see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md for further information
= note: inside `std::ptr::write::<u64>` at /root/rust/library/core/src/ptr/mod.rs:1287:9
note: inside `minimal_lexical::stackvec::StackVec::push_unchecked` at /root/build/src/stackvec.rs:82:13
--> /root/build/src/stackvec.rs:82:13
|
82 | ptr::write(self.as_mut_ptr().add(self.len()), value);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
... backtrace continues...
```
Which leaves us with the question "where did we make this pointer?" because for every other diagnostic you get a "was created by" note, so I suspect people might be tempted to think there is a Miri bug here. I certainly was.
---
This code duplication is so awful, I'm going to take a look at cleaning it up later. The fact that `ptr_get_alloc_id` can fail in this situation makes things annoying.
Initial work on Miri permissive-exposed-provenance
Miri portions of the changes for portions of a permissive ptr-to-int model for Miri. This is more restrictive than what we currently have so it will probably need a flag once I figure out how to hook that up.
> This implements a form of permissive exposed-address provenance, wherein the only way to expose the address is with a cast to usize (ideally expose_addr). This is more restrictive than C in that stuff like reading the representation bytes (via unions, type-punning, transmute) does not expose the address, only expose_addr. This is less restrictive than C in that a pointer casted from an integer has union provenance of all exposed pointers, not any udi stuff.
There's a few TODOs here, namely related to `fn memory_read` and friends. We pass it the maybe/unreified provenance before `ptr_get_alloc` reifies it into a concrete one, so it doesn't have the `AllocId` (or the SB tag, but that's getting ahead of ourselves). One way this could be fixed is changing `ptr_get_alloc` and (`ptr_try_get_alloc_id` on the rustc side) to return a pointer with the tag fixed up. We could also take in different arguments, but I'm not sure what works best.
The other TODOs here are how permissive this model could be. This currently does not enforce that a ptr-to-int cast happens before the corresponding int-to-ptr (colloquial meaning of happens before, not atomic meaning). Example:
```
let ptr = 0x2000 as *const i32;
let a: i32 = 5;
let a_ptr = &a as *const i32;
// value is 0x2000;
a_ptr as usize;
println!("{}", unsafe { *ptr }); // this is valid
```
We also allow the resulting pointer to dereference different non-contiguous allocations (the "not any udi stuff" mentioned above), which I'm not sure if is allowed by LLVM.
This is the Miri side of https://github.com/rust-lang/rust/pull/95826.
