The issue of passing around SIMD types as values between functions has
seen [quite a lot] of [discussion], and although we thought [we fixed
it][quite a lot] it [wasn't]! This PR is a change to rustc to, again,
try to fix this issue.
The fundamental problem here remains the same, if a SIMD vector argument
is passed by-value in LLVM's function type, then if the caller and
callee disagree on target features a miscompile happens. We solve this
by never passing SIMD vectors by-value, but LLVM will still thwart us
with its argument promotion pass to promote by-ref SIMD arguments to
by-val SIMD arguments.
This commit is an attempt to thwart LLVM thwarting us. We, just before
codegen, will take yet another look at the LLVM module and demote any
by-value SIMD arguments we see. This is a very manual attempt by us to
ensure the codegen for a module keeps working, and it unfortunately is
likely producing suboptimal code, even in release mode. The saving grace
for this, in theory, is that if SIMD types are passed by-value across
a boundary in release mode it's pretty unlikely to be performance
sensitive (as it's already doing a load/store, and otherwise
perf-sensitive bits should be inlined).
The implementation here is basically a big wad of C++. It was largely
copied from LLVM's own argument promotion pass, only doing the reverse.
In local testing this...
Closes#50154Closes#52636Closes#54583Closes#55059
[quite a lot]: https://github.com/rust-lang/rust/pull/47743
[discussion]: https://github.com/rust-lang/rust/issues/44367
[wasn't]: https://github.com/rust-lang/rust/issues/50154
Prefer unwrap_or_else to unwrap_or in case of function calls/allocations
The contents of `unwrap_or` are evaluated eagerly, so it's not a good pick in case of function calls and allocations. This PR also changes a few `unwrap_or`s with `unwrap_or_default`.
An added bonus is that in some cases this change also reveals if the object it's called on is an `Option` or a `Result` (based on whether the closure takes an argument).
The issue of passing around SIMD types as values between functions has
seen [quite a lot] of [discussion], and although we thought [we fixed
it][quite a lot] it [wasn't]! This PR is a change to rustc to, again,
try to fix this issue.
The fundamental problem here remains the same, if a SIMD vector argument
is passed by-value in LLVM's function type, then if the caller and
callee disagree on target features a miscompile happens. We solve this
by never passing SIMD vectors by-value, but LLVM will still thwart us
with its argument promotion pass to promote by-ref SIMD arguments to
by-val SIMD arguments.
This commit is an attempt to thwart LLVM thwarting us. We, just before
codegen, will take yet another look at the LLVM module and demote any
by-value SIMD arguments we see. This is a very manual attempt by us to
ensure the codegen for a module keeps working, and it unfortunately is
likely producing suboptimal code, even in release mode. The saving grace
for this, in theory, is that if SIMD types are passed by-value across
a boundary in release mode it's pretty unlikely to be performance
sensitive (as it's already doing a load/store, and otherwise
perf-sensitive bits should be inlined).
The implementation here is basically a big wad of C++. It was largely
copied from LLVM's own argument promotion pass, only doing the reverse.
In local testing this...
Closes#50154Closes#52636Closes#54583Closes#55059
[quite a lot]: https://github.com/rust-lang/rust/pull/47743
[discussion]: https://github.com/rust-lang/rust/issues/44367
[wasn't]: https://github.com/rust-lang/rust/issues/50154
Exit with code 101 on fatal codegen errors
Fixes#54992.
This PR installs a custom fatal error handler that prints the error from LLVM and exits with 101. There should be no visible change in the output from LLVM. This allows distinguishing a fatal LLVM error with a compilation error by exit code.
This PR also modifies the LLVM codegen backend to ICE instead of emitting a fatal error when encountering a LLVM worker thread panic for the same reason.
r? @cuviper
rustc: Allow targets to specify SIMD args are by-val
The upcoming SIMD support in the wasm target is unique from the other
platforms where it's either unconditionally available or not available,
there's no halfway where a subsection of the program can use it but no
other parts of the program can use it. In this world it's valid for wasm
SIMD args to always be passed by value and there's no need to pass them
by reference.
This commit adds a new custom target specification option
`simd_types_indirect` which defaults to `true`, but the wasm backend
disables this and sets it to `false`.
The upcoming SIMD support in the wasm target is unique from the other
platforms where it's either unconditionally available or not available,
there's no halfway where a subsection of the program can use it but no
other parts of the program can use it. In this world it's valid for wasm
SIMD args to always be passed by value and there's no need to pass them
by reference.
This commit adds a new custom target specification option
`simd_types_indirect` which defaults to `true`, but the wasm backend
disables this and sets it to `false`.
Support for disabling PLT for better function call performance
This PR gives `rustc` the ability to skip the PLT when generating function calls into shared libraries. This can improve performance by reducing branch indirection.
AFAIK, the only advantage of using the PLT is to allow for ELF lazy binding. However, since Rust already [enables full relro for security](https://github.com/rust-lang/rust/pull/43170), lazy binding was disabled anyway.
This is a little known feature which is supported by [GCC](https://gcc.gnu.org/onlinedocs/gcc/Code-Gen-Options.html) and [Clang](https://clang.llvm.org/docs/ClangCommandLineReference.html#cmdoption-clang-fplt) as `-fno-plt` (some Linux distros [enable it by default](https://git.archlinux.org/svntogit/packages.git/tree/trunk/makepkg.conf?h=packages/pacman#n40) for all builds).
Implementation inspired by [this patch](https://reviews.llvm.org/D39079#change-YvkpNDlMs_LT) which adds `-fno-plt` support to Clang.
## Performance
I didn't run a lot of benchmarks, but these are the results on my machine for a `clap` [benchmark](https://github.com/clap-rs/clap/blob/master/benches/05_ripgrep.rs):
```
name control ns/iter no-plt ns/iter diff ns/iter diff % speedup
build_app_long 11,097 10,733 -364 -3.28% x 1.03
build_app_short 11,089 10,742 -347 -3.13% x 1.03
build_help_long 186,835 182,713 -4,122 -2.21% x 1.02
build_help_short 80,949 78,455 -2,494 -3.08% x 1.03
parse_clean 12,385 12,044 -341 -2.75% x 1.03
parse_complex 19,438 19,017 -421 -2.17% x 1.02
parse_lots 431,493 421,421 -10,072 -2.33% x 1.02
```
A small performance improvement across the board, with no downsides. It's likely binaries which make a lot of function calls into dynamic libraries could see even more improvements. [This comment](https://patchwork.ozlabs.org/patch/468993/#1028255) suggests that, in some cases, `-fno-plt` could improve PIC/PIE code performance by 10%.
## Security benefits
**Bonus**: some of the speculative execution attacks rely on the PLT, by disabling it we reduce a big attack surface and reduce the need for [`retpoline`](https://reviews.llvm.org/D41723).
## Remaining PLT calls
The compiled binaries still have plenty of PLT calls, coming from C/C++ libraries. Building dependencies with `CFLAGS=-fno-plt CXXFLAGS=-fno-plt` removes them.
Disable the PLT where possible to improve performance
for indirect calls into shared libraries.
This optimization is enabled by default where possible.
- Add the `NonLazyBind` attribute to `rustllvm`:
This attribute informs LLVM to skip PLT calls in codegen.
- Disable PLT unconditionally:
Apply the `NonLazyBind` attribute on every function.
- Only enable no-plt when full relro is enabled:
Ensures we only enable it when we have linker support.
- Add `-Z plt` as a compiler option
codegen_llvm: verify that inline assembly operands are scalars
Another set of inline assembly fixes. This time let's emit an error message when the operand value cannot be coerced into the operand constraint.
Two questions:
1) Should I reuse `E0668` which was introduced in #54568 or just use `E0669` as it stands because they do mean different things, but maybe that's not too user-friendly. Just a thought.
2) The `try_fold` returns the operand which failed to be converted into a scalar value, any suggestions on how to use that in the error message?
Thanks!
wasm: Explicitly export all symbols with LLD
This commit fixes an oddity on the wasm target where LTO can produce
working executables but plain old optimizations doesn't. The compiler
already knows what set of symbols it would like to export, but LLD only
discovers this list transitively through symbol visibilities. LLD may
not, however, always find all the symbols that we'd like to export.
For example if you depend on an rlib with a `#[no_mangle]` symbol, then
if you don't actually use anything from the rlib then the symbol won't
appear in the final artifact! It will appear, however, with LTO. This
commit attempts to rectify this situation by ensuring that all symbols
rustc would otherwise preserve through LTO are also preserved through
the linking process with LLD by default.
[NLL] Improve "borrow later used here" messages
* In the case of two conflicting borrows, the later used message says which borrow it's referring to
* If the later use is a function call (from the users point of view) say that the later use is for the call. Point just to the function.
r? @pnkfelix
Closes#48643
This commit fixes an oddity on the wasm target where LTO can produce
working executables but plain old optimizations doesn't. The compiler
already knows what set of symbols it would like to export, but LLD only
discovers this list transitively through symbol visibilities. LLD may
not, however, always find all the symbols that we'd like to export.
For example if you depend on an rlib with a `#[no_mangle]` symbol, then
if you don't actually use anything from the rlib then the symbol won't
appear in the final artifact! It will appear, however, with LTO. This
commit attempts to rectify this situation by ensuring that all symbols
rustc would otherwise preserve through LTO are also preserved through
the linking process with LLD by default.
do not normalize all non-scalar constants to a ConstValue::ScalarPair
We still need `ConstValue::ScalarPair` for match handling (matching slices and strings), but that will never see anything `Undef`. For non-fat-ptr `ScalarPair`, just point to the allocation like larger data structures do.
Fixes https://github.com/rust-lang/rust/issues/54387
r? @eddyb
Panic when using mem::uninitialized or mem::zeroed on an uninhabited type
All code by @japaric. This re-submits one half of https://github.com/rust-lang/rust/pull/53508. This is likely not the one that introduced the perf regression, but just to be sure I'll do a perf run anyway.
rust: Add a `-C default-linker-libraries` option
This commit adds a new codegen option for the compiler which disables
rustc's passing of `-nodefaultlibs` by default on relevant platforms.
Sometimes Rust is linked with C code which fails to link with
`-nodefaultlibs` and is unnecessarily onerous to get linking correctly
with `-nodefaultlibs`.
An example of this is that when you compile C code with sanitizers and
then pass `-fsanitize=address` to the linker, it's incompatible with
`-nodefaultlibs` also being passed to the linker.
In these situations it's easiest to turn off Rust's default passing of
`-nodefaultlibs`, which was more ideological to start with than
anything! Preserving the default is somewhat important but having this
be opt-in shouldn't cause any breakage.
Closes#54237
Do not put noalias annotations by default
This will be re-enabled sooner or later depending on results of further
investigation.
Fixes#54462
Beta backport is: #54640
r? @nikomatsakis
This commit adds a new codegen option for the compiler which disables
rustc's passing of `-nodefaultlibs` by default on relevant platforms.
Sometimes Rust is linked with C code which fails to link with
`-nodefaultlibs` and is unnecessarily onerous to get linking correctly
with `-nodefaultlibs`.
An example of this is that when you compile C code with sanitizers and
then pass `-fsanitize=address` to the linker, it's incompatible with
`-nodefaultlibs` also being passed to the linker.
In these situations it's easiest to turn off Rust's default passing of
`-nodefaultlibs`, which was more ideological to start with than
anything! Preserving the default is somewhat important but having this
be opt-in shouldn't cause any breakage.
Closes#54237
