Commit Graph

405 Commits

Author SHA1 Message Date
AngelicosPhosphoros
60208a0517 Tweak the threshold for chunked swapping
Thanks to 98892 for the tests I brought in here, as it demonstrated that 3×usize is currently suboptimal.
2024-01-19 23:00:34 -08:00
Nikita Popov
ce2d91dccd Directly use volatile_load intrinsic
This makes the test work if libstd is compiled with debug assertions.
2024-01-19 10:52:01 +01:00
Nikita Popov
7a0415ce37 Add codegen test for ScalarPair with i128 on LLVM 17 2024-01-19 10:52:01 +01:00
bors
bf2637f4e8 Auto merge of #119954 - scottmcm:option-unwrap-failed, r=WaffleLapkin
Split out `option::unwrap_failed` like we have `result::unwrap_failed`

...and like `option::expect_failed`
2024-01-16 15:32:39 +00:00
bors
1ead4761e9 Auto merge of #119878 - scottmcm:inline-always-unwrap, r=workingjubilee
Tune the inlinability of `unwrap`

Fixes #115463
cc `@thomcc`

This tweaks `unwrap` on ~~`Option` &~~ `Result` to be two parts:
- `#[inline(always)]` for checking the discriminant
- `#[cold]` for actually panicking

The idea here is that checking the discriminant on a `Result` ~~or `Option`~~ should always be trivial enough to be worth inlining, even in `opt-level=z`, especially compared to passing it to a function.

As seen in the issue and codegen test, this will hopefully help particularly for things like `.try_into().unwrap()`s that are actually infallible, but in a way that's only visible with the inlining.

EDIT: I've restricted this to `Result` to avoid combining effects
2024-01-15 09:20:46 +00:00
Scott McMurray
23483664a2 Split out option::unwrap_failed like we have result::unwrap_failed
...and like `option::expect_failed`
2024-01-14 12:45:01 -08:00
bors
2319be8e26 Auto merge of #119452 - AngelicosPhosphoros:make_nonzeroint_get_assume_nonzero, r=scottmcm
Add assume into `NonZeroIntX::get`

LLVM currently don't support range metadata for function arguments so it fails to optimize non zero integers using their invariant if they are provided using by-value function arguments.

Related to https://github.com/rust-lang/rust/issues/119422
Related to https://github.com/llvm/llvm-project/issues/76628
Related to https://github.com/rust-lang/rust/issues/49572
2024-01-12 20:18:04 +00:00
Scott McMurray
b858c591dd Tune the inlinability of Result::unwrap 2024-01-12 10:57:58 -08:00
The 8472
93b34a5ffa mark vec::IntoIter pointers as !nonnull 2024-01-07 03:44:04 +01:00
AngelicosPhosphoros
8f432d4ae6 Add assume into NonZeroIntX::get
LLVM currently don't support range metadata for function arguments so it fails to optimize non zero integers using their invariant if they are provided using by-value function arguments.

Related to https://github.com/rust-lang/rust/issues/119422
Related to https://github.com/llvm/llvm-project/issues/76628
Related to https://github.com/rust-lang/rust/issues/49572
2024-01-06 14:26:37 +01:00
bors
432fffa8af Auto merge of #118991 - nikic:scalar-pair, r=nagisa
Separate immediate and in-memory ScalarPair representation

Currently, we assume that ScalarPair is always represented using a two-element struct, both as an immediate value and when stored in memory.

This currently works fairly well, but runs into problems with https://github.com/rust-lang/rust/pull/116672, where a ScalarPair involving an i128 type can no longer be represented as a two-element struct in memory. For example, the tuple `(i32, i128)` needs to be represented in-memory as `{ i32, [3 x i32], i128 }` to satisfy alignment requirements. Using `{ i32, i128 }` instead will result in the second element being stored at the wrong offset (prior to LLVM 18).

Resolve this issue by no longer requiring that the immediate and in-memory type for ScalarPair are the same. The in-memory type will now look the same as for normal struct types (and will include padding filler and similar), while the immediate type stays a simple two-element struct type. This also means that booleans in immediate ScalarPair are now represented as i1 rather than i8, just like we do everywhere else.

The core change here is to llvm_type (which now treats ScalarPair as a normal struct) and immediate_llvm_type (which returns the two-element struct that llvm_type used to produce). The rest is fixing things up to no longer assume these are the same. In particular, this switches places that try to get pointers to the ScalarPair elements to use byte-geps instead of struct-geps.
2024-01-05 14:31:56 +00:00
Nikita Popov
3cd6cde0be Make test compatible with 32-bit as well 2024-01-05 11:45:57 +01:00
Matthias Krüger
c505d760a6
Rollup merge of #119555 - Kobzol:maybeuninit-rvo-codegen-test, r=nikic
Add codegen test for RVO on MaybeUninit

Codegen test for https://github.com/rust-lang/rust/issues/90595. Currently, this only works with `-Cpanic=abort`, but hopefully in the [future](https://www.npopov.com/2024/01/01/This-year-in-LLVM-2023.html#writable-and-dead_on_unwind) it should also work in the presence of panics.

r? ``@nikic``
2024-01-04 08:33:26 +01:00
Jakub Beránek
0c56ccff04
Add codegen test for RVO on MaybeUninit
Currently, this only works with `-Cpanic=abort`.
2024-01-03 21:18:07 +01:00
León Orell Valerian Liehr
fcec407f4a
Rollup merge of #119523 - maurer:fix-sparc-llvm-18, r=nikic
llvm: Allow `noundef` in codegen tests

LLVM 18 will automatically infer `noundef` in some situations. Adjust codegen tests to accept this.

See llvm/llvm-project#76553 for why `noundef` is being generated now.

``@rustbot`` label:+llvm-main
2024-01-03 16:08:32 +01:00
Matthew Maurer
ee86b1f84c llvm: Allow noundef in codegen tests
LLVM 18 will automatically infer `noundef` in some situations.
Adjust codegen tests to accept this.

See llvm/llvm-project#76553 for why `noundef` is being generated now.
2024-01-02 18:02:17 +00:00
Nikita Popov
8e64fc94d8 Address review comments 2024-01-02 15:03:14 +01:00
Camille GILLOT
6dfda0d32f Revert codegen test change. 2023-12-24 20:08:58 +00:00
Camille GILLOT
2837727471 Replace legacy ConstProp by GVN. 2023-12-24 20:08:57 +00:00
Camille GILLOT
a03c972816 Enable GVN by default. 2023-12-24 20:08:57 +00:00
Augie Fackler
58fdbd1479 tests: fix overaligned-constant to not over-specify getelementptr instr
On LLVM 18 we get slightly different arguments here, so it's easier to
just regex those away. The important details are all still asserted as I
understand things.

Fixes #119193.

@rustbot label: +llvm-main
2023-12-21 15:53:28 -05:00
bors
920e0051cf Auto merge of #119056 - cjgillot:codegen-overalign, r=wesleywiser
Tolerate overaligned MIR constants for codegen.

Fixes https://github.com/rust-lang/rust/issues/117761

cc `@saethlin`
2023-12-21 04:01:36 +00:00
bors
51c0db6a91 Auto merge of #106790 - the8472:rawvec-niche, r=scottmcm
add more niches to rawvec

Previously RawVec only had a single niche in its `NonNull` pointer. With this change it now has `isize::MAX` niches since half the value-space of the capacity field is never needed, we can't have a capacity larger than isize::MAX.
2023-12-20 02:19:10 +00:00
Camille GILLOT
503af0deb2 Fortify test. 2023-12-17 23:31:58 +00:00
Camille GILLOT
3ea5cfaa11 Tolerate overaligned MIR constants for codegen. 2023-12-17 22:56:42 +00:00
Nikita Popov
c2fd26a115 Separate immediate and in-memory ScalarPair representation
Currently, we assume that ScalarPair is always represented using
a two-element struct, both as an immediate value and when stored
in memory.

This currently works fairly well, but runs into problems with
https://github.com/rust-lang/rust/pull/116672, where a ScalarPair
involving an i128 type can no longer be represented as a two-element
struct in memory. For example, the tuple `(i32, i128)` needs to be
represented in-memory as `{ i32, [3 x i32], i128 }` to satisfy
alignment requirement. Using `{ i32, i128 }` instead will result in
the second element being stored at the wrong offset (prior to
LLVM 18).

Resolve this issue by no longer requiring that the immediate and
in-memory type for ScalarPair are the same. The in-memory type
will now look the same as for normal struct types (and will include
padding filler and similar), while the immediate type stays a
simple two-element struct type. This also means that booleans in
immediate ScalarPair are now represented as i1 rather than i8,
just like we do everywhere else.

The core change here is to llvm_type (which now treats ScalarPair
as a normal struct) and immediate_llvm_type (which returns the
two-element struct that llvm_type used to produce). The rest is
fixing things up to no longer assume these are the same. In
particular, this switches places that try to get pointers to the
ScalarPair elements to use byte-geps instead of struct-geps.
2023-12-15 17:42:05 +01:00
Wesley Wiser
ce290514df
Adapt debug-accessibility tests for msvc-style enums 2023-12-15 11:45:03 +00:00
David Wood
07931c5a08
codegen_llvm: set DW_AT_accessibility
Sets the accessibility of types and fields in DWARF using
`DW_AT_accessibility` attribute.

`DW_AT_accessibility` (public/protected/private) isn't exactly right for
Rust,  but neither is `DW_AT_visibility` (local/exported/qualified), and
there's no way to set `DW_AT_visbility` in LLVM's API.

Signed-off-by: David Wood <david@davidtw.co>
2023-12-15 11:36:41 +00:00
bors
9d49eb76c4 Auto merge of #118417 - anforowicz:default-hidden-visibility, r=TaKO8Ki
Add unstable `-Zdefault-hidden-visibility` cmdline flag for `rustc`.

The new flag has been described in the Major Change Proposal at
https://github.com/rust-lang/compiler-team/issues/656
2023-12-14 09:16:15 +00:00
bors
e6d1b0ec98 Auto merge of #118491 - cuviper:aarch64-stack-probes, r=wesleywiser
Enable stack probes on aarch64 for LLVM 18

I tested this on `aarch64-unknown-linux-gnu` with LLVM main (~18).

cc #77071, to be closed once we upgrade our LLVM submodule.
2023-12-14 02:01:13 +00:00
Lukasz Anforowicz
981c4e3ce6 Add unstable -Zdefault-hidden-visibility cmdline flag for rustc.
The new flag has been described in the Major Change Proposal at
https://github.com/rust-lang/compiler-team/issues/656
2023-12-13 21:14:23 +00:00
Jakub Okoński
95b5a80f47
Fix alignment passed down to LLVM for simd_masked_load 2023-12-12 13:11:59 +01:00
The 8472
502df1b7d4 add more niches to rawvec 2023-12-11 23:38:48 +01:00
Jakub Okoński
97ae5095f5
Add simd_masked_{load,store} platform-intrinsics
This maps to the LLVM intrinsics: llvm.masked.load and llvm.masked.store
2023-12-09 12:36:08 +01:00
Josh Stone
b99b5e5752 Enable stack probes on aarch64 for LLVM 18 2023-12-07 17:17:00 -08:00
Ramon de C Valle
97032d63bd CFI: Add char to CFI integer normalization
Adds char to CFI integer normalization to conform to #118032 for
cross-language CFI support.
2023-12-07 11:28:16 -08:00
bendn
73afc00cf9
use assume(idx < self.len()) in [T]::get_unchecked 2023-12-04 06:00:12 +07:00
bors
3f1e30a0a5 Auto merge of #118077 - calebzulawski:sync-portable-simd-2023-11-19, r=workingjubilee
Portable SIMD subtree update

Syncs nightly to the latest changes from rust-lang/portable-simd

r? `@rust-lang/libs`
2023-12-02 18:04:01 +00:00
bors
f45631b10f Auto merge of #116892 - ojeda:rethunk, r=wesleywiser
Add `-Zfunction-return={keep,thunk-extern}` option

This is intended to be used for Linux kernel RETHUNK builds.

With this commit (optionally backported to Rust 1.73.0), plus a patched Linux kernel to pass the flag, I get a RETHUNK build with Rust enabled that is `objtool`-warning-free and is able to boot in QEMU and load a sample Rust kernel module.

Issue: https://github.com/rust-lang/rust/issues/116853.
2023-11-30 22:10:30 +00:00
Miguel Ojeda
2d476222e8 Add -Zfunction-return={keep,thunk-extern} option
This is intended to be used for Linux kernel RETHUNK builds.

With this commit (optionally backported to Rust 1.73.0), plus a
patched Linux kernel to pass the flag, I get a RETHUNK build with
Rust enabled that is `objtool`-warning-free and is able to boot in
QEMU and load a sample Rust kernel module.

Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
2023-11-30 20:21:31 +01:00
bors
07921b50ba Auto merge of #118036 - DianQK:thinlto-tests, r=tmiasko
Add thinlto support to codegen, assembly and coverage tests

Using `--emit=llvm-ir` with thinlto usually result in multiple IR files.
Resolve test case failure issue reported in #113923.
2023-11-30 13:33:32 +00:00
DianQK
c41bf96039
Add thinlto support to codegen, assembly and coverage tests 2023-11-30 18:48:03 +08:00
Krasimir Georgiev
81cd7c5b11 update test for new LLVM 18 codegen
LLVM at HEAD now emits `or disjoint`: https://buildkite.com/llvm-project/rust-llvm-integrate-prototype/builds/24076#018c1596-8153-488e-b622-951266a02f6c/741-774
2023-11-28 12:10:59 +00:00
bors
49b3924bd4 Auto merge of #117947 - Dirbaio:drop-llvm-15, r=cuviper
Update the minimum external LLVM to 16.

With this change, we'll have stable support for LLVM 16 and 17.
For reference, the previous increase to LLVM 15 was #114148

[Relevant zulip discussion](https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/riscv.20forced-atomics)
2023-11-27 21:54:03 +00:00
Caleb Zulawski
4d9607869a Update std::simd usage and test outputs 2023-11-26 09:02:25 -05:00
Arlie Davis
9429d68842 convert ehcont-guard to an unstable option 2023-11-21 14:24:23 -08:00
Arlie Davis
e11d8d147b Add support for generating the EHCont section
In the future Windows will enable Control-flow Enforcement Technology
(CET aka Shadow Stacks). To protect the path where the context is
updated during exception handling, the binary is required to enumerate
valid unwind entrypoints in a dedicated section which is validated when
the context is being set during exception handling.

The required support for EHCONT has already been merged into LLVM,
long ago. This change adds the Rust codegen option to enable it.

Reference:

* https://reviews.llvm.org/D40223

This also adds a new `ehcont-guard` option to the bootstrap config which
enables EHCont Guard when building std.
2023-11-21 13:41:23 -08:00
Dario Nieuwenhuis
7de6d04bc8 Update the minimum external LLVM to 16. 2023-11-21 22:40:16 +01:00
bors
0b24479638 Auto merge of #116555 - paulmenage:llvm-module-flag, r=wesleywiser
Add -Z llvm_module_flag

Allow adding values to the `!llvm.module.flags` metadata for a generated module.  The syntax is

`-Z llvm_module_flag=<name>:<type>:<value>:<behavior>`

Currently only u32 values are supported but the type is required to be specified for forward compatibility.  The `behavior` element must match one of the named LLVM metadata behaviors.viors.

This flag is expected to be perma-unstable.
2023-11-15 16:54:31 +00:00
Augie Fackler
5d8d700fd3 tests: update check for inferred nneg on zext
This was broken by upstream
llvm/llvm-project@dc6d077396. It's easy
enough to use a regex match to support both, so we do that.

r? @nikic
@rustbot label: +llvm-main
2023-11-13 10:43:33 -05:00