Add the attributes to functions according to the settings.
"xray-always" overrides "xray-never", and they both override
"xray-ignore-loops" and "xray-instruction-threshold", but we'll
let lints deal with warnings about silly attribute combinations.
Rollup of 8 pull requests
Successful merges:
- #107022 (Implement `SpecOptionPartialEq` for `cmp::Ordering`)
- #107100 (Use proper `InferCtxt` when probing for associated types in astconv)
- #107103 (Use new solver in `evaluate_obligation` query (when new solver is enabled))
- #107190 (Recover from more const arguments that are not wrapped in curly braces)
- #107306 (Correct suggestions for closure arguments that need a borrow)
- #107339 (internally change regions to be covariant)
- #107344 (Minor tweaks in the new solver)
- #107373 (Don't merge vtables when full debuginfo is enabled.)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
Don't merge vtables when full debuginfo is enabled.
This PR makes the compiler not emit the `unnamed_addr` attribute for vtables when full debuginfo is enabled, so that they don't get merged even if they have the same contents. This allows debuggers to more reliably map from a dyn pointer to the self-type of a trait object by looking at the vtable's debuginfo.
The PR only changes the behavior of the LLVM backend as other backends don't emit vtable debuginfo (as far as I can tell).
The performance impact of this change should be small as [measured](https://github.com/rust-lang/rust/pull/103514#issuecomment-1290833854) in a previous PR.
...and remove it from `PointeeInfo`, which isn't meant for this.
There are still various places (marked with FIXMEs) that assume all pointers
have the same size and alignment. Fixing this requires parsing non-default
address spaces in the data layout string, which will be done in a followup.
Various cleanups around pre-TyCtxt queries and functions
part of #105462
based on https://github.com/rust-lang/rust/pull/106776 (everything starting at [0e2b39f](0e2b39fd1f) is new in this PR)
r? `@petrochenkov`
I think this should be most of the uncontroversial part of #105462.
Rollup of 7 pull requests
Successful merges:
- #104505 (Remove double spaces after dots in comments)
- #106784 (prevent E0512 from emitting [type error] by checking the references_error)
- #106834 (new trait solver: only consider goal changed if response is not identity)
- #106889 (Mention the lack of `windows_mut` in `windows`)
- #106963 (Use `scope_expr_id` from `ProbeCtxt`)
- #106970 (Switch to `EarlyBinder` for `item_bounds` query)
- #106980 (Hide `_use_mk_alias_ty_instead` in `<AliasTy as Debug>::fmt`)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
Previously, it was only put on scalars with range validity invariants
like bool, was uninit was obviously invalid for those.
Since then, we have normatively declared all uninit primitives to be
undefined behavior and can therefore put `noundef` on them.
The remaining concern was the `mem::uninitialized` function, which cause
quite a lot of UB in the older parts of the ecosystem. This function now
doesn't return uninit values anymore, making users of it safe from this
change.
The only real sources of UB where people could encounter uninit
primitives are `MaybeUninit::uninit().assume_init()`, which has always
be clear in the docs about being UB and from heap allocations (like
reading from the spare capacity of a vec. This is hopefully rare enough
to not break anything.
Fix aarch64-unknown-linux-gnu_ilp32 target
This was broken because the synthetic object files produced by rustc were for 64-bit AArch64, which caused link failures when combined with 32-bit ILP32 object files.
This PR updates the object crate to 0.30.1 which adds support for generating ILP32 AArch64 object files.
This was broken because the synthetic object files produced by rustc
were for 64-bit AArch64, which caused link failures when combined with
32-bit ILP32 object files.
This PR updates the object crate to 0.30.1 which adds support for
generating ILP32 AArch64 object files.
Migrate `codegen_ssa` to diagnostics structs - [Part 3]
Completes migrating `codegen_ssa` module except 2 outstanding errors that depend on other crates:
1. [`rustc_middle::mir::interpret::InterpError`](b6097f2e1b/compiler/rustc_middle/src/mir/interpret/error.rs (L475)): I saw `rustc_middle` is unassigned, I am open to take this work.
2. `codegen_llvm`'s use of `fn span_invalid_monomorphization_error`, which I started to replace in the [last commit](9a31b3cdda) of this PR, but would like to know the team's preference on how we should keep replacing the other macros:
2.1. Update macros to expect a `Diagnostic`
2.2. Remove macros and expand the code on each use.
See [some examples of the different options in this experimental commit](64aee83e80)
_Part 2 - https://github.com/rust-lang/rust/pull/103792_
r? ``@davidtwco``
Cc ``@compiler-errors``
Remove wrapper functions for some unstable options
They are trivial and just forward to the option. Like most other options, we can just access it directly.
Allow building std with cranelift
- Don't pass llvm-specific args when using cranelift
- Don't use `asm` in compiler_builtins when using cranelift
r? `@bjorn3` cc `@Mark-Simulacrum`
Correct branch-protection ModFlagBehavior for Aarch64 on LLVM-15
When building with Fat LTO and BTI enabled on aarch64, the BTI is set to `Module::Min` for alloc shim but is set to `Module::Error` for the crate. This was fine when we were using LLVM-14 but LLVM-15 changes it's behaviour to support for compiling with different `mbranch-protection` flags.
Refer:
b0343a38a5
fixes https://github.com/rust-lang/rust/issues/102162
When building with Fat LTO and BTI enabled on aarch64, the BTI is set to
`Module::Min` for alloc shim but is set to `Module::Error` for the
crate. This was fine when we were using LLVM-14 but LLVM-15 changes it's
behaviour to support for compiling with different `mbranch-protection`
flags.
Refer:
b0343a38a5
Following the s390x ELF ABI and based on the clang implementation,
provide appropriate definitions of va_list in library/core/src/ffi/mod.rs
and va_arg handling in compiler/rustc_codegen_llvm/src/va_arg.rs.
Fixes the following test cases on s390x:
src/test/run-make-fulldeps/c-link-to-rust-va-list-fn
src/test/ui/abi/variadic-ffi.rs
Fixes https://github.com/rust-lang/rust/issues/84628.
Use struct types during codegen in less places
This makes it easier to use cg_ssa from a backend like Cranelift that doesn't have any struct types at all. After this PR struct types are still used for function arguments and return values. Removing those usages is harder but should still be doable.
Don't internalize __llvm_profile_counter_bias
Currently, LLVM profiling runtime counter relocation cannot be used by rust during LTO because symbols are being internalized before all symbol information is known.
This mode makes LLVM emit a __llvm_profile_counter_bias symbol which is referenced by the profiling initialization, which itself is pulled in by the rust driver here [1].
It is enabled with -Cllvm-args=-runtime-counter-relocation for platforms which are opt-in to this mode like Linux. On these platforms there will be no link error, rather just surprising behavior for a user which request runtime counter relocation. The profiling runtime will not see that symbol go on as if it were never there. On Fuchsia, the profiling runtime must have this symbol which will cause a hard link error.
As an aside, I don't have enough context as to why rust's LTO model is how it is. AFAICT, the internalize pass is only safe to run at link time when all symbol information is actually known, this being an example as to why. I think special casing this symbol as a known one that LLVM can emit which should not have it's visbility de-escalated should be fine given how seldom this pattern of defining an undefined symbol to get initilization code pulled in is. From a quick grep, __llvm_profile_runtime is the only symbol that rustc does this for.
[1] 0265a3e93b/compiler/rustc_codegen_ssa/src/back/linker.rs (L598)
Add LLVM KCFI support to the Rust compiler
This PR adds LLVM Kernel Control Flow Integrity (KCFI) support to the Rust compiler. It initially provides forward-edge control flow protection for operating systems kernels for Rust-compiled code only by aggregating function pointers in groups identified by their return and parameter types. (See llvm/llvm-project@cff5bef.)
Forward-edge control flow protection for C or C++ and Rust -compiled code "mixed binaries" (i.e., for when C or C++ and Rust -compiled code share the same virtual address space) will be provided in later work as part of this project by identifying C char and integer type uses at the time types are encoded (see Type metadata in the design document in the tracking issue #89653).
LLVM KCFI can be enabled with -Zsanitizer=kcfi.
Thank you again, `@bjorn3,` `@eddyb,` `@nagisa,` and `@ojeda,` for all the help!
This commit adds LLVM Kernel Control Flow Integrity (KCFI) support to
the Rust compiler. It initially provides forward-edge control flow
protection for operating systems kernels for Rust-compiled code only by
aggregating function pointers in groups identified by their return and
parameter types. (See llvm/llvm-project@cff5bef.)
Forward-edge control flow protection for C or C++ and Rust -compiled
code "mixed binaries" (i.e., for when C or C++ and Rust -compiled code
share the same virtual address space) will be provided in later work as
part of this project by identifying C char and integer type uses at the
time types are encoded (see Type metadata in the design document in the
tracking issue #89653).
LLVM KCFI can be enabled with -Zsanitizer=kcfi.
Co-authored-by: bjorn3 <17426603+bjorn3@users.noreply.github.com>
Currently, LLVM profiling runtime counter relocation cannot be
used by rust during LTO because symbols are being internalized
before all symbol information is known.
This mode makes LLVM emit a __llvm_profile_counter_bias symbol
which is referenced by the profiling initialization, which itself
is pulled in by the rust driver here [1].
It is enabled with -Cllvm-args=-runtime-counter-relocation for
platforms which are opt-in to this mode like Linux. On these
platforms there will be no link error, rather just surprising
behavior for a user which request runtime counter relocation.
The profiling runtime will not see that symbol go on as if it
were never there. On Fuchsia, the profiling runtime must have
this symbol which will cause a hard link error.
As an aside, I don't have enough context as to why rust's LTO
model is how it is. AFAICT, the internalize pass is only safe
to run at link time when all symbol information is actually
known, this being an example as to why. I think special casing
this symbol as a known one that LLVM can emit which should not
have it's visbility de-escalated should be fine given how
seldom this pattern of defining an undefined symbol to get
initilization code pulled in is. From a quick grep,
__llvm_profile_runtime is the only symbol that rustc does this
for.
[1] 0265a3e93b/compiler/rustc_codegen_ssa/src/back/linker.rs (L598)
This ensures that the error is printed even for unused variables,
as well as unifying the handling between the LLVM and GCC backends.
This also fixes unusual behavior around exported Rust-defined variables
with linkage attributes. With the previous behavior, it appears to be
impossible to define such a variable such that it can actually be imported
and used by another crate. This is because on the importing side, the
variable is required to be a pointer, but on the exporting side, the
type checker rejects static variables of pointer type because they do
not implement `Sync`. Even if it were possible to import such a type, it
appears that code generation on the importing side would add an unexpected
additional level of pointer indirection, which would break type safety.
This highlighted that the semantics of linkage on Rust-defined variables
is different to linkage on foreign items. As such, we now model the
difference with two different codegen attributes: linkage for Rust-defined
variables, and import_linkage for foreign items.
This change gives semantics to the test
src/test/ui/linkage-attr/auxiliary/def_illtyped_external.rs which was
previously expected to fail to compile. Therefore, convert it into a
test that is expected to successfully compile.
The update to the GCC backend is speculative and untested.
Rollup of 9 pull requests
Successful merges:
- #104199 (Keep track of the start of the argument block of a closure)
- #105050 (Remove useless borrows and derefs)
- #105153 (Create a hacky fail-fast mode that stops tests at the first failure)
- #105164 (Restore `use` suggestion for `dyn` method call requiring `Sized`)
- #105193 (Disable coverage instrumentation for naked functions)
- #105200 (Remove useless filter in unused extern crate check.)
- #105201 (Do not call fn_sig on non-functions.)
- #105208 (Add AmbiguityError for inconsistent resolution for an import)
- #105214 (update Miri)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
codegen-llvm: never combine DSOLocal and DllImport
Prevent DllImport from being attached to DSOLocal definitions in the LLVM IR. The combination makes no sense, since definitions local to the compilation unit will never be imported from external objects.
Additionally, LLVM will refuse the IR if it encounters the combination (introduced in [1]):
```
if (GV.hasDLLImportStorageClass())
Assert(!GV.isDSOLocal(),
"GlobalValue with DLLImport Storage is dso_local!", &GV);
```
Right now, codegen-llvm will only apply DllImport to constants and rely on call-stubs for functions. Hence, we simply extend the codegen of constants to skip DllImport for any local definitions.
This was discovered when switching the EFI targets to the static relocation model [2]. With this fixed, we can start another attempt at this.
[1] 509132b368
[2] https://github.com/rust-lang/rust/issues/101656
Print all features with --print target-features
This fixes `rustc --print target-features` with respect to aliases and tied features.
Before this change, the print command assumed that each LLVM feature corresponds exactly to one rustc feature. In the case of aliases and tied features, this assumption failed and some features (such as aarch64's "pacg") were missing. With this change, every target feature is listed.
Prevent DllImport from being attached to DSOLocal definitions in the
LLVM IR. The combination makes no sense, since definitions local to the
compilation unit will never be imported from external objects.
Additionally, LLVM will refuse the IR if it encounters the
combination (introduced in [1]):
if (GV.hasDLLImportStorageClass())
Assert(!GV.isDSOLocal(),
"GlobalValue with DLLImport Storage is dso_local!", &GV);
Right now, codegen-llvm will only apply DllImport to constants and rely
on call-stubs for functions. Hence, we simply extend the codegen of
constants to skip DllImport for any local definitions.
This was discovered when switching the EFI targets to the static
relocation model [2]. With this fixed, we can start another attempt at
this.
[1] 509132b368
[2] https://github.com/rust-lang/rust/issues/101656
After https://github.com/llvm/llvm-project/commit/8689f5e landed, LLVM takes the intersection of v8a and v8r as default.
This commit brings back v8a support by explicitly specifying v8a in the feature list.
This should solve #97724.
Use `as_deref` in compiler (but only where it makes sense)
This simplifies some code :3
(there are some changes that are not exacly `as_deref`, but more like "clever `Option`/`Result` method use")
Mark functions created for `raw-dylib` on x86 with DllImport storage class
Fix for #104453
## Issue Details
On x86 Windows, LLVM uses 'L' as the prefix for any private global symbols (`PrivateGlobalPrefix`), so when the `raw-dylib` feature creates an undecorated function symbol that begins with an 'L' LLVM misinterprets that as a private global symbol that it created and so fails the compilation at a later stage since such a symbol must have a definition.
## Fix Details
Mark the function we are creating for `raw-dylib` with `DllImport` storage class (this was already being done for MSVC at a later point for `callee::get_fn` but not for GNU (due to "backwards compatibility")): this will cause LLVM to prefix the name with `__imp_` and so it won't mistake it for a private global symbol.
Pass 128-bit C-style enum enumerator values to LLVM
Pass the full 128 bits of C-style enum enumerators through to LLVM. This means that debuginfo for C-style repr128 enums is now emitted correctly for DWARF platforms (as compared to not being correctly emitted on any platform).
Tracking issue: #56071
Improve generating Custom entry function
This commit is aimed at making compiler-generated entry functions (Basically just C `main` right now) more generic so other targets can do similar things for custom entry. This was initially implemented as part of https://github.com/rust-lang/rust/pull/100316.
Currently, this moves the entry function name and Call convention to the target spec.
Signed-off-by: Ayush Singh <ayushsingh1325@gmail.com>
Fix some misleading target feature aliases
This is the first half of a fix for #100752. It looks like these aliases were added in #78361 and slipped under the radar, as these features are not AVX512. These features _do_ add AVX512 instructions when used _in combination_ with AVX512F, but without AVX512F, these features still provide 128-bit and 256-bit vector instructions. A user might be mislead into thinking these features imply AVX512F (which is true of the actual AVX512 features). This PR allows using the names as defined by LLVM, which matches Intel documentation.
A future PR should change the `std::arch` intrinsics to use these names, and finally remove these aliases from rustc.
r? ```@workingjubilee```
cc ```@Amanieu```
For the next commit, `FunctionCx::codegen_*_terminator` need to take a
`&mut Bx` instead of consuming a `Bx`. This triggers a cascade of
similar changes across multiple functions. The resulting code is more
concise and replaces many `&mut bx` expressions with `bx`.
Perform simple scalar replacement of aggregates (SROA) MIR opt
This is a re-open of https://github.com/rust-lang/rust/pull/85796
I copied the debuginfo implementation (first commit) from `@eddyb's` own SROA PR.
This pass replaces plain field accesses by simple locals when possible.
To be eligible, the replaced locals:
- must not be enums or unions;
- must not be used whole;
- must not have their address taken.
The storage and deinit statements are duplicated on each created local.
cc `@tmiasko` who reviewed the former version of this PR.
interpret: support for per-byte provenance
Also factors the provenance map into its own module.
The third commit does the same for the init mask. I can move it in a separate PR if you prefer.
Fixes https://github.com/rust-lang/miri/issues/2181
r? `@oli-obk`
llvm: dwo only emitted when object code emitted
Fixes#103932.
`CompiledModule` should not think a DWARF object was emitted when a bitcode-only compilation has happened, this can confuse archive file creation (which expects to create an archive containing non-existent dwo files).
r? ``````@michaelwoerister``````
This commit is aimed at making compiler generated entry functions
(Basically just C `main` right now) more generic so other targets can do
similar things for custom entry. This was initially implemented as part
of https://github.com/rust-lang/rust/pull/100316.
Currently, this moves the entry function name and Call convention to the
target spec.
Signed-off-by: Ayush Singh <ayushsingh1325@gmail.com>
Fix Access Violation when using lld & ThinLTO on windows-msvc
Users report an AV at runtime of the compiled binary when using lld and ThinLTO on windows-msvc. The AV occurs when accessing a static value which is defined in one crate but used in another. Based on the disassembly of the cross-crate use, it appears that the use is not correctly linked with the definition and is instead assigned a garbage pointer value.
If we look at the symbol tables for each crates' obj file, we can see what is happening:
*lib.obj*:
```
COFF SYMBOL TABLE
...
00E 00000000 SECT2 notype External | _ZN10reproducer7memrchr2FN17h612b61ca0e168901E
...
```
*bin.obj*:
```
COFF SYMBOL TABLE
...
010 00000000 UNDEF notype External | __imp__ZN10reproducer7memrchr2FN17h612b61ca0e168901E
...
```
The use of the symbol has the "import" style symbol name but the declaration doesn't generate any symbol with the same name. As a result, linking the files generates a warning from lld:
> rust-lld: warning: bin.obj: locally defined symbol imported: reproducer::memrchr::FN::h612b61ca0e168901 (defined in lib.obj) [LNK4217]
and the symbol reference remains undefined at runtime leading to the AV.
To fix this, we just need to detect that we are performing ThinLTO (and thus, static linking) and omit the `dllimport` attribute on the extern item in LLVM IR.
Fixes#81408
`CompiledModule` should not think a DWARF object was emitted when a
bitcode-only compilation has happened, this can confuse archive file
creation (which expects to create an archive containing non-existent dwo
files).
Signed-off-by: David Wood <david.wood@huawei.com>
LLVM 16: Update RISCV data layout
The RISCV data layout was changed in 974e2e690b.
This updates all `riscv64*` targets, though I don't really know what the difference between the `gc` and `imac` ones is.
Passes `x test codegen` at LLVM head and with the currently bundled LLVM version. Without this patch, some tests fail with:
> error: internal compiler error: compiler/rustc_codegen_llvm/src/context.rs:192:13: data-layout for target `riscv64gc-unknown-none-elf`, `e-m:e-p:64:64-i64:64-i128:128-n64-S128`, differs from LLVM target's `riscv64` default layout, `e-m:e-p:64:64-i64:64-i128:128-n32:64-S128
Moved type_array function to rustc_codegen_ssa::BaseTypeMethods trait.
This allows using normal alloca function to create arrays as suggested in
https://github.com/rust-lang/rust/pull/104022.
Signed-off-by: Ayush Singh <ayushsingh1325@gmail.com>
LLVM 16: Switch to using MemoryEffects
This adapts the compiler to the changes required by 304f1d59ca.
AFAICT, `WriteOnly` isn't used by the compiler, all `ReadNone` uses were migrated and the remaining use of `ReadOnly` is only for function parameters.
To simplify the FFI, this PR uses an enum to represent `MemoryEffects` across the FFI boundary, which then gets mapped to the matching static factory method when constructing the attribute.
Fixes#103961.
`@rustbot` label +llvm-main
r? `@nikic`
asm: Work around LLVM bug on AArch64
Upstream issue: https://github.com/llvm/llvm-project/issues/58384
LLVM gets confused if we assign a 32-bit value to a 64-bit register, so pass the 32-bit register name to LLVM in that case.
asm: Match clang behavior for inlateout fixed register operands
We have 2 options for representing LLVM constraints for `inlateout` operands on a fixed register (e.g. `r0`): `={r0},0` or `={r0},{r0}`.
This PR changes the behavior to the latter, which matches the behavior of Clang since https://reviews.llvm.org/D87279.
Users report an AV at runtime of the compiled binary when using lld and
ThinLTO on windows-msvc. The AV occurs when accessing a static value
which is defined in one crate but used in another. Based on the
disassembly of the cross-crate use, it appears that the use is not
correctly linked with the definition and is instead assigned a garbage
pointer value.
If we look at the symbol tables for each crates' obj file, we can see
what is happening:
*lib.obj*:
```
COFF SYMBOL TABLE
...
00E 00000000 SECT2 notype External | _ZN10reproducer7memrchr2FN17h612b61ca0e168901E
...
```
*bin.obj*:
```
COFF SYMBOL TABLE
...
010 00000000 UNDEF notype External | __imp__ZN10reproducer7memrchr2FN17h612b61ca0e168901E
...
```
The use of the symbol has the "import" style symbol name but the
declaration doesn't generate any symbol with the same name. As a result,
linking the files generates a warning from lld:
> rust-lld: warning: bin.obj: locally defined symbol imported: reproducer::memrchr::FN::h612b61ca0e168901 (defined in lib.obj) [LNK4217]
and the symbol reference remains undefined at runtime leading to the AV.
To fix this, we just need to detect that we are performing ThinLTO (and
thus, static linking) and omit the `dllimport` attribute on the extern
item in LLVM IR.
The new implementation doesn't use weak lang items and instead changes
`#[alloc_error_handler]` to an attribute macro just like
`#[global_allocator]`.
The attribute will generate the `__rg_oom` function which is called by
the compiler-generated `__rust_alloc_error_handler`. If no `__rg_oom`
function is defined in any crate then the compiler shim will call
`__rdl_oom` in the alloc crate which will simply panic.
This also fixes link errors with `-C link-dead-code` with
`default_alloc_error_handler`: `__rg_oom` was previously defined in the
alloc crate and would attempt to reference the `oom` lang item, even if
it didn't exist. This worked as long as `__rg_oom` was excluded from
linking since it was not called.
This is a prerequisite for the stabilization of
`default_alloc_error_handler` (#102318).
Don't use usub.with.overflow intrinsic
The canonical form of a usub.with.overflow check in LLVM are separate sub + icmp instructions, rather than a usub.with.overflow intrinsic. Using usub.with.overflow will generally result in worse optimization potential.
The backend will attempt to form usub.with.overflow when it comes to actual instruction selection. This is not fully reliable, but I believe this is a better tradeoff than using the intrinsic in IR.
Fixes#103285.
The canonical form of a usub.with.overflow check in LLVM are
separate sub + icmp instructions, rather than a usub.with.overflow
intrinsic. Using usub.with.overflow will generally result in worse
optimization potential.
The backend will attempt to form usub.with.overflow when it comes
to actual instruction selection. This is not fully reliable, but
I believe this is a better tradeoff than using the intrinsic in
IR.
Fixes#103285.
We have 2 options for representing LLVM constraints for `inlateout`
operands on a fixed register (e.g. `r0`): `={r0},0` or `={r0},{r0}`.
This PR changes the behavior to the latter, which matches the behavior
of Clang since https://reviews.llvm.org/D87279.
More dupe word typos
I only picked those changes (from the regex search) that I am pretty certain doesn't change meaning and is just a typo fix. Do correct me if any fix is undesirable and I can revert those. Thanks.
Remove `-Ztime`
Because it has a lot of overlap with `-Ztime-passes` but is generally less useful. Plus some related cleanups.
Best reviewed one commit at a time.
r? `@davidtwco`
The compiler currently has `-Ztime` and `-Ztime-passes`. I've used
`-Ztime-passes` for years but only recently learned about `-Ztime`.
What's the difference? Let's look at the `-Zhelp` output:
```
-Z time=val -- measure time of rustc processes (default: no)
-Z time-passes=val -- measure time of each rustc pass (default: no)
```
The `-Ztime-passes` description is clear, but the `-Ztime` one is less so.
Sounds like it measures the time for the entire process?
No. The real difference is that `-Ztime-passes` prints out info about passes,
and `-Ztime` does the same, but only for a subset of those passes. More
specifically, there is a distinction in the profiling code between a "verbose
generic activity" and an "extra verbose generic activity". `-Ztime-passes`
prints both kinds, while `-Ztime` only prints the first one. (It took me
a close reading of the source code to determine this difference.)
In practice this distinction has low value. Perhaps in the past the "extra
verbose" output was more voluminous, but now that we only print stats for a
pass if it exceeds 5ms or alters the RSS, `-Ztime-passes` is less spammy. Also,
a lot of the "extra verbose" cases are for individual lint passes, and you need
to also use `-Zno-interleave-lints` to see those anyway.
Therefore, this commit removes `-Ztime` and the associated machinery. One thing
to note is that the existing "extra verbose" activities all have an extra
string argument, so the commit adds the ability to accept an extra argument to
the "verbose" activities.
resolve error when attempting to link a universal library on macOS
Previously attempting to link universal libraries into libraries (but not binaries) would produce an error that "File too small to be an archive". This works around this by invoking `lipo -thin` to extract a library for the target platform when passed a univeral library.
Fixes#55235
It's worth acknowledging that this implementation is kind of a horrible hack. Unfortunately I don't know how to do anything better, hopefully this PR will be a jumping off point.
Previously attempting to link universal libraries into libraries (but not binaries) would produce an error that "File too small to be an archive". This works around this by using `object` to extract a library for the target platform when passed a univeral library.
Fixes#55235
Declare `main` as visibility hidden on targets that default to hidden.
On targets with `default_hidden_visibility` set, which is currrently just WebAssembly, declare the generated `main` function with visibility hidden. This makes it consistent with clang's WebAssembly target, where `main` is just a user function that gets the same visibility as any other user function, which is hidden on WebAssembly unless explicitly overridden.
This will help simplify use cases which in the future may want to automatically wasm-export all visibility-"default" symbols. `main` isn't intended to be wasm-exported, and marking it hidden prevents it from being wasm-exported in that scenario.
LLVM [D131158] changed the SystemZ data layout to always set 64-bit
vector alignment, which used to be conditional on the "vector" feature.
[D131158]: https://reviews.llvm.org/D131158
On targets with `default_hidden_visibility` set, which is currrently
just WebAssembly, declare the generated `main` function with visibility
hidden. This makes it consistent with clang's WebAssembly target, where
`main` is just a user function that gets the same visibility as any
other user function, which is hidden on WebAssembly unless explicitly
overridden.
This will help simplify use cases which in the future may want to
automatically wasm-export all visibility-"default" symbols. `main` isn't
intended to be wasm-exported, and marking it hidden prevents it from
being wasm-exported in that scenario.
Remove support for legacy PM
This removes support for optimizing with LLVM's legacy pass manager, as well as the unstable `-Znew-llvm-pass-manager` option. We have been defaulting to the new PM since LLVM 13 (except for s390x that waited for 14), and LLVM 15 removed support altogether. The only place we still use the legacy PM is for writing the output file, just like `llc` does.
cc #74705
r? ``@nikic``
Implement simd_as for pointers
Expands `simd_as` (and `simd_cast`) to handle pointer-to-pointer, pointer-to-integer, and integer-to-pointer conversions.
cc ``@programmerjake`` ``@thomcc``
On later stages, the feature is already stable.
Result of running:
rg -l "feature.let_else" compiler/ src/librustdoc/ library/ | xargs sed -s -i "s#\\[feature.let_else#\\[cfg_attr\\(bootstrap, feature\\(let_else\\)#"
Add support for MIPS VZ ISA extension
[Link to relevant LLVM line where virt extension is specified](83fab8cee9/llvm/lib/Target/Mips/Mips.td (L172-L173))
This has been tested on mips-unknown-linux-musl with a target-cpu that is >= MIPS32 5 and `target-features=+virt`. The example was checked in a disassembler to ensure the correct assembly sequence was being generated using the virtualization instructions.
Needed additional work:
* MIPS is missing from [the Rust reference CPU feature lists](https://doc.rust-lang.org/reference/attributes/codegen.html#available-features)
Example docs for later:
```md
#### `mips` or `mips64`
This platform requires that `#[target_feature]` is only applied to [`unsafe`
functions][unsafe function]. This target's feature support is currently unstable
and must be enabled by `#![feature(mips_target_feature)]` ([Issue #44839])
[Issue #44839]: https://github.com/rust-lang/rust/issues/44839
Further documentation on these features can be found in the [MIPS Instruction Set
Reference Manual], or elsewhere on [mips.com].
[MIPS Instruction Set Reference Manual]: https://s3-eu-west-1.amazonaws.com/downloads-mips/documents/MD00086-2B-MIPS32BIS-AFP-6.06.pdf
[developer.arm.com]: https://www.mips.com/products/architectures/ase/
Feature | Implicitly Enables | Description
---------------|--------------------|-------------------
`fp64` | | 64-bit Floating Point
`msa` | | "MIPS SIMD Architecture"
`virt` | | Virtualization instructions (VZ ASE)
```
If the above is good I can also submit a PR for that if there's interest in documenting it while it's still unstable. Otherwise that can be dropped, I just wrote it before realizing it was possibly not a good idea.
Relevant to #44839
Add inline-llvm option for disabling/enabling LLVM inlining
In this PR, a new -Z option `inline-llvm` is added in order to be able to turn on/off LLVM inlining.
The capability of turning on/off inlining in LLVM backend is needed for testing performance implications of using recently enabled inlining in rustc's frontend (with -Z inline-mir=yes option, #91743). It would be interesting to see the performance effect using rustc's frontend inlining only without LLVM inlining enabled. Currently LLVM is still doing inlining no mater what value inline-mir is set to. With the option `inline-llvm` being added in this PR, user can turn off LLVM inlining by using `-Z inline-llvm=no` option (the default of inline-llvm is 'yes', LLVM inlining enabled).
Rollup of 5 pull requests
Successful merges:
- #99517 (Display raw pointer as *{mut,const} T instead of *-ptr in errors)
- #99928 (Do not leak type variables from opaque type relation)
- #100473 (Attempt to normalize `FnDef` signature in `InferCtxt::cmp`)
- #100653 (Move the cast_float_to_int fallback code to GCC)
- #100941 (Point at the string inside literal and mention if we need string inte…)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
Move the cast_float_to_int fallback code to GCC
Now that we require at least LLVM 13, that codegen backend is always
using its intrinsic `fptosi.sat` and `fptoui.sat` conversions, so it
doesn't need the manual implementation. However, the GCC backend still
needs it, so we can move all of that code down there.
Use object instead of LLVM for reading bitcode from rlibs
Together with changes I plan to make as part of https://github.com/rust-lang/rust/pull/97485 this will allow entirely removing usage of LLVM's archive reader and thus allow removing `archive_ro.rs` and `ArchiveWrapper.cpp`.
Rollup of 9 pull requests
Successful merges:
- #95376 (Add `vec::Drain{,Filter}::keep_rest`)
- #100092 (Fall back when relating two opaques by substs in MIR typeck)
- #101019 (Suggest returning closure as `impl Fn`)
- #101022 (Erase late bound regions before comparing types in `suggest_dereferences`)
- #101101 (interpret: make read-pointer-as-bytes a CTFE-only error with extra information)
- #101123 (Remove `register_attr` feature)
- #101175 (Don't --bless in pre-push hook)
- #101176 (rustdoc: remove unused CSS selectors for `.table-display`)
- #101180 (Add another MaybeUninit array test with const)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
interpret: make read-pointer-as-bytes a CTFE-only error with extra information
Next step in the reaction to https://github.com/rust-lang/rust/issues/99923. Also teaches Miri to implicitly strip provenance in more situations when transmuting pointers to integers, which fixes https://github.com/rust-lang/miri/issues/2456.
Pointer-to-int transmutation during CTFE now produces a message like this:
```
= help: this code performed an operation that depends on the underlying bytes representing a pointer
= help: the absolute address of a pointer is not known at compile-time, so such operations are not supported
```
r? ``@oli-obk``
rustc_middle: Remove `Visibility::Invisible`
It had a different meaning in the past, but now it's only used as an implementation detail of import resolution.
Add pointer masking convenience functions
This PR adds the following public API:
```rust
impl<T: ?Sized> *const T {
fn mask(self, mask: usize) -> *const T;
}
impl<T: ?Sized> *mut T {
fn mask(self, mask: usize) -> *const T;
}
// mod intrinsics
fn mask<T>(ptr: *const T, mask: usize) -> *const T
```
This is equivalent to `ptr.map_addr(|a| a & mask)` but also uses a cool llvm intrinsic.
Proposed in https://github.com/rust-lang/rust/pull/95643#issuecomment-1121562352
cc `@Gankra` `@scottmcm` `@RalfJung`
r? rust-lang/libs-api
Currently they try to be very precise. But they are wrong, i.e. they
don't match what's happening in the loop below. This code isn't hot
enough for it to matter that much.
Because `PassMode::Cast` is by far the largest variant, but is
relatively rare.
This requires making `PassMode` not impl `Copy`, and `Clone` is no
longer necessary. This causes lots of sigil adjusting, but nothing very
notable.
I couldn't find where exactly it's documented, but apperantly pointers to void
type are invalid in llvm - void is only allowed as a return type of functions.
This commit adds the following functions all of which have a signature
`pointer, usize -> pointer`:
- `<*mut T>::mask`
- `<*const T>::mask`
- `intrinsics::ptr_mask`
These functions are equivalent to `.map_addr(|a| a & mask)` but they
utilize `llvm.ptrmask` llvm intrinsic.
*masks your pointers*
Now that we require at least LLVM 13, that codegen backend is always
using its intrinsic `fptosi.sat` and `fptoui.sat` conversions, so it
doesn't need the manual implementation. However, the GCC backend still
needs it, so we can move all of that code down there.
Update the minimum external LLVM to 13
With this change, we'll have stable support for LLVM 13 through 15 (pending release).
For reference, the previous increase to LLVM 12 was #90175.
r? `@nagisa`
Add support for generating unique profraw files by default when using `-C instrument-coverage`
Currently, enabling the rustc flag `-C instrument-coverage` instruments the given crate and by default uses the naming scheme `default.profraw` for any instrumented profile files generated during the execution of a binary linked against this crate. This leads to multiple binaries being executed overwriting one another and causing only the last executable run to contain actual coverage results.
This can be overridden by manually setting the environment variable `LLVM_PROFILE_FILE` to use a unique naming scheme.
This PR adds a change to add support for a reasonable default for rustc to use when enabling coverage instrumentation similar to how the Rust compiler treats generating these same `profraw` files when PGO is enabled.
The new naming scheme is set to `default_%m_%p.profraw` to ensure the uniqueness of each file being generated using [LLVMs special pattern strings](https://clang.llvm.org/docs/SourceBasedCodeCoverage.html#running-the-instrumented-program).
Today the compiler sets the default for PGO `profraw` files to `default_%m.profraw` to ensure a unique file for each run. The same can be done for the instrumented profile files generated via the `-C instrument-coverage` flag as well which LLVM has API support for.
Linked Issue: https://github.com/rust-lang/rust/issues/100381
r? `@wesleywiser`
debuginfo: Generalize C++-like encoding for enums.
The updated encoding should be able to handle niche layouts where more than one variant has fields (as introduced in https://github.com/rust-lang/rust/pull/94075).
The new encoding is more uniform as there is no structural difference between direct-tag, niche-tag, and no-tag layouts anymore. The only difference between those cases is that the "dataful" variant in a niche-tag enum will have a `(start, end)` pair denoting the tag range instead of a single value.
The new encoding now also supports 128-bit tags, which occur in at least some standard library types. These tags are represented as `u64` pairs so that debuggers (which don't always have support for 128-bit integers) can reliably deal with them. The downside is that this adds quite a bit of complexity to the encoding and especially to the corresponding NatVis.
The new encoding seems to increase the size of (x86_64-pc-windows-msvc) debuginfo by 10-15%. The size of binaries is not affected (release builds were built with `-Cdebuginfo=2`, numbers are in kilobytes):
EXE | before | after | relative
-- | -- | -- | --
cargo (debug) | 40453 | 40450 | +0%
ripgrep (debug) | 10275 | 10273 | +0%
cargo (release) | 16186 | 16185 | +0%
ripgrep (release) | 4727 | 4726 | +0%
PDB | before | after | relative
-- | -- | -- | --
cargo (debug) | 236524 | 261412 | +11%
ripgrep (debug) | 53140 | 59060 | +11%
cargo (release) | 148516 | 169620 | +14%
ripgrep (release) | 10676 | 11804 | +11%
Given that the new encoding is more general, this is to be expected. Only platforms using C++-like debuginfo are affected -- which currently is only `*-pc-windows-msvc`.
*TODO*
- [x] Properly update documentation
- [x] Add regression tests for new optimized enum layouts as introduced by #94075.
r? `@wesleywiser`
https://reviews.llvm.org/D120026 changed atomics on thumbv6m to
use libatomic, to ensure that atomic load/store are compatible with
atomic RMW/CAS. However, Rust wants to expose only load/store
without libcalls.
https://reviews.llvm.org/D130480 added support for this behind
the +atomics-32 target feature, so enable that feature.
Introduce an ArchiveBuilderBuilder
This avoids monomorphizing all linker code for each codegen backend and will allow passing in extra information to the archive builder from the codegen backend. I'm going to use this in https://github.com/rust-lang/rust/pull/97485 to allow passing in the right function to extract symbols from object files to a generic archive builder to be used by cg_llvm, cg_clif and cg_gcc.
This avoids monomorphizing all linker code for each codegen backend and
will allow passing in extra information to the archive builder from the
codegen backend.
codegen: use new {re,de,}allocator annotations in llvm
This obviates the patch that teaches LLVM internals about
_rust_{re,de}alloc functions by putting annotations directly in the IR
for the optimizer.
The sole test change is required to anchor FileCheck to the body of the
`box_uninitialized` method, so it doesn't see the `allocalign` on
`__rust_alloc` and get mad about the string `alloca` showing up. Since I
was there anyway, I added some checks on the attributes to prove the
right attributes got set.
r? `@nikic`
This obviates the patch that teaches LLVM internals about
_rust_{re,de}alloc functions by putting annotations directly in the IR
for the optimizer.
The sole test change is required to anchor FileCheck to the body of the
`box_uninitialized` method, so it doesn't see the `allocalign` on
`__rust_alloc` and get mad about the string `alloca` showing up. Since I
was there anyway, I added some checks on the attributes to prove the
right attributes got set.
While we're here, we also emit allocator attributes on
__rust_alloc_zeroed. This should allow LLVM to perform more
optimizations for zeroed blocks, and probably fixes#90032. [This
comment](https://github.com/rust-lang/rust/issues/24194#issuecomment-308791157)
mentions "weird UB-like behaviour with bitvec iterators in
rustc_data_structures" so we may need to back this change out if things
go wrong.
The new test cases require LLVM 15, so we copy them into LLVM
14-supporting versions, which we can delete when we drop LLVM 14.
