In https://reviews.llvm.org/D71059 LLVM 11, the time trace profiler was
extended to support multiple threads.
`timeTraceProfilerInitialize` creates a thread local profiler instance.
When a thread finishes `timeTraceProfilerFinishThread` moves a thread
local instance into a global collection of instances. Finally when all
codegen work is complete `timeTraceProfilerWrite` writes data from the
current thread local instance and the instances in global collection
of instances.
Previously, the profiler was intialized on a single thread only. Since
this thread performs no code generation on its own, the resulting
profile was empty.
Update LLVM codegen to initialize & finish time trace profiler on each
code generation thread.
Repace use of `static_nobundle` with `native_link_modifiers` within Rust codebase
This fixes warnings when building Rust and running tests:
```
warning: library kind `static-nobundle` has been superseded by specifying `-bundle` on library kind `static`. Try `static:-bundle`
warning: `rustc_llvm` (lib) generated 2 warnings (1 duplicate)
```
Add -Z no-unique-section-names to reduce ELF header bloat.
This change adds a new compiler flag that can help reduce the size of ELF binaries that contain many functions.
By default, when enabling function sections (which is the default for most targets), the LLVM backend will generate different section names for each function. For example, a function `func` would generate a section called `.text.func`. Normally this is fine because the linker will merge all those sections into a single one in the binary. However, starting with [LLVM 12](https://github.com/llvm/llvm-project/commit/ee5d1a04), the backend will also generate unique section names for exception handling, resulting in thousands of `.gcc_except_table.*` sections ending up in the final binary because some linkers like LLD don't currently merge or strip these EH sections (see discussion [here](https://reviews.llvm.org/D83655)). This can bloat the ELF headers and string table significantly in binaries that contain many functions.
The new option is analogous to Clang's `-fno-unique-section-names`, and instructs LLVM to generate the same `.text` and `.gcc_except_table` section for each function, resulting in a smaller final binary.
The motivation to add this new option was because we have a binary that ended up with so many ELF sections (over 65,000) that it broke some existing ELF tools, which couldn't handle so many sections.
Here's our old binary:
```
$ readelf --sections old.elf | head -1
There are 71746 section headers, starting at offset 0x2a246508:
$ readelf --sections old.elf | grep shstrtab
[71742] .shstrtab STRTAB 0000000000000000 2977204c ad44bb 00 0 0 1
```
That's an 11MB+ string table. Here's the new binary using this option:
```
$ readelf --sections new.elf | head -1
There are 43 section headers, starting at offset 0x29143ca8:
$ readelf --sections new.elf | grep shstrtab
[40] .shstrtab STRTAB 0000000000000000 29143acc 0001db 00 0 0 1
```
The whole binary size went down by over 20MB, which is quite significant.
This fixes warning when building Rust and running tests:
```
warning: library kind `static-nobundle` has been superseded by specifying `-bundle` on library kind `static`. Try `static:-bundle`
warning: `rustc_llvm` (lib) generated 2 warnings (1 duplicate)
```
This change adds a new compiler flag that can help reduce the size of
ELF binaries that contain many functions.
By default, when enabling function sections (which is the default for most
targets), the LLVM backend will generate different section names for each
function. For example, a function "func" would generate a section called
".text.func". Normally this is fine because the linker will merge all those
sections into a single one in the binary. However, starting with LLVM 12
(llvm/llvm-project@ee5d1a0), the backend will
also generate unique section names for exception handling, resulting in
thousands of ".gcc_except_table.*" sections ending up in the final binary
because some linkers don't currently merge or strip these EH sections.
This can bloat the ELF headers and string table significantly in
binaries that contain many functions.
The new option is analogous to Clang's -fno-unique-section-names, and
instructs LLVM to generate the same ".text" and ".gcc_except_table"
section for each function, resulting in smaller object files and
potentially a smaller final binary.
Implement `#[link_ordinal(n)]`
Allows the use of `#[link_ordinal(n)]` with `#[link(kind = "raw-dylib")]`, allowing Rust to link against DLLs that export symbols by ordinal rather than by name. As long as the ordinal matches, the name of the function in Rust is not required to match the name of the corresponding function in the exporting DLL.
Part of #58713.
Enable AutoFDO.
This largely involves implementing the options debug-info-for-profiling
and profile-sample-use and forwarding them on to LLVM.
AutoFDO can be used on x86-64 Linux like this:
rustc -O -Clink-arg='Wl,--no-rosegment' -Cdebug-info-for-profiling main.rs -o main
perf record -b ./main
create_llvm_prof --binary=main --out=code.prof
rustc -O -Cprofile-sample-use=code.prof main.rs -o main2
Now `main2` will have feedback directed optimization applied to it.
The create_llvm_prof tool can be obtained from this github repository:
https://github.com/google/autofdo
The option -Clink-arg='Wl,--no-rosegment' is necessary to avoid lld
putting an extra RO segment before the executable code, which would make
the binary silently incompatible with create_llvm_prof.
This largely involves implementing the options debug-info-for-profiling
and profile-sample-use and forwarding them on to LLVM.
AutoFDO can be used on x86-64 Linux like this:
rustc -O -Cdebug-info-for-profiling main.rs -o main
perf record -b ./main
create_llvm_prof --binary=main --out=code.prof
rustc -O -Cprofile-sample-use=code.prof main.rs -o main2
Now `main2` will have feedback directed optimization applied to it.
The create_llvm_prof tool can be obtained from this github repository:
https://github.com/google/autofdoFixes#64892.
No functional changes intended.
The LLVM commit
e463b69736
changed an argument of fatal_error_handler_t from std::string to char*.
This adapts RustWrapper accordingly.
thinLTOResolvePrevailingInModule became thinLTOFinalizeInModule and
gained the ability to propagate noRecurse and noUnwind function
attributes. I ran codegen tests with it both on and off, as the upstream
patch uses it in both modes, and the tests pass both ways. Given that,
it seemed reasonable to go ahead and let the propagation be enabled in
rustc, and see what happens. See https://reviews.llvm.org/D36850 for
more examples of how the new version of the function gets used.
Change ab41eef9aca3 in LLVM split MemorySanitizerPass into
MemorySanitizerPass for functions and ModuleMemorySanitizerPass for
modules. There's a related change for ThreadSanitizerPass, and in here
since we're using a ModulePassManager I only add the module flavor of
the pass on LLVM 14.
r? @nikic cc @nagisa
These were deleted in https://reviews.llvm.org/D108614, and in C++ I
definitely see the argument for their removal. I didn't try and
propagate the changes up into higher layers of rustc in this change
because my initial goal was to get rustc working against LLVM HEAD
promptly, but I'm happy to follow up with some refactoring to make the
API on the Rust side match the LLVM API more directly (though the way
the enum works in Rust makes the API less scary IMO).
r? @nagisa cc @nikic
The above-mentioned commit (part of the LLVM 14 development cycle)
removes a method that rustc uses somewhat extensively. We mostly switch
to lower-level methods that exist in all versions of LLVM we use, so no
new ifdef logic is required in most cases.
PassWrapper: adapt for LLVM 14 changes
These API changes appear to have all taken place in
https://reviews.llvm.org/D105007, which moved HWAddressSanitizerPass and
AddressSanitizerPass to only accept their options type as a ctor
argument instead of the sequence of bools etc. This required a couple of
parameter additions, which I made match the default prior to the
mentioned upstream LLVM change.
This patch restores rustc to building (though not quite passing all
tests, I've mailed other patches for those issues) against LLVM HEAD.
These API changes appear to have all taken place in
https://reviews.llvm.org/D105007, which moved HWAddressSanitizerPass and
AddressSanitizerPass to only accept their options type as a ctor
argument instead of the sequence of bools etc. This required a couple of
parameter additions, which I made match the default prior to the
mentioned upstream LLVM change.
This patch restores rustc to building (though not quite passing all
tests, I've mailed other patches for those issues) against LLVM HEAD.
PassWrapper: handle move of OptimizationLevel class out of PassBuilder
This is the first build break of the LLVM 14 cycle, and was caused by
https://reviews.llvm.org/D107025. Mercifully an easy fix.
Rather than relying on `getPointerElementType()` from LLVM function
pointers, we now pass the function type explicitly when building `call`
or `invoke` instructions.
rfc3052 followup: Remove authors field from Cargo manifests
Since RFC 3052 soft deprecated the authors field, hiding it from
crates.io, docs.rs, and making Cargo not add it by default, and it is
not generally up to date/useful information for contributors, we may as well
remove it from crates in this repo.
Since RFC 3052 soft deprecated the authors field anyway, hiding it from
crates.io, docs.rs, and making Cargo not add it by default, and it is
not generally up to date/useful information, we should remove it from
crates in this repo.
Partial support for raw-dylib linkage
First cut of functionality for issue #58713: add support for `#[link(kind = "raw-dylib")]` on `extern` blocks in lib crates compiled to .rlib files. Does not yet support `#[link_name]` attributes on functions, or the `#[link_ordinal]` attribute, or `#[link(kind = "raw-dylib")]` on `extern` blocks in bin crates; I intend to publish subsequent PRs to fill those gaps. It's also not yet clear whether this works for functions in `extern "stdcall"` blocks; I also intend to investigate that shortly and make any necessary changes as a follow-on PR.
This implementation calls out to an LLVM function to construct the actual `.idata` sections as temporary `.lib` files on disk and then links those into the generated .rlib.
BPF target support
This adds `bpfel-unknown-none` and `bpfeb-unknown-none`, two new no_std targets that generate little and big endian BPF. The approach taken is very similar to the cuda target, where `TargetOptions::obj_is_bitcode` is enabled and code generation is done by the linker.
I added the targets to `dist-various-2`. There are [some tests](https://github.com/alessandrod/bpf-linker/tree/main/tests/assembly) in bpf-linker and I'm planning to add more. Those are currently not ran as part of rust CI.
This does not yet support #[link_name] attributes on functions, the #[link_ordinal]
attribute, #[link(kind = "raw-dylib")] on extern blocks in bin crates, or
stdcall functions on 32-bit x86.
This commit updates how rustc compiler metadata is stored in rlibs.
Previously metadata was stored as a raw file that has the same format as
`--emit metadata`. After this commit, however, the metadata is encoded
into a small object file which has one section which is the contents of
the metadata.
The motivation for this commit is to fix a common case where #83730
arises. The problem is that when rustc crates a `dylib` crate type it
needs to include entire rlib files into the dylib, so it passes
`--whole-archive` (or the equivalent) to the linker. The problem with
this, though, is that the linker will attempt to read all files in the
archive. If the metadata file were left as-is (today) then the linker
would generate an error saying it can't read the file. The previous
solution was to alter the rlib just before linking, creating a new
archive in a temporary directory which has the metadata file removed.
This problem from before this commit is now removed if the metadata file
is stored in an object file that the linker can read. The only caveat we
have to take care of is to ensure that the linker never actually
includes the contents of the object file into the final output. We apply
similar tricks as the `.llvmbc` bytecode sections to do this.
This involved changing the metadata loading code a bit, namely updating
some of the LLVM C APIs used to use non-deprecated ones and fiddling
with the lifetimes a bit to get everything to work out. Otherwise though
this isn't intended to be a functional change really, only that metadata
is stored differently in archives now.
This should end up fixing #83730 because by default dylibs will no
longer have their rlib dependencies "altered" meaning that
split-debuginfo will continue to have valid paths pointing at the
original rlibs. (note that we still "alter" rlibs if LTO is enabled to
remove Rust object files and we also "alter" for the #[link(cfg)]
feature, but that's rarely used).
Closes#83730
PassWrapper: update for LLVM change D102093
In https://reviews.llvm.org/D102093 lots of things stopped taking the
DebugLogging boolean parameter. Mercifully we appear to always set
DebugPassManager to false, so I don't think we're losing anything by not
passing this parameter.
In https://reviews.llvm.org/D102093 lots of things stopped taking the
DebugLogging boolean parameter. Mercifully we appear to always set
DebugPassManager to false, so I don't think we're losing anything by not
passing this parameter.
This lets me build against llvm/main as of March 23rd, 2021. I'm not
entirely sure this is _correct_, but it appears to be functionally
identical to what was done in LLVM: existing callsites of
setInlineAsmDiagnosticHandler were moved to SetDiagnosticHandler() on
the context object, which we already set up in both places that we
called setInlineAsmDiagnosticHandler().
`fast-math` implies things like functions not being able to accept as an
argument or return as a result, say, `inf` which made these functions
confusingly named or behaving incorrectly, depending on how you
interpret it. Since the time when these intrinsics have been implemented
the intrinsics user's (stdsimd) approach has changed significantly and
so now it is required that these intrinsics operate normally rather than
in "whatever" way.
Fixes#84268
This should have no real effect in most cases, as e.g. `hidden`
visibility already implies `dso_local` (or at least LLVM IR does not
preserve the `dso_local` setting if the item is already `hidden`), but
it should fix `-Crelocation-model=static` and improve codegen in
executables.
Note that this PR does not exhaustively port the logic in [clang]. Only
the obviously correct portion and what is necessary to fix a regression
from LLVM 12 that relates to `-Crelocation_model=static`.
Fixes#83335
[clang]: 3001d080c8/clang/lib/CodeGen/CodeGenModule.cpp (L945-L1039)
THis came up in the review of #83425: it's hard to imagine a use of
LLVM_VERSION_LE() or LLVM_VERSION_EQ() that's not asking for trouble
when a point release gets created, so let's just discard them to prevent
the issue.
As far as I can tell what we've been getting is llvm::MaybeAlign(), so
just use that for now. This is required sometime after
24539f1ef2471d07bd87f833cb0288fc0f251f4b.
This changed in 54fb3ca96e261f7107cb1b5778c34cb0e0808be6 - I'm not
entirely sure it's correct that we're leaving config empty, but the one
case in LLVM that looked similar did that.
This works around a design defect in the LLVM 12 pass builder
implementation. In LLVM 13, the PreLink ThinLTO pipeline properly
respects the OptimizerLastEPCallbacks.
The definition of this struct changes in LLVM 12 due to the addition
of branch coverage support. To avoid future mismatches, declare our
own struct and then convert between them.
This commit adds a new ABI to be selected via `extern
"C-cmse-nonsecure-call"` on function pointers in order for the compiler to
apply the corresponding cmse_nonsecure_call callsite attribute.
For Armv8-M targets supporting TrustZone-M, this will perform a
non-secure function call by saving, clearing and calling a non-secure
function pointer using the BLXNS instruction.
See the page on the unstable book for details.
Signed-off-by: Hugues de Valon <hugues.devalon@arm.com>
This commit modifies the FFI bindings to LLVM required for Split DWARF
support in rustc. In particular:
- `addPassesToEmitFile`'s wrapper, `LLVMRustWriteOutputFile` now takes
a `DwoPath` `const char*`. When disabled, `nullptr` should be provided
which will preserve existing behaviour. When enabled, the path to the
`.dwo` file should be provided.
- `createCompileUnit`'s wrapper, `LLVMRustDIBuilderCreateCompileUnit`
now has two additional arguments, for the `DWOId` and to enable
`SplitDebugInlining`. `DWOId` should always be zero.
- `createTargetMachine`'s wrapper, `LLVMRustCreateTargetMachine` has an
additional argument which should be provided the path to the `.dwo`
when enabled.
Signed-off-by: David Wood <david@davidtw.co>
Add wasm32 support to inline asm
There is some contention around inline asm and wasm, and I really only made this to figure out the process of hacking on rustc, but I figured as long as the code existed, it was worth uploading.
cc `@Amanieu`
* `rustc` should now compile under LLVM 9 or 10
* Compiler generates an error if `-Z instrument-coverage` is specified
but LLVM version is less than 11
* Coverage tests that require `-Z instrument-coverage` and run codegen
should be skipped if LLVM version is less than 11
Changes the coverage map injected into binaries compiled with
`-Zinstrument-coverage` to LLVM Coverage Mapping Format, Version 4 (from
Version 3). Note, binaries compiled with this version will require LLVM
tools from at least LLVM Version 11.
This commit grepped for LLVM_VERSION_GE, LLVM_VERSION_LT, get_major_version and
min-llvm-version and statically evaluated every expression possible
(and sensible) assuming that the LLVM version is >=9 now
Add support for SHA256 source file hashing
Adds support for `-Z src-hash-algorithm sha256`, which became available in LLVM 11.
Using an older version of LLVM will cause an error `invalid checksum kind` if the hash algorithm is set to sha256.
r? `@eddyb`
cc #70401 `@est31`
rustc_llvm: unwrap LLVMMetadataRef before casting
Directly casting the opaque pointer was [reported] to cause an
"incomplete type" error with GCC 9.3:
```
llvm-wrapper/RustWrapper.cpp:939:31: required from here
/usr/include/c++/9.3/type_traits:1301:12: error: invalid use of incomplete type 'struct LLVMOpaqueMetadata'
1301 | struct is_base_of
| ^~~~~~~~~~
In file included from [...]/rust/src/llvm-project/llvm/include/llvm-c/BitReader.h:23,
from llvm-wrapper/LLVMWrapper.h:1,
from llvm-wrapper/RustWrapper.cpp:1:
[...]/rust/src/llvm-project/llvm/include/llvm-c/Types.h:89:16: note: forward declaration of 'struct LLVMOpaqueMetadata'
89 | typedef struct LLVMOpaqueMetadata *LLVMMetadataRef;
| ^~~~~~~~~~~~~~~~~~
```
[reported]: https://zulip-archive.rust-lang.org/182449tcompilerhelp/12215halprustcllvmbuildfail.html#214915124
A simple `unwrap` fixes the issue.
r? `@eddyb`
Directly casting the opaque pointer was [reported] to cause an
"incomplete type" error with GCC 9.3:
```
llvm-wrapper/RustWrapper.cpp:939:31: required from here
/usr/include/c++/9.3/type_traits:1301:12: error: invalid use of incomplete type 'struct LLVMOpaqueMetadata'
1301 | struct is_base_of
| ^~~~~~~~~~
In file included from [...]/rust/src/llvm-project/llvm/include/llvm-c/BitReader.h:23,
from llvm-wrapper/LLVMWrapper.h:1,
from llvm-wrapper/RustWrapper.cpp:1:
[...]/rust/src/llvm-project/llvm/include/llvm-c/Types.h:89:16: note: forward declaration of 'struct LLVMOpaqueMetadata'
89 | typedef struct LLVMOpaqueMetadata *LLVMMetadataRef;
| ^~~~~~~~~~~~~~~~~~
```
[reported]: https://zulip-archive.rust-lang.org/182449tcompilerhelp/12215halprustcllvmbuildfail.html#214915124
A simple `unwrap` fixes the issue.
Use unwrapDIPtr because the Scope may be null.
I ran into an assertion when using debug information on Windows with LLVM assertions enabled.
It seems like we are using unwrap here (which in turn calls isa and requires the pointer to be non-null) but we expect the value to be null because that is what we are passing from rustc.
This change uses unwrapDIPtr which explicitly allows nullptr.
The FFI prototype for this method on the rust side has the `LLVMMetadataRef` parameter as `Scope: Option<&'a DIScope>`, and we always pass `None` when `msvc_like_names` is true.
Use llvm::computeLTOCacheKey to determine post-ThinLTO CGU reuse
During incremental ThinLTO compilation, we attempt to re-use the
optimized (post-ThinLTO) bitcode file for a module if it is 'safe' to do
so.
Up until now, 'safe' has meant that the set of modules that our current
modules imports from/exports to is unchanged from the previous
compilation session. See PR #67020 and PR #71131 for more details.
However, this turns out be insufficient to guarantee that it's safe
to reuse the post-LTO module (i.e. that optimizing the pre-LTO module
would produce the same result). When LLVM optimizes a module during
ThinLTO, it may look at other information from the 'module index', such
as whether a (non-imported!) global variable is used. If this
information changes between compilation runs, we may end up re-using an
optimized module that (for example) had dead-code elimination run on a
function that is now used by another module.
Fortunately, LLVM implements its own ThinLTO module cache, which is used
when ThinLTO is performed by a linker plugin (e.g. when clang is used to
compile a C proect). Using this cache directly would require extensive
refactoring of our code - but fortunately for us, LLVM provides a
function that does exactly what we need.
The function `llvm::computeLTOCacheKey` is used to compute a SHA-1 hash
from all data that might influence the result of ThinLTO on a module.
In addition to the module imports/exports that we manually track, it
also hashes information about global variables (e.g. their liveness)
which might be used during optimization. By using this function, we
shouldn't have to worry about new LLVM passes breaking our module re-use
behavior.
In LLVM, the output of this function forms part of the filename used to
store the post-ThinLTO module. To keep our current filename structure
intact, this PR just writes out the mapping 'CGU name -> Hash' to a
file. To determine if a post-LTO module should be reused, we compare
hashes from the previous session.
This should unblock PR #75199 - by sheer chance, it seems to have hit
this issue due to the particular CGU partitioning and optimization
decisions that end up getting made.
Secure entry functions do not support if arguments are passed on the
stack. An "unsupported" diagnostic will be emitted by LLVM if that is
the case.
This commits adds support in Rust for that diagnostic so that an error
will be output if that is the case!
Signed-off-by: Hugues de Valon <hugues.devalon@arm.com>
