mikros/rust
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
baf382e63c
rust/compiler/rustc_llvm/llvm-wrapper/PassWrapper.cpp

1802 lines
61 KiB
C++
Raw Normal View History

Rewrite pass management with LLVM Beforehand, it was unclear whether rust was performing the "recommended set" of optimizations provided by LLVM for code. This commit changes the way we run passes to closely mirror that of clang, which in theory does it correctly. The notable changes include: * Passes are no longer explicitly added one by one. This would be difficult to keep up with as LLVM changes and we don't guaranteed always know the best order in which to run passes * Passes are now managed by LLVM's PassManagerBuilder object. This is then used to populate the various pass managers run. * We now run both a FunctionPassManager and a module-wide PassManager. This is what clang does, and I presume that we *may* see a speed boost from the module-wide passes just having to do less work. I have no measured this. * The codegen pass manager has been extracted to its own separate pass manager to not get mixed up with the other passes * All pass managers now include passes for target-specific data layout and analysis passes Some new features include: * You can now print all passes being run with `-Z print-llvm-passes` * When specifying passes via `--passes`, the passes are now appended to the default list of passes instead of overwriting them. * The output of `--passes list` is now generated by LLVM instead of maintaining a list of passes ourselves * Loop vectorization is turned on by default as an optimization pass and can be disabled with `-Z no-vectorize-loops`
2013-08-22 22:58:42 -05:00
#include <stdio.h>
When writing LLVM IR output demangled fn name in comments `--emit=llvm-ir` looks like this now: ``` ; <alloc::vec::Vec<T> as core::ops::index::IndexMut<core::ops::range::RangeFull>>::index_mut ; Function Attrs: inlinehint uwtable define internal { i8*, i64 } @"_ZN106_$LT$alloc..vec..Vec$LT$T$GT$$u20$as$u20$core..ops..index..IndexMut$LT$core..ops..range..RangeFull$GT$$GT$9index_mut17h7f7b576609f30262E"(%"alloc::vec::Vec<u8>"* dereferenceable(24)) unnamed_addr #0 { start: ... ``` cc https://github.com/integer32llc/rust-playground/issues/15
2017-06-29 09:52:43 -05:00
#include <vector>
rustc: Tweak the `isExported` callback for ThinLTO Previously we were too eagerly exporting almost all symbols used in ThinLTO which can cause a whole host of problems downstream! This commit instead fixes this error by aligning more closely with `lib/LTO/LTO.cpp` in LLVM's codebase which is to only change the linkage of summaries which are computed as dead. Closes #46374
2017-11-29 14:00:10 -06:00
#include <set>
When writing LLVM IR output demangled fn name in comments `--emit=llvm-ir` looks like this now: ``` ; <alloc::vec::Vec<T> as core::ops::index::IndexMut<core::ops::range::RangeFull>>::index_mut ; Function Attrs: inlinehint uwtable define internal { i8*, i64 } @"_ZN106_$LT$alloc..vec..Vec$LT$T$GT$$u20$as$u20$core..ops..index..IndexMut$LT$core..ops..range..RangeFull$GT$$GT$9index_mut17h7f7b576609f30262E"(%"alloc::vec::Vec<u8>"* dereferenceable(24)) unnamed_addr #0 { start: ... ``` cc https://github.com/integer32llc/rust-playground/issues/15
2017-06-29 09:52:43 -05:00
Move `rustllvm` into `rustc_llvm`
2020-07-26 12:11:30 -05:00
#include "LLVMWrapper.h"
Refactor optimization pass handling. Refactor the optimization passes to explicitly use the passes. This commit just re-implements the same passes as were already being run. It also adds an option (behind `-Z`) to run the LLVM lint pass on the unoptimized IR.
2013-05-27 18:15:31 -05:00
Support LLVM 12 in rustc
2020-11-03 15:47:16 -06:00
#include "llvm/Analysis/AliasAnalysis.h"
rustc: Update LLVM This commit updates the LLVM submodule in use to the current HEAD of the LLVM repository. This is primarily being done to start picking up unwinding support for MSVC, which is currently unimplemented in the revision of LLVM we are using. Along the way a few changes had to be made: * As usual, lots of C++ debuginfo bindings in LLVM changed, so there were some significant changes to our RustWrapper.cpp * As usual, some pass management changed in LLVM, so clang was re-scrutinized to ensure that we're doing the same thing as clang. * Some optimization options are now passed directly into the `PassManagerBuilder` instead of through CLI switches to LLVM. * The `NoFramePointerElim` option was removed from LLVM, favoring instead the `no-frame-pointer-elim` function attribute instead. Additionally, LLVM has picked up some new optimizations which required fixing an existing soundness hole in the IR we generate. It appears that the current LLVM we use does not expose this hole. When an enum is moved, the previous slot in memory is overwritten with a bit pattern corresponding to "dropped". When the drop glue for this slot is run, however, the switch on the discriminant can often start executing the `unreachable` block of the switch due to the discriminant now being outside the normal range. This was patched over locally for now by having the `unreachable` block just change to a `ret void`.
2015-05-14 14:10:43 -05:00
#include "llvm/Analysis/TargetLibraryInfo.h"
#include "llvm/Analysis/TargetTransformInfo.h"
Bump minimum required LLVM version to 6.0
2018-12-09 04:20:20 -06:00
#include "llvm/CodeGen/TargetSubtargetInfo.h"
Explicitly include InitializePasses.h
2019-12-31 07:02:22 -06:00
#include "llvm/InitializePasses.h"
Support removed LLVM intrinsics by invoking its AutoUpgrade mechanism.
2016-08-03 14:37:57 -05:00
#include "llvm/IR/AutoUpgrade.h"
When writing LLVM IR output demangled fn name in comments `--emit=llvm-ir` looks like this now: ``` ; <alloc::vec::Vec<T> as core::ops::index::IndexMut<core::ops::range::RangeFull>>::index_mut ; Function Attrs: inlinehint uwtable define internal { i8*, i64 } @"_ZN106_$LT$alloc..vec..Vec$LT$T$GT$$u20$as$u20$core..ops..index..IndexMut$LT$core..ops..range..RangeFull$GT$$GT$9index_mut17h7f7b576609f30262E"(%"alloc::vec::Vec<u8>"* dereferenceable(24)) unnamed_addr #0 { start: ... ``` cc https://github.com/integer32llc/rust-playground/issues/15
2017-06-29 09:52:43 -05:00
#include "llvm/IR/AssemblyAnnotationWriter.h"
Bump minimum required LLVM version to 6.0
2018-12-09 04:20:20 -06:00
#include "llvm/IR/IntrinsicInst.h"
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
#include "llvm/IR/Verifier.h"
Store LLVM bitcode in object files, not compressed This commit is an attempted resurrection of #70458 where LLVM bitcode emitted by rustc into rlibs is stored into object file sections rather than in a separate file. The main rationale for doing this is that when rustc emits bitcode it will no longer use a custom compression scheme which makes it both easier to interoperate with existing tools and also cuts down on compile time since this compression isn't happening. The blocker for this in #70458 turned out to be that native linkers didn't handle the new sections well, causing the sections to either trigger bugs in the linker or actually end up in the final linked artifact. This commit attempts to address these issues by ensuring that native linkers ignore the new sections by inserting custom flags with module-level inline assembly. Note that this does not currently change the API of the compiler at all. The pre-existing `-C bitcode-in-rlib` flag is co-opted to indicate whether the bitcode should be present in the object file or not. Finally, note that an important consequence of this commit, which is also one of its primary purposes, is to enable rustc's `-Clto` bitcode loading to load rlibs produced with `-Clinker-plugin-lto`. The goal here is that when you're building with LTO Cargo will tell rustc to skip codegen of all intermediate crates and only generate LLVM IR. Today rustc will generate both object code and LLVM IR, but the object code is later simply thrown away, wastefully.
2020-04-23 13:45:55 -05:00
#include "llvm/Object/ObjectFile.h"
#include "llvm/Object/IRObjectFile.h"
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
#include "llvm/Passes/PassBuilder.h"
Add a codegen option to allow loading LLVM pass plugins
2021-11-19 10:01:41 -06:00
#include "llvm/Passes/PassPlugin.h"
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
#include "llvm/Passes/StandardInstrumentations.h"
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
#include "llvm/Support/CBindingWrapping.h"
#include "llvm/Support/FileSystem.h"
#include "llvm/Support/Host.h"
RustWrapper: adapt for an LLVM API change No functional changes intended. The LLVM commit https://github.com/llvm/llvm-project/commit/89b57061f7b769e9ea9bf6ed686e284f3e55affe moved TargetRegistry.(h|cpp) from Support to MC. This adapts RustWrapper accordingly.
2021-10-11 05:31:43 -05:00
#if LLVM_VERSION_LT(14, 0)
#include "llvm/Support/TargetRegistry.h"
#else
#include "llvm/MC/TargetRegistry.h"
#endif
trans: Clean up handling the LLVM data layout Turns out for OSX our data layout was subtly wrong and the LLVM update must have exposed this. Instead of fixing this I've removed all data layouts from the compiler to just use the defaults that LLVM provides for all targets. All data layouts (and a number of dead modules) are removed from the compiler here. Custom target specifications can still provide a custom data layout, but it is now an optional key as the default will be used if one isn't specified.
2015-07-16 17:48:16 -05:00
#include "llvm/Target/TargetMachine.h"
Rewrite pass management with LLVM Beforehand, it was unclear whether rust was performing the "recommended set" of optimizations provided by LLVM for code. This commit changes the way we run passes to closely mirror that of clang, which in theory does it correctly. The notable changes include: * Passes are no longer explicitly added one by one. This would be difficult to keep up with as LLVM changes and we don't guaranteed always know the best order in which to run passes * Passes are now managed by LLVM's PassManagerBuilder object. This is then used to populate the various pass managers run. * We now run both a FunctionPassManager and a module-wide PassManager. This is what clang does, and I presume that we *may* see a speed boost from the module-wide passes just having to do less work. I have no measured this. * The codegen pass manager has been extracted to its own separate pass manager to not get mixed up with the other passes * All pass managers now include passes for target-specific data layout and analysis passes Some new features include: * You can now print all passes being run with `-Z print-llvm-passes` * When specifying passes via `--passes`, the passes are now appended to the default list of passes instead of overwriting them. * The output of `--passes list` is now generated by LLVM instead of maintaining a list of passes ourselves * Loop vectorization is turned on by default as an optimization pass and can be disabled with `-Z no-vectorize-loops`
2013-08-22 22:58:42 -05:00
#include "llvm/Transforms/IPO/PassManagerBuilder.h"
[LLVM 4.0] Update AlwaysInliner pass header and constructor
2016-09-24 11:37:04 -05:00
#include "llvm/Transforms/IPO/AlwaysInliner.h"
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
#include "llvm/Transforms/IPO/FunctionImport.h"
Enable AutoFDO. This largely involves implementing the options debug-info-for-profiling and profile-sample-use and forwarding them on to LLVM. AutoFDO can be used on x86-64 Linux like this: rustc -O -Cdebug-info-for-profiling main.rs -o main perf record -b ./main create_llvm_prof --binary=main --out=code.prof rustc -O -Cprofile-sample-use=code.prof main.rs -o main2 Now `main2` will have feedback directed optimization applied to it. The create_llvm_prof tool can be obtained from this github repository: https://github.com/google/autofdo Fixes #64892.
2021-05-07 02:41:37 -05:00
#include "llvm/Transforms/Utils/AddDiscriminators.h"
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
#include "llvm/Transforms/Utils/FunctionImportUtils.h"
#include "llvm/LTO/LTO.h"
Rewrite pass management with LLVM Beforehand, it was unclear whether rust was performing the "recommended set" of optimizations provided by LLVM for code. This commit changes the way we run passes to closely mirror that of clang, which in theory does it correctly. The notable changes include: * Passes are no longer explicitly added one by one. This would be difficult to keep up with as LLVM changes and we don't guaranteed always know the best order in which to run passes * Passes are now managed by LLVM's PassManagerBuilder object. This is then used to populate the various pass managers run. * We now run both a FunctionPassManager and a module-wide PassManager. This is what clang does, and I presume that we *may* see a speed boost from the module-wide passes just having to do less work. I have no measured this. * The codegen pass manager has been extracted to its own separate pass manager to not get mixed up with the other passes * All pass managers now include passes for target-specific data layout and analysis passes Some new features include: * You can now print all passes being run with `-Z print-llvm-passes` * When specifying passes via `--passes`, the passes are now appended to the default list of passes instead of overwriting them. * The output of `--passes list` is now generated by LLVM instead of maintaining a list of passes ourselves * Loop vectorization is turned on by default as an optimization pass and can be disabled with `-Z no-vectorize-loops`
2013-08-22 22:58:42 -05:00
#include "llvm-c/Transforms/PassManagerBuilder.h"
Refactor optimization pass handling. Refactor the optimization passes to explicitly use the passes. This commit just re-implements the same passes as were already being run. It also adds an option (behind `-Z`) to run the LLVM lint pass on the unoptimized IR.
2013-05-27 18:15:31 -05:00
Retain compatibility with LLVM 6, 7, 8 and 9
2019-11-17 18:00:00 -06:00
#include "llvm/Transforms/Instrumentation.h"
#include "llvm/Transforms/Instrumentation/AddressSanitizer.h"
Add support for enabling the LLVM time-trace feature I found this helpful while investigating an LLVM performance issue. Passing `-Z llvm-time-trace` causes a `llvm_timings.json` file to be created. This file can be inspected in either the Chrome Profiler tools or with any other compatible tool like SpeedScope. More information on the LLVM feature: - https://aras-p.info/blog/2019/01/16/time-trace-timeline-flame-chart-profiler-for-Clang/ - https://reviews.llvm.org/rL357340
2020-01-31 17:58:28 -06:00
#include "llvm/Support/TimeProfiler.h"
Explicitly register GCOV profiling pass as well
2021-04-05 04:26:48 -05:00
#include "llvm/Transforms/Instrumentation/GCOVProfiler.h"
Explicitly register instrprof pass Don't use "passes" for this purpose, explicitly insert it into the correct place in the pipeline instead.
2021-04-05 03:45:04 -05:00
#include "llvm/Transforms/Instrumentation/InstrProfiling.h"
Retain compatibility with LLVM 6, 7, 8 and 9
2019-11-17 18:00:00 -06:00
#include "llvm/Transforms/Instrumentation/ThreadSanitizer.h"
#include "llvm/Transforms/Instrumentation/MemorySanitizer.h"
HWASan support
2021-01-22 20:32:38 -06:00
#include "llvm/Transforms/Instrumentation/HWAddressSanitizer.h"
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
#include "llvm/Transforms/Utils/CanonicalizeAliases.h"
#include "llvm/Transforms/Utils/NameAnonGlobals.h"
Enable AutoFDO. This largely involves implementing the options debug-info-for-profiling and profile-sample-use and forwarding them on to LLVM. AutoFDO can be used on x86-64 Linux like this: rustc -O -Cdebug-info-for-profiling main.rs -o main perf record -b ./main create_llvm_prof --binary=main --out=code.prof rustc -O -Cprofile-sample-use=code.prof main.rs -o main2 Now `main2` will have feedback directed optimization applied to it. The create_llvm_prof tool can be obtained from this github repository: https://github.com/google/autofdo Fixes #64892.
2021-05-07 02:41:37 -05:00
#include "llvm/Transforms/Utils.h"
Retain compatibility with LLVM 6, 7, 8 and 9
2019-11-17 18:00:00 -06:00
Rewrite pass management with LLVM Beforehand, it was unclear whether rust was performing the "recommended set" of optimizations provided by LLVM for code. This commit changes the way we run passes to closely mirror that of clang, which in theory does it correctly. The notable changes include: * Passes are no longer explicitly added one by one. This would be difficult to keep up with as LLVM changes and we don't guaranteed always know the best order in which to run passes * Passes are now managed by LLVM's PassManagerBuilder object. This is then used to populate the various pass managers run. * We now run both a FunctionPassManager and a module-wide PassManager. This is what clang does, and I presume that we *may* see a speed boost from the module-wide passes just having to do less work. I have no measured this. * The codegen pass manager has been extracted to its own separate pass manager to not get mixed up with the other passes * All pass managers now include passes for target-specific data layout and analysis passes Some new features include: * You can now print all passes being run with `-Z print-llvm-passes` * When specifying passes via `--passes`, the passes are now appended to the default list of passes instead of overwriting them. * The output of `--passes list` is now generated by LLVM instead of maintaining a list of passes ourselves * Loop vectorization is turned on by default as an optimization pass and can be disabled with `-Z no-vectorize-loops`
2013-08-22 22:58:42 -05:00
using namespace llvm;
Refactor optimization pass handling. Refactor the optimization passes to explicitly use the passes. This commit just re-implements the same passes as were already being run. It also adds an option (behind `-Z`) to run the LLVM lint pass on the unoptimized IR.
2013-05-27 18:15:31 -05:00
Rewrite pass management with LLVM Beforehand, it was unclear whether rust was performing the "recommended set" of optimizations provided by LLVM for code. This commit changes the way we run passes to closely mirror that of clang, which in theory does it correctly. The notable changes include: * Passes are no longer explicitly added one by one. This would be difficult to keep up with as LLVM changes and we don't guaranteed always know the best order in which to run passes * Passes are now managed by LLVM's PassManagerBuilder object. This is then used to populate the various pass managers run. * We now run both a FunctionPassManager and a module-wide PassManager. This is what clang does, and I presume that we *may* see a speed boost from the module-wide passes just having to do less work. I have no measured this. * The codegen pass manager has been extracted to its own separate pass manager to not get mixed up with the other passes * All pass managers now include passes for target-specific data layout and analysis passes Some new features include: * You can now print all passes being run with `-Z print-llvm-passes` * When specifying passes via `--passes`, the passes are now appended to the default list of passes instead of overwriting them. * The output of `--passes list` is now generated by LLVM instead of maintaining a list of passes ourselves * Loop vectorization is turned on by default as an optimization pass and can be disabled with `-Z no-vectorize-loops`
2013-08-22 22:58:42 -05:00
typedef struct LLVMOpaquePass *LLVMPassRef;
typedef struct LLVMOpaqueTargetMachine *LLVMTargetMachineRef;
DEFINE_STDCXX_CONVERSION_FUNCTIONS(Pass, LLVMPassRef)
DEFINE_STDCXX_CONVERSION_FUNCTIONS(TargetMachine, LLVMTargetMachineRef)
Refactor optimization pass handling. Refactor the optimization passes to explicitly use the passes. This commit just re-implements the same passes as were already being run. It also adds an option (behind `-Z`) to run the LLVM lint pass on the unoptimized IR.
2013-05-27 18:15:31 -05:00
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
extern "C" void LLVMInitializePasses() {
Further refactor optimization pass handling This refactors pass handling to use the argument names, so it can be used in a similar manner to `opt`. This may be slightly less efficient than the previous version, but it is much easier to maintain. It also adds in the ability to specify a custom pipeline on the command line, this overrides the normal passes, however. This should completely close #2396.
2013-05-29 03:08:20 -05:00
PassRegistry &Registry = *PassRegistry::getPassRegistry();
initializeCore(Registry);
initializeCodeGen(Registry);
initializeScalarOpts(Registry);
initializeVectorization(Registry);
initializeIPO(Registry);
initializeAnalysis(Registry);
initializeTransformUtils(Registry);
initializeInstCombine(Registry);
initializeInstrumentation(Registry);
initializeTarget(Registry);
}
Refactor optimization pass handling. Refactor the optimization passes to explicitly use the passes. This commit just re-implements the same passes as were already being run. It also adds an option (behind `-Z`) to run the LLVM lint pass on the unoptimized IR.
2013-05-27 18:15:31 -05:00
Add support for enabling the LLVM time-trace feature I found this helpful while investigating an LLVM performance issue. Passing `-Z llvm-time-trace` causes a `llvm_timings.json` file to be created. This file can be inspected in either the Chrome Profiler tools or with any other compatible tool like SpeedScope. More information on the LLVM feature: - https://aras-p.info/blog/2019/01/16/time-trace-timeline-flame-chart-profiler-for-Clang/ - https://reviews.llvm.org/rL357340
2020-01-31 17:58:28 -06:00
extern "C" void LLVMTimeTraceProfilerInitialize() {
Fix timeTraceProfilerInitialize for LLVM 10
2020-02-04 13:35:50 -06:00
timeTraceProfilerInitialize(
/* TimeTraceGranularity */ 0,
/* ProcName */ "rustc");
Add support for enabling the LLVM time-trace feature I found this helpful while investigating an LLVM performance issue. Passing `-Z llvm-time-trace` causes a `llvm_timings.json` file to be created. This file can be inspected in either the Chrome Profiler tools or with any other compatible tool like SpeedScope. More information on the LLVM feature: - https://aras-p.info/blog/2019/01/16/time-trace-timeline-flame-chart-profiler-for-Clang/ - https://reviews.llvm.org/rL357340
2020-01-31 17:58:28 -06:00
}
Initialize LLVM time trace profiler on each code generation thread In https://reviews.llvm.org/D71059 LLVM 11, the time trace profiler was extended to support multiple threads. `timeTraceProfilerInitialize` creates a thread local profiler instance. When a thread finishes `timeTraceProfilerFinishThread` moves a thread local instance into a global collection of instances. Finally when all codegen work is complete `timeTraceProfilerWrite` writes data from the current thread local instance and the instances in global collection of instances. Previously, the profiler was intialized on a single thread only. Since this thread performs no code generation on its own, the resulting profile was empty. Update LLVM codegen to initialize & finish time trace profiler on each code generation thread.
2021-11-04 19:00:00 -05:00
extern "C" void LLVMTimeTraceProfilerFinishThread() {
timeTraceProfilerFinishThread();
}
Add support for enabling the LLVM time-trace feature I found this helpful while investigating an LLVM performance issue. Passing `-Z llvm-time-trace` causes a `llvm_timings.json` file to be created. This file can be inspected in either the Chrome Profiler tools or with any other compatible tool like SpeedScope. More information on the LLVM feature: - https://aras-p.info/blog/2019/01/16/time-trace-timeline-flame-chart-profiler-for-Clang/ - https://reviews.llvm.org/rL357340
2020-01-31 17:58:28 -06:00
extern "C" void LLVMTimeTraceProfilerFinish(const char* FileName) {
StringRef FN(FileName);
std::error_code EC;
raw_fd_ostream OS(FN, EC, sys::fs::CD_CreateAlways);
timeTraceProfilerWrite(OS);
timeTraceProfilerCleanup();
}
audit LLVM C++ types in ArchiveWrapper and PassWrapper
2016-08-01 16:16:16 -05:00
enum class LLVMRustPassKind {
Other,
Register LLVM passes with the correct LLVM pass manager. LLVM was upgraded to a new version in this commit: https://github.com/rust-lang/rust/commit/f9d4149c29e8b989fa3624993be379f380e48dcf which was part of this pull request: https://github.com/rust-lang/rust/issues/26025 Consider the following two lines from that commit: https://github.com/rust-lang/rust/commit/f9d4149c29e8b989fa3624993be379f380e48dcf#diff-a3b24dbe2ea7c1981f9ac79f9745f40aL462 https://github.com/rust-lang/rust/commit/f9d4149c29e8b989fa3624993be379f380e48dcf#diff-a3b24dbe2ea7c1981f9ac79f9745f40aL469 The purpose of these lines is to register LLVM passes. Prior to the that commit, the passes being handled were assumed to be ModulePasses (a specific type of LLVM pass) since they were being added to a ModulePass manager. After that commit, both lines were refactored (presumably in an attempt to DRY out the code), but the ModulePasses were changed to be registered to a FunctionPass manager. This change resulted in ModulePasses being run, but a Function object was being passed as a parameter to the pass instead of a Module, which resulted in segmentation faults. In this commit, I changed relevant sections of the code to check the type of the passes being added and register them to the appropriate pass manager. Closes https://github.com/rust-lang/rust/issues/31067
2016-01-24 19:22:24 -06:00
Function,
Module,
};
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
static LLVMRustPassKind toRust(PassKind Kind) {
switch (Kind) {
audit LLVM C++ types in ArchiveWrapper and PassWrapper
2016-08-01 16:16:16 -05:00
case PT_Function:
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
return LLVMRustPassKind::Function;
audit LLVM C++ types in ArchiveWrapper and PassWrapper
2016-08-01 16:16:16 -05:00
case PT_Module:
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
return LLVMRustPassKind::Module;
audit LLVM C++ types in ArchiveWrapper and PassWrapper
2016-08-01 16:16:16 -05:00
default:
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
return LLVMRustPassKind::Other;
audit LLVM C++ types in ArchiveWrapper and PassWrapper
2016-08-01 16:16:16 -05:00
}
}
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
extern "C" LLVMPassRef LLVMRustFindAndCreatePass(const char *PassName) {
Stub out various legacy PM functions with LLVM 15
2022-04-19 06:22:39 -05:00
#if LLVM_VERSION_LT(15, 0)
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
StringRef SR(PassName);
PassRegistry *PR = PassRegistry::getPassRegistry();
Refactor optimization pass handling. Refactor the optimization passes to explicitly use the passes. This commit just re-implements the same passes as were already being run. It also adds an option (behind `-Z`) to run the LLVM lint pass on the unoptimized IR.
2013-05-27 18:15:31 -05:00
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
const PassInfo *PI = PR->getPassInfo(SR);
if (PI) {
return wrap(PI->createPass());
}
Switching from NULL to nullptr in src/rustllvm.
2016-12-30 06:21:21 -06:00
return nullptr;
Stub out various legacy PM functions with LLVM 15
2022-04-19 06:22:39 -05:00
#else
report_fatal_error("Legacy PM not supported with LLVM 15");
#endif
Register LLVM passes with the correct LLVM pass manager. LLVM was upgraded to a new version in this commit: https://github.com/rust-lang/rust/commit/f9d4149c29e8b989fa3624993be379f380e48dcf which was part of this pull request: https://github.com/rust-lang/rust/issues/26025 Consider the following two lines from that commit: https://github.com/rust-lang/rust/commit/f9d4149c29e8b989fa3624993be379f380e48dcf#diff-a3b24dbe2ea7c1981f9ac79f9745f40aL462 https://github.com/rust-lang/rust/commit/f9d4149c29e8b989fa3624993be379f380e48dcf#diff-a3b24dbe2ea7c1981f9ac79f9745f40aL469 The purpose of these lines is to register LLVM passes. Prior to the that commit, the passes being handled were assumed to be ModulePasses (a specific type of LLVM pass) since they were being added to a ModulePass manager. After that commit, both lines were refactored (presumably in an attempt to DRY out the code), but the ModulePasses were changed to be registered to a FunctionPass manager. This change resulted in ModulePasses being run, but a Function object was being passed as a parameter to the pass instead of a Module, which resulted in segmentation faults. In this commit, I changed relevant sections of the code to check the type of the passes being added and register them to the appropriate pass manager. Closes https://github.com/rust-lang/rust/issues/31067
2016-01-24 19:22:24 -06:00
}
Move sanitizer passes creation from ssa to llvm
2019-11-17 18:00:00 -06:00
extern "C" LLVMPassRef LLVMRustCreateAddressSanitizerFunctionPass(bool Recover) {
Stub out various legacy PM functions with LLVM 15
2022-04-19 06:22:39 -05:00
#if LLVM_VERSION_LT(15, 0)
Move sanitizer passes creation from ssa to llvm
2019-11-17 18:00:00 -06:00
const bool CompileKernel = false;
Detect use-after-scope bugs with AddressSanitizer Enable use-after-scope checks by default when using AddressSanitizer. They allow to detect incorrect use of stack objects after their scope have already ended. The detection is based on LLVM lifetime intrinsics. To facilitate the use of this functionality, the lifetime intrinsics are now emitted regardless of optimization level if enabled sanitizer makes use of them.
2020-01-26 18:00:00 -06:00
const bool UseAfterScope = true;
Move sanitizer passes creation from ssa to llvm
2019-11-17 18:00:00 -06:00
Detect use-after-scope bugs with AddressSanitizer Enable use-after-scope checks by default when using AddressSanitizer. They allow to detect incorrect use of stack objects after their scope have already ended. The detection is based on LLVM lifetime intrinsics. To facilitate the use of this functionality, the lifetime intrinsics are now emitted regardless of optimization level if enabled sanitizer makes use of them.
2020-01-26 18:00:00 -06:00
return wrap(createAddressSanitizerFunctionPass(CompileKernel, Recover, UseAfterScope));
Stub out various legacy PM functions with LLVM 15
2022-04-19 06:22:39 -05:00
#else
report_fatal_error("Legacy PM not supported with LLVM 15");
#endif
Move sanitizer passes creation from ssa to llvm
2019-11-17 18:00:00 -06:00
}
extern "C" LLVMPassRef LLVMRustCreateModuleAddressSanitizerPass(bool Recover) {
Stub out various legacy PM functions with LLVM 15
2022-04-19 06:22:39 -05:00
#if LLVM_VERSION_LT(15, 0)
Move sanitizer passes creation from ssa to llvm
2019-11-17 18:00:00 -06:00
const bool CompileKernel = false;
return wrap(createModuleAddressSanitizerLegacyPassPass(CompileKernel, Recover));
Stub out various legacy PM functions with LLVM 15
2022-04-19 06:22:39 -05:00
#else
report_fatal_error("Legacy PM not supported with LLVM 15");
#endif
Move sanitizer passes creation from ssa to llvm
2019-11-17 18:00:00 -06:00
}
extern "C" LLVMPassRef LLVMRustCreateMemorySanitizerPass(int TrackOrigins, bool Recover) {
Stub out various legacy PM functions with LLVM 15
2022-04-19 06:22:39 -05:00
#if LLVM_VERSION_LT(15, 0)
Move sanitizer passes creation from ssa to llvm
2019-11-17 18:00:00 -06:00
const bool CompileKernel = false;
return wrap(createMemorySanitizerLegacyPassPass(
MemorySanitizerOptions{TrackOrigins, Recover, CompileKernel}));
Stub out various legacy PM functions with LLVM 15
2022-04-19 06:22:39 -05:00
#else
report_fatal_error("Legacy PM not supported with LLVM 15");
#endif
Move sanitizer passes creation from ssa to llvm
2019-11-17 18:00:00 -06:00
}
extern "C" LLVMPassRef LLVMRustCreateThreadSanitizerPass() {
Stub out various legacy PM functions with LLVM 15
2022-04-19 06:22:39 -05:00
#if LLVM_VERSION_LT(15, 0)
Move sanitizer passes creation from ssa to llvm
2019-11-17 18:00:00 -06:00
return wrap(createThreadSanitizerLegacyPassPass());
Stub out various legacy PM functions with LLVM 15
2022-04-19 06:22:39 -05:00
#else
report_fatal_error("Legacy PM not supported with LLVM 15");
#endif
Move sanitizer passes creation from ssa to llvm
2019-11-17 18:00:00 -06:00
}
HWASan support
2021-01-22 20:32:38 -06:00
extern "C" LLVMPassRef LLVMRustCreateHWAddressSanitizerPass(bool Recover) {
Stub out various legacy PM functions with LLVM 15
2022-04-19 06:22:39 -05:00
#if LLVM_VERSION_LT(15, 0)
HWASan support
2021-01-22 20:32:38 -06:00
const bool CompileKernel = false;
return wrap(createHWAddressSanitizerLegacyPassPass(CompileKernel, Recover));
Stub out various legacy PM functions with LLVM 15
2022-04-19 06:22:39 -05:00
#else
report_fatal_error("Legacy PM not supported with LLVM 15");
#endif
HWASan support
2021-01-22 20:32:38 -06:00
}
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
extern "C" LLVMRustPassKind LLVMRustPassKind(LLVMPassRef RustPass) {
assert(RustPass);
Pass *Pass = unwrap(RustPass);
return toRust(Pass->getPassKind());
Register LLVM passes with the correct LLVM pass manager. LLVM was upgraded to a new version in this commit: https://github.com/rust-lang/rust/commit/f9d4149c29e8b989fa3624993be379f380e48dcf which was part of this pull request: https://github.com/rust-lang/rust/issues/26025 Consider the following two lines from that commit: https://github.com/rust-lang/rust/commit/f9d4149c29e8b989fa3624993be379f380e48dcf#diff-a3b24dbe2ea7c1981f9ac79f9745f40aL462 https://github.com/rust-lang/rust/commit/f9d4149c29e8b989fa3624993be379f380e48dcf#diff-a3b24dbe2ea7c1981f9ac79f9745f40aL469 The purpose of these lines is to register LLVM passes. Prior to the that commit, the passes being handled were assumed to be ModulePasses (a specific type of LLVM pass) since they were being added to a ModulePass manager. After that commit, both lines were refactored (presumably in an attempt to DRY out the code), but the ModulePasses were changed to be registered to a FunctionPass manager. This change resulted in ModulePasses being run, but a Function object was being passed as a parameter to the pass instead of a Module, which resulted in segmentation faults. In this commit, I changed relevant sections of the code to check the type of the passes being added and register them to the appropriate pass manager. Closes https://github.com/rust-lang/rust/issues/31067
2016-01-24 19:22:24 -06:00
}
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
extern "C" void LLVMRustAddPass(LLVMPassManagerRef PMR, LLVMPassRef RustPass) {
Stub out various legacy PM functions with LLVM 15
2022-04-19 06:22:39 -05:00
#if LLVM_VERSION_LT(15, 0)
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
assert(RustPass);
Pass *Pass = unwrap(RustPass);
PassManagerBase *PMB = unwrap(PMR);
PMB->add(Pass);
Stub out various legacy PM functions with LLVM 15
2022-04-19 06:22:39 -05:00
#else
report_fatal_error("Legacy PM not supported with LLVM 15");
#endif
}
Stub out more PassManagerBuilder functions
2022-04-20 02:25:13 -05:00
extern "C" LLVMPassManagerBuilderRef LLVMRustPassManagerBuilderCreate() {
#if LLVM_VERSION_LT(15, 0)
return LLVMPassManagerBuilderCreate();
#else
report_fatal_error("Legacy PM not supported with LLVM 15");
#endif
}
extern "C" void LLVMRustPassManagerBuilderDispose(LLVMPassManagerBuilderRef PMB) {
#if LLVM_VERSION_LT(15, 0)
LLVMPassManagerBuilderDispose(PMB);
#else
report_fatal_error("Legacy PM not supported with LLVM 15");
#endif
}
extern "C" void LLVMRustPassManagerBuilderPopulateFunctionPassManager(
LLVMPassManagerBuilderRef PMB, LLVMPassManagerRef PM) {
#if LLVM_VERSION_LT(15, 0)
LLVMPassManagerBuilderPopulateFunctionPassManager(PMB, PM);
#else
report_fatal_error("Legacy PM not supported with LLVM 15");
#endif
}
extern "C" void LLVMRustPassManagerBuilderPopulateModulePassManager(
LLVMPassManagerBuilderRef PMB, LLVMPassManagerRef PM) {
#if LLVM_VERSION_LT(15, 0)
LLVMPassManagerBuilderPopulateModulePassManager(PMB, PM);
#else
report_fatal_error("Legacy PM not supported with LLVM 15");
#endif
}
Stub out various legacy PM functions with LLVM 15
2022-04-19 06:22:39 -05:00
extern "C" void LLVMRustPassManagerBuilderPopulateLTOPassManager(
LLVMPassManagerBuilderRef PMB, LLVMPassManagerRef PM, bool Internalize, bool RunInliner) {
#if LLVM_VERSION_LT(15, 0)
LLVMPassManagerBuilderPopulateLTOPassManager(PMB, PM, Internalize, RunInliner);
#else
report_fatal_error("Legacy PM not supported with LLVM 15");
#endif
Rewrite pass management with LLVM Beforehand, it was unclear whether rust was performing the "recommended set" of optimizations provided by LLVM for code. This commit changes the way we run passes to closely mirror that of clang, which in theory does it correctly. The notable changes include: * Passes are no longer explicitly added one by one. This would be difficult to keep up with as LLVM changes and we don't guaranteed always know the best order in which to run passes * Passes are now managed by LLVM's PassManagerBuilder object. This is then used to populate the various pass managers run. * We now run both a FunctionPassManager and a module-wide PassManager. This is what clang does, and I presume that we *may* see a speed boost from the module-wide passes just having to do less work. I have no measured this. * The codegen pass manager has been extracted to its own separate pass manager to not get mixed up with the other passes * All pass managers now include passes for target-specific data layout and analysis passes Some new features include: * You can now print all passes being run with `-Z print-llvm-passes` * When specifying passes via `--passes`, the passes are now appended to the default list of passes instead of overwriting them. * The output of `--passes list` is now generated by LLVM instead of maintaining a list of passes ourselves * Loop vectorization is turned on by default as an optimization pass and can be disabled with `-Z no-vectorize-loops`
2013-08-22 22:58:42 -05:00
}
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
extern "C"
Remove support for building against LLVM 4 With emscripten removed in #55626, we no longer need to support building against LLVM 4.
2018-11-05 07:52:08 -06:00
void LLVMRustPassManagerBuilderPopulateThinLTOPassManager(
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
LLVMPassManagerBuilderRef PMBR,
LLVMPassManagerRef PMR
) {
Stub out various legacy PM functions with LLVM 15
2022-04-19 06:22:39 -05:00
#if LLVM_VERSION_LT(15, 0)
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
unwrap(PMBR)->populateThinLTOPassManager(*unwrap(PMR));
Stub out various legacy PM functions with LLVM 15
2022-04-19 06:22:39 -05:00
#else
report_fatal_error("Legacy PM not supported with LLVM 15");
#endif
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
}
Stub out more PassManagerBuilder functions
2022-04-20 02:25:13 -05:00
extern "C" void LLVMRustPassManagerBuilderUseInlinerWithThreshold(
LLVMPassManagerBuilderRef PMB, unsigned Threshold) {
#if LLVM_VERSION_LT(15, 0)
LLVMPassManagerBuilderUseInlinerWithThreshold(PMB, Threshold);
#else
report_fatal_error("Legacy PM not supported with LLVM 15");
#endif
}
Don't add extra passes into the function pass manager Exception for specific cases like linting, additional passes should be going into the module pass manager (even if they are function passes). The separate function pass manager is only used for very early optimization passes. Rather than apparending passes to the MPM, use the OptimizerLast and EnabledOnOptLevel0 pass manager builder extension hooks, which allow adding passes directly before finalization (alias canonicalization and name-anon-globals). The main effect and purpose of this change is to add sanitizer passes at the end of the pipeline, which is where they belong. In LLVM 9 the address sanitizer can't be used as a pass in the early function pass manager, because it has a dependence on a module-level analysis pass.
2019-07-13 13:17:16 -05:00
extern "C"
void LLVMRustAddLastExtensionPasses(
LLVMPassManagerBuilderRef PMBR, LLVMPassRef *Passes, size_t NumPasses) {
Stub out various legacy PM functions with LLVM 15
2022-04-19 06:22:39 -05:00
#if LLVM_VERSION_LT(15, 0)
Don't add extra passes into the function pass manager Exception for specific cases like linting, additional passes should be going into the module pass manager (even if they are function passes). The separate function pass manager is only used for very early optimization passes. Rather than apparending passes to the MPM, use the OptimizerLast and EnabledOnOptLevel0 pass manager builder extension hooks, which allow adding passes directly before finalization (alias canonicalization and name-anon-globals). The main effect and purpose of this change is to add sanitizer passes at the end of the pipeline, which is where they belong. In LLVM 9 the address sanitizer can't be used as a pass in the early function pass manager, because it has a dependence on a module-level analysis pass.
2019-07-13 13:17:16 -05:00
auto AddExtensionPasses = [Passes, NumPasses](
const PassManagerBuilder &Builder, PassManagerBase &PM) {
for (size_t I = 0; I < NumPasses; I++) {
PM.add(unwrap(Passes[I]));
}
};
// Add the passes to both of the pre-finalization extension points,
// so they are run for optimized and non-optimized builds.
unwrap(PMBR)->addExtension(PassManagerBuilder::EP_OptimizerLast,
AddExtensionPasses);
unwrap(PMBR)->addExtension(PassManagerBuilder::EP_EnabledOnOptLevel0,
AddExtensionPasses);
Stub out various legacy PM functions with LLVM 15
2022-04-19 06:22:39 -05:00
#else
report_fatal_error("Legacy PM not supported with LLVM 15");
#endif
Don't add extra passes into the function pass manager Exception for specific cases like linting, additional passes should be going into the module pass manager (even if they are function passes). The separate function pass manager is only used for very early optimization passes. Rather than apparending passes to the MPM, use the OptimizerLast and EnabledOnOptLevel0 pass manager builder extension hooks, which allow adding passes directly before finalization (alias canonicalization and name-anon-globals). The main effect and purpose of this change is to add sanitizer passes at the end of the pipeline, which is where they belong. In LLVM 9 the address sanitizer can't be used as a pass in the early function pass manager, because it has a dependence on a module-level analysis pass.
2019-07-13 13:17:16 -05:00
}
Implement feature extraction from `TargetMachine` Add the `LLVMRustHasFeature` function to check whether a `TargetMachine` has a given feature.
2016-02-16 10:07:30 -06:00
#ifdef LLVM_COMPONENT_X86
#define SUBTARGET_X86 SUBTARGET(X86)
#else
#define SUBTARGET_X86
#endif
#ifdef LLVM_COMPONENT_ARM
#define SUBTARGET_ARM SUBTARGET(ARM)
#else
#define SUBTARGET_ARM
#endif
#ifdef LLVM_COMPONENT_AARCH64
#define SUBTARGET_AARCH64 SUBTARGET(AArch64)
#else
#define SUBTARGET_AARCH64
#endif
[AVR] Add AVR platform support
2016-05-06 08:32:10 -05:00
#ifdef LLVM_COMPONENT_AVR
#define SUBTARGET_AVR SUBTARGET(AVR)
#else
#define SUBTARGET_AVR
#endif
compiler/rustc_llvm: Enable M68k LLVM target
2021-08-25 02:49:17 -05:00
#ifdef LLVM_COMPONENT_M68k
#define SUBTARGET_M68K SUBTARGET(M68k)
#else
#define SUBTARGET_M68K
#endif
Implement feature extraction from `TargetMachine` Add the `LLVMRustHasFeature` function to check whether a `TargetMachine` has a given feature.
2016-02-16 10:07:30 -06:00
#ifdef LLVM_COMPONENT_MIPS
#define SUBTARGET_MIPS SUBTARGET(Mips)
#else
#define SUBTARGET_MIPS
#endif
#ifdef LLVM_COMPONENT_POWERPC
#define SUBTARGET_PPC SUBTARGET(PPC)
#else
#define SUBTARGET_PPC
#endif
build llvm with systemz backend enabled, and link to related libraries when building rust against system llvm closes #36077
2016-08-28 13:18:28 -05:00
#ifdef LLVM_COMPONENT_SYSTEMZ
#define SUBTARGET_SYSTEMZ SUBTARGET(SystemZ)
#else
#define SUBTARGET_SYSTEMZ
#endif
also enable the MSP430 backend in Makefiles
2016-11-09 15:56:10 -06:00
#ifdef LLVM_COMPONENT_MSP430
#define SUBTARGET_MSP430 SUBTARGET(MSP430)
#else
#define SUBTARGET_MSP430
#endif
[RISCV] Enable LLVM backend.
2018-07-24 05:03:28 -05:00
#ifdef LLVM_COMPONENT_RISCV
#define SUBTARGET_RISCV SUBTARGET(RISCV)
#else
#define SUBTARGET_RISCV
#endif
further enable the Sparc LLVM backend
2016-12-03 10:53:31 -06:00
#ifdef LLVM_COMPONENT_SPARC
#define SUBTARGET_SPARC SUBTARGET(Sparc)
#else
#define SUBTARGET_SPARC
#endif
Add Hexagon support This requires an updated LLVM with D31999 and D32000 to build libcore. A basic hello world builds and runs successfully on the hexagon simulator.
2017-04-09 01:03:31 -05:00
#ifdef LLVM_COMPONENT_HEXAGON
#define SUBTARGET_HEXAGON SUBTARGET(Hexagon)
#else
#define SUBTARGET_HEXAGON
#endif
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
#define GEN_SUBTARGETS \
SUBTARGET_X86 \
SUBTARGET_ARM \
SUBTARGET_AARCH64 \
[AVR] Add AVR platform support
2016-05-06 08:32:10 -05:00
SUBTARGET_AVR \
compiler/rustc_llvm: Enable M68k LLVM target
2021-08-25 02:49:17 -05:00
SUBTARGET_M68K \
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
SUBTARGET_MIPS \
SUBTARGET_PPC \
SUBTARGET_SYSTEMZ \
Merge branch 'master' into sparc64
2016-12-31 21:40:10 -06:00
SUBTARGET_MSP430 \
Add Hexagon support This requires an updated LLVM with D31999 and D32000 to build libcore. A basic hello world builds and runs successfully on the hexagon simulator.
2017-04-09 01:03:31 -05:00
SUBTARGET_SPARC \
[RISCV] Enable LLVM backend.
2018-07-24 05:03:28 -05:00
SUBTARGET_HEXAGON \
SUBTARGET_RISCV \
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
#define SUBTARGET(x) \
namespace llvm { \
extern const SubtargetFeatureKV x##FeatureKV[]; \
extern const SubtargetFeatureKV x##SubTypeKV[]; \
Implement feature extraction from `TargetMachine` Add the `LLVMRustHasFeature` function to check whether a `TargetMachine` has a given feature.
2016-02-16 10:07:30 -06:00
}
GEN_SUBTARGETS
#undef SUBTARGET
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
extern "C" bool LLVMRustHasFeature(LLVMTargetMachineRef TM,
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
const char *Feature) {
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
TargetMachine *Target = unwrap(TM);
const MCSubtargetInfo *MCInfo = Target->getMCSubtargetInfo();
Enable target_feature on any LLVM 6+ In `LLVMRustHasFeature()`, rather than using `MCInfo->getFeatureTable()` that is specific to Rust's LLVM fork, we can use this in LLVM 6: /// Check whether the subtarget features are enabled/disabled as per /// the provided string, ignoring all other features. bool checkFeatures(StringRef FS) const; Now rustc using external LLVM can also have `target_feature`.
2018-03-27 14:27:45 -05:00
return MCInfo->checkFeatures(std::string("+") + Feature);
Implement feature extraction from `TargetMachine` Add the `LLVMRustHasFeature` function to check whether a `TargetMachine` has a given feature.
2016-02-16 10:07:30 -06:00
}
audit LLVM C++ types in ArchiveWrapper and PassWrapper
2016-08-01 16:16:16 -05:00
enum class LLVMRustCodeModel {
rustc_target: Stop using "string typing" for code models Introduce `enum CodeModel` instead.
2020-05-06 19:34:27 -05:00
Tiny,
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
Small,
Kernel,
Medium,
Large,
llvm6: CodeModel::{JIT,}Default no longer exists LLVM has since removed the `CodeModel::Default` enum value in favor of an `Optional` implementationg throughout LLVM. Let's mirror the same change in Rust and update the various bindings we call accordingly. Removed in llvm-mirror/llvm@9aafb854c
2018-01-22 19:01:36 -06:00
None,
audit LLVM C++ types in ArchiveWrapper and PassWrapper
2016-08-01 16:16:16 -05:00
};
rustc_target: Stop using "string typing" for code models Introduce `enum CodeModel` instead.
2020-05-06 19:34:27 -05:00
static Optional<CodeModel::Model> fromRust(LLVMRustCodeModel Model) {
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
switch (Model) {
rustc_target: Stop using "string typing" for code models Introduce `enum CodeModel` instead.
2020-05-06 19:34:27 -05:00
case LLVMRustCodeModel::Tiny:
return CodeModel::Tiny;
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
case LLVMRustCodeModel::Small:
return CodeModel::Small;
case LLVMRustCodeModel::Kernel:
return CodeModel::Kernel;
case LLVMRustCodeModel::Medium:
return CodeModel::Medium;
case LLVMRustCodeModel::Large:
return CodeModel::Large;
rustc_target: Stop using "string typing" for code models Introduce `enum CodeModel` instead.
2020-05-06 19:34:27 -05:00
case LLVMRustCodeModel::None:
return None;
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
default:
[rustllvm] Use report_fatal_error over llvm_unreachable This makes it more robust when assertions are disabled, crashing instead of causing UB. Also introduces a tidy check to enforce this rule, which in turn necessitated making tidy run on src/rustllvm. Fixes #44020
2017-11-20 10:47:29 -06:00
report_fatal_error("Bad CodeModel.");
audit LLVM C++ types in ArchiveWrapper and PassWrapper
2016-08-01 16:16:16 -05:00
}
}
enum class LLVMRustCodeGenOptLevel {
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
None,
Less,
Default,
Aggressive,
audit LLVM C++ types in ArchiveWrapper and PassWrapper
2016-08-01 16:16:16 -05:00
};
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
static CodeGenOpt::Level fromRust(LLVMRustCodeGenOptLevel Level) {
switch (Level) {
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
case LLVMRustCodeGenOptLevel::None:
return CodeGenOpt::None;
case LLVMRustCodeGenOptLevel::Less:
return CodeGenOpt::Less;
case LLVMRustCodeGenOptLevel::Default:
return CodeGenOpt::Default;
case LLVMRustCodeGenOptLevel::Aggressive:
return CodeGenOpt::Aggressive;
default:
[rustllvm] Use report_fatal_error over llvm_unreachable This makes it more robust when assertions are disabled, crashing instead of causing UB. Also introduces a tidy check to enforce this rule, which in turn necessitated making tidy run on src/rustllvm. Fixes #44020
2017-11-20 10:47:29 -06:00
report_fatal_error("Bad CodeGenOptLevel.");
audit LLVM C++ types in ArchiveWrapper and PassWrapper
2016-08-01 16:16:16 -05:00
}
}
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
enum class LLVMRustPassBuilderOptLevel {
O0,
O1,
O2,
O3,
Os,
Oz,
};
PassWrapper: handle move of OptimizationLevel class out of PassBuilder This is the first build break of the LLVM 14 cycle, and was caused by https://reviews.llvm.org/D107025. Mercifully an easy fix.
2021-08-05 11:13:58 -05:00
#if LLVM_VERSION_LT(14,0)
using OptimizationLevel = PassBuilder::OptimizationLevel;
#endif
static OptimizationLevel fromRust(LLVMRustPassBuilderOptLevel Level) {
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
switch (Level) {
case LLVMRustPassBuilderOptLevel::O0:
PassWrapper: handle move of OptimizationLevel class out of PassBuilder This is the first build break of the LLVM 14 cycle, and was caused by https://reviews.llvm.org/D107025. Mercifully an easy fix.
2021-08-05 11:13:58 -05:00
return OptimizationLevel::O0;
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
case LLVMRustPassBuilderOptLevel::O1:
PassWrapper: handle move of OptimizationLevel class out of PassBuilder This is the first build break of the LLVM 14 cycle, and was caused by https://reviews.llvm.org/D107025. Mercifully an easy fix.
2021-08-05 11:13:58 -05:00
return OptimizationLevel::O1;
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
case LLVMRustPassBuilderOptLevel::O2:
PassWrapper: handle move of OptimizationLevel class out of PassBuilder This is the first build break of the LLVM 14 cycle, and was caused by https://reviews.llvm.org/D107025. Mercifully an easy fix.
2021-08-05 11:13:58 -05:00
return OptimizationLevel::O2;
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
case LLVMRustPassBuilderOptLevel::O3:
PassWrapper: handle move of OptimizationLevel class out of PassBuilder This is the first build break of the LLVM 14 cycle, and was caused by https://reviews.llvm.org/D107025. Mercifully an easy fix.
2021-08-05 11:13:58 -05:00
return OptimizationLevel::O3;
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
case LLVMRustPassBuilderOptLevel::Os:
PassWrapper: handle move of OptimizationLevel class out of PassBuilder This is the first build break of the LLVM 14 cycle, and was caused by https://reviews.llvm.org/D107025. Mercifully an easy fix.
2021-08-05 11:13:58 -05:00
return OptimizationLevel::Os;
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
case LLVMRustPassBuilderOptLevel::Oz:
PassWrapper: handle move of OptimizationLevel class out of PassBuilder This is the first build break of the LLVM 14 cycle, and was caused by https://reviews.llvm.org/D107025. Mercifully an easy fix.
2021-08-05 11:13:58 -05:00
return OptimizationLevel::Oz;
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
default:
report_fatal_error("Bad PassBuilderOptLevel.");
}
}
codegen_llvm: `RelocMode` -> `RelocModel`
2020-04-23 12:49:00 -05:00
enum class LLVMRustRelocModel {
Added LLVMRustRelocMode Replaces the llvm-c exposed LLVMRelocMode, which does not include all relocation model variants, with a LLVMRustRelocMode modeled after LLVMRustCodeMode.
2017-04-28 17:21:59 -05:00
Static,
PIC,
DynamicNoPic,
ROPI,
RWPI,
ROPIRWPI,
};
codegen_llvm: `RelocMode` -> `RelocModel`
2020-04-23 12:49:00 -05:00
static Reloc::Model fromRust(LLVMRustRelocModel RustReloc) {
Added LLVMRustRelocMode Replaces the llvm-c exposed LLVMRelocMode, which does not include all relocation model variants, with a LLVMRustRelocMode modeled after LLVMRustCodeMode.
2017-04-28 17:21:59 -05:00
switch (RustReloc) {
codegen_llvm: `RelocMode` -> `RelocModel`
2020-04-23 12:49:00 -05:00
case LLVMRustRelocModel::Static:
Added LLVMRustRelocMode Replaces the llvm-c exposed LLVMRelocMode, which does not include all relocation model variants, with a LLVMRustRelocMode modeled after LLVMRustCodeMode.
2017-04-28 17:21:59 -05:00
return Reloc::Static;
codegen_llvm: `RelocMode` -> `RelocModel`
2020-04-23 12:49:00 -05:00
case LLVMRustRelocModel::PIC:
Added LLVMRustRelocMode Replaces the llvm-c exposed LLVMRelocMode, which does not include all relocation model variants, with a LLVMRustRelocMode modeled after LLVMRustCodeMode.
2017-04-28 17:21:59 -05:00
return Reloc::PIC_;
codegen_llvm: `RelocMode` -> `RelocModel`
2020-04-23 12:49:00 -05:00
case LLVMRustRelocModel::DynamicNoPic:
Added LLVMRustRelocMode Replaces the llvm-c exposed LLVMRelocMode, which does not include all relocation model variants, with a LLVMRustRelocMode modeled after LLVMRustCodeMode.
2017-04-28 17:21:59 -05:00
return Reloc::DynamicNoPIC;
codegen_llvm: `RelocMode` -> `RelocModel`
2020-04-23 12:49:00 -05:00
case LLVMRustRelocModel::ROPI:
Added LLVMRustRelocMode Replaces the llvm-c exposed LLVMRelocMode, which does not include all relocation model variants, with a LLVMRustRelocMode modeled after LLVMRustCodeMode.
2017-04-28 17:21:59 -05:00
return Reloc::ROPI;
codegen_llvm: `RelocMode` -> `RelocModel`
2020-04-23 12:49:00 -05:00
case LLVMRustRelocModel::RWPI:
Added LLVMRustRelocMode Replaces the llvm-c exposed LLVMRelocMode, which does not include all relocation model variants, with a LLVMRustRelocMode modeled after LLVMRustCodeMode.
2017-04-28 17:21:59 -05:00
return Reloc::RWPI;
codegen_llvm: `RelocMode` -> `RelocModel`
2020-04-23 12:49:00 -05:00
case LLVMRustRelocModel::ROPIRWPI:
Added LLVMRustRelocMode Replaces the llvm-c exposed LLVMRelocMode, which does not include all relocation model variants, with a LLVMRustRelocMode modeled after LLVMRustCodeMode.
2017-04-28 17:21:59 -05:00
return Reloc::ROPI_RWPI;
}
[rustllvm] Use report_fatal_error over llvm_unreachable This makes it more robust when assertions are disabled, crashing instead of causing UB. Also introduces a tidy check to enforce this rule, which in turn necessitated making tidy run on src/rustllvm. Fixes #44020
2017-11-20 10:47:29 -06:00
report_fatal_error("Bad RelocModel.");
Added LLVMRustRelocMode Replaces the llvm-c exposed LLVMRelocMode, which does not include all relocation model variants, with a LLVMRustRelocMode modeled after LLVMRustCodeMode.
2017-04-28 17:21:59 -05:00
}
Enable -mergefunc-use-aliases If the Rust LLVM fork is used, enable the -mergefunc-use-aliases flag, which will create aliases for merged functions, rather than inserting a call from one to the other. A number of codegen tests needed to be adjusted, because functions that previously fell below the thunk limit are now being merged. Merging is prevented either using -C no-prepopulate-passes, or by making the functions non-identical. I expect that this is going to break something, somewhere, because it isn't able to deal with aliases properly, but we won't find out until we try :) This fixes #52651.
2018-11-29 16:05:23 -06:00
#ifdef LLVM_RUSTLLVM
Add help for target CPUs, features, relocation and code models.
2016-07-10 09:22:13 -05:00
/// getLongestEntryLength - Return the length of the longest entry in the table.
Update CPU printing for SubtargetSubTypeKV CPUs now use SubtargetSubTypeKV rather than SubtargetFeatureKV and no longer have a description.
2019-07-06 11:52:26 -05:00
template<typename KV>
static size_t getLongestEntryLength(ArrayRef<KV> Table) {
Add help for target CPUs, features, relocation and code models.
2016-07-10 09:22:13 -05:00
size_t MaxLen = 0;
for (auto &I : Table)
MaxLen = std::max(MaxLen, std::strlen(I.Key));
return MaxLen;
}
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
extern "C" void LLVMRustPrintTargetCPUs(LLVMTargetMachineRef TM) {
const TargetMachine *Target = unwrap(TM);
const MCSubtargetInfo *MCInfo = Target->getMCSubtargetInfo();
Add 'native' to -C target-cpu=help
2017-09-07 20:07:16 -05:00
const Triple::ArchType HostArch = Triple(sys::getProcessTriple()).getArch();
const Triple::ArchType TargetArch = Target->getTargetTriple().getArch();
Update CPU printing for SubtargetSubTypeKV CPUs now use SubtargetSubTypeKV rather than SubtargetFeatureKV and no longer have a description.
2019-07-06 11:52:26 -05:00
const ArrayRef<SubtargetSubTypeKV> CPUTable = MCInfo->getCPUTable();
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
unsigned MaxCPULen = getLongestEntryLength(CPUTable);
printf("Available CPUs for this target:\n");
Add 'native' to -C target-cpu=help
2017-09-07 20:07:16 -05:00
if (HostArch == TargetArch) {
const StringRef HostCPU = sys::getHostCPUName();
printf(" %-*s - Select the CPU of the current host (currently %.*s).\n",
MaxCPULen, "native", (int)HostCPU.size(), HostCPU.data());
}
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
for (auto &CPU : CPUTable)
Update CPU printing for SubtargetSubTypeKV CPUs now use SubtargetSubTypeKV rather than SubtargetFeatureKV and no longer have a description.
2019-07-06 11:52:26 -05:00
printf(" %-*s\n", MaxCPULen, CPU.Key);
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
printf("\n");
Add help for target CPUs, features, relocation and code models.
2016-07-10 09:22:13 -05:00
}
Categorize and explain target features support
2021-04-08 00:00:47 -05:00
extern "C" size_t LLVMRustGetTargetFeaturesCount(LLVMTargetMachineRef TM) {
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
const TargetMachine *Target = unwrap(TM);
const MCSubtargetInfo *MCInfo = Target->getMCSubtargetInfo();
const ArrayRef<SubtargetFeatureKV> FeatTable = MCInfo->getFeatureTable();
Categorize and explain target features support
2021-04-08 00:00:47 -05:00
return FeatTable.size();
}
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
Categorize and explain target features support
2021-04-08 00:00:47 -05:00
extern "C" void LLVMRustGetTargetFeature(LLVMTargetMachineRef TM, size_t Index,
const char** Feature, const char** Desc) {
const TargetMachine *Target = unwrap(TM);
const MCSubtargetInfo *MCInfo = Target->getMCSubtargetInfo();
const ArrayRef<SubtargetFeatureKV> FeatTable = MCInfo->getFeatureTable();
const SubtargetFeatureKV Feat = FeatTable[Index];
*Feature = Feat.Key;
*Desc = Feat.Desc;
Add help for target CPUs, features, relocation and code models.
2016-07-10 09:22:13 -05:00
}
Pass -DLLVM_RUSTLLVM to compile against rust llvm fork. If using system llvm don't try use modifications made in the fork.
2016-07-24 04:49:10 -05:00
#else
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
extern "C" void LLVMRustPrintTargetCPUs(LLVMTargetMachineRef) {
printf("Target CPU help is not supported by this LLVM version.\n\n");
Pass -DLLVM_RUSTLLVM to compile against rust llvm fork. If using system llvm don't try use modifications made in the fork.
2016-07-24 04:49:10 -05:00
}
Categorize and explain target features support
2021-04-08 00:00:47 -05:00
extern "C" size_t LLVMRustGetTargetFeaturesCount(LLVMTargetMachineRef) {
return 0;
Pass -DLLVM_RUSTLLVM to compile against rust llvm fork. If using system llvm don't try use modifications made in the fork.
2016-07-24 04:49:10 -05:00
}
Categorize and explain target features support
2021-04-08 00:00:47 -05:00
extern "C" void LLVMRustGetTargetFeature(LLVMTargetMachineRef, const char**, const char**) {}
Pass -DLLVM_RUSTLLVM to compile against rust llvm fork. If using system llvm don't try use modifications made in the fork.
2016-07-24 04:49:10 -05:00
#endif
Fix warnings about the `native` target-cpu This fixes a regression from #53031 where specifying `-C target-cpu=native` is printing a lot of warnings from LLVM about `native` being an unknown CPU. It turns out that `native` is indeed an unknown CPU and we have to perform a mapping to an actual CPU name, but this mapping is only performed in one location rather than all locations we inform LLVM about the target CPU. This commit centralizes the mapping of `native` to LLVM's value of the native CPU, ensuring that all locations we inform LLVM about the `target-cpu` it's never `native`. Closes #53322
2018-08-23 13:03:22 -05:00
extern "C" const char* LLVMRustGetHostCPUName(size_t *len) {
StringRef Name = sys::getHostCPUName();
*len = Name.size();
return Name.data();
}
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
extern "C" LLVMTargetMachineRef LLVMRustCreateTargetMachine(
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
const char *TripleStr, const char *CPU, const char *Feature,
codegen_llvm: `RelocMode` -> `RelocModel`
2020-04-23 12:49:00 -05:00
const char *ABIStr, LLVMRustCodeModel RustCM, LLVMRustRelocModel RustReloc,
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
LLVMRustCodeGenOptLevel RustOptLevel, bool UseSoftFloat,
rustllvm: Fix warnings about unused function parameters
2020-05-21 12:53:41 -05:00
bool FunctionSections,
std: Add a new wasm32-unknown-unknown target This commit adds a new target to the compiler: wasm32-unknown-unknown. This target is a reimagining of what it looks like to generate WebAssembly code from Rust. Instead of using Emscripten which can bring with it a weighty runtime this instead is a target which uses only the LLVM backend for WebAssembly and a "custom linker" for now which will hopefully one day be direct calls to lld. Notable features of this target include: * There is zero runtime footprint. The target assumes nothing exists other than the wasm32 instruction set. * There is zero toolchain footprint beyond adding the target. No custom linker is needed, rustc contains everything. * Very small wasm modules can be generated directly from Rust code using this target. * Most of the standard library is stubbed out to return an error, but anything related to allocation works (aka `HashMap`, `Vec`, etc). * Naturally, any `#[no_std]` crate should be 100% compatible with this new target. This target is currently somewhat janky due to how linking works. The "linking" is currently unconditional whole program LTO (aka LLVM is being used as a linker). Naturally that means compiling programs is pretty slow! Eventually though this target should have a linker. This target is also intended to be quite experimental. I'm hoping that this can act as a catalyst for further experimentation in Rust with WebAssembly. Breaking changes are very likely to land to this target, so it's not recommended to rely on it in any critical capacity yet. We'll let you know when it's "production ready". --- Currently testing-wise this target is looking pretty good but isn't complete. I've got almost the entire `run-pass` test suite working with this target (lots of tests ignored, but many passing as well). The `core` test suite is still getting LLVM bugs fixed to get that working and will take some time. Relatively simple programs all seem to work though! --- It's worth nothing that you may not immediately see the "smallest possible wasm module" for the input you feed to rustc. For various reasons it's very difficult to get rid of the final "bloat" in vanilla rustc (again, a real linker should fix all this). For now what you'll have to do is: cargo install --git https://github.com/alexcrichton/wasm-gc wasm-gc foo.wasm bar.wasm And then `bar.wasm` should be the smallest we can get it! --- In any case for now I'd love feedback on this, particularly on the various integration points if you've got better ideas of how to approach them!
2017-10-22 22:01:00 -05:00
bool DataSections,
Add -Z no-unique-section-names to reduce ELF header bloat. This change adds a new compiler flag that can help reduce the size of ELF binaries that contain many functions. By default, when enabling function sections (which is the default for most targets), the LLVM backend will generate different section names for each function. For example, a function "func" would generate a section called ".text.func". Normally this is fine because the linker will merge all those sections into a single one in the binary. However, starting with LLVM 12 (llvm/llvm-project@ee5d1a0), the backend will also generate unique section names for exception handling, resulting in thousands of ".gcc_except_table.*" sections ending up in the final binary because some linkers don't currently merge or strip these EH sections. This can bloat the ELF headers and string table significantly in binaries that contain many functions. The new option is analogous to Clang's -fno-unique-section-names, and instructs LLVM to generate the same ".text" and ".gcc_except_table" section for each function, resulting in smaller object files and potentially a smaller final binary.
2021-10-11 14:09:32 -05:00
bool UniqueSectionNames,
std: Add a new wasm32-unknown-unknown target This commit adds a new target to the compiler: wasm32-unknown-unknown. This target is a reimagining of what it looks like to generate WebAssembly code from Rust. Instead of using Emscripten which can bring with it a weighty runtime this instead is a target which uses only the LLVM backend for WebAssembly and a "custom linker" for now which will hopefully one day be direct calls to lld. Notable features of this target include: * There is zero runtime footprint. The target assumes nothing exists other than the wasm32 instruction set. * There is zero toolchain footprint beyond adding the target. No custom linker is needed, rustc contains everything. * Very small wasm modules can be generated directly from Rust code using this target. * Most of the standard library is stubbed out to return an error, but anything related to allocation works (aka `HashMap`, `Vec`, etc). * Naturally, any `#[no_std]` crate should be 100% compatible with this new target. This target is currently somewhat janky due to how linking works. The "linking" is currently unconditional whole program LTO (aka LLVM is being used as a linker). Naturally that means compiling programs is pretty slow! Eventually though this target should have a linker. This target is also intended to be quite experimental. I'm hoping that this can act as a catalyst for further experimentation in Rust with WebAssembly. Breaking changes are very likely to land to this target, so it's not recommended to rely on it in any critical capacity yet. We'll let you know when it's "production ready". --- Currently testing-wise this target is looking pretty good but isn't complete. I've got almost the entire `run-pass` test suite working with this target (lots of tests ignored, but many passing as well). The `core` test suite is still getting LLVM bugs fixed to get that working and will take some time. Relatively simple programs all seem to work though! --- It's worth nothing that you may not immediately see the "smallest possible wasm module" for the input you feed to rustc. For various reasons it's very difficult to get rid of the final "bloat" in vanilla rustc (again, a real linker should fix all this). For now what you'll have to do is: cargo install --git https://github.com/alexcrichton/wasm-gc wasm-gc foo.wasm bar.wasm And then `bar.wasm` should be the smallest we can get it! --- In any case for now I'd love feedback on this, particularly on the various integration points if you've got better ideas of how to approach them!
2017-10-22 22:01:00 -05:00
bool TrapUnreachable,
Make LLVM emit assembly comments with -Z asm-comments. Fixes #35741.
2018-08-12 12:59:18 -05:00
bool Singlethread,
add -Z emit-stack-sizes
2018-09-13 12:43:15 -05:00
bool AsmComments,
Change linker for x86_64-fortanix-unknown-sgx to rust-lld For SGX, the relocation using the relocation table is done by the code in rust/src/libstd/sys/sgx/abi/reloc.rs and this code should not require relocation. Setting RelaxELFRelocations flag if allows this to happen, hence adding a Target Option for it.
2019-12-02 06:22:45 -06:00
bool EmitStackSizeSection,
Use .init_array rather than .ctors LLVM TargetMachines default to using the (now-legacy) .ctors representation of init functions. Mixing .ctors and .init_array representations can cause issues when linking with lld. This happens in practice for: * Our profiling runtime which is currently implicitly built with .init_array since it is built by clang, which sets this field. * External C/C++ code that may be linked into the same process. To support legacy systems which may use .ctors, targets may now specify that they use .ctors via the use_ctors attribute which defaults to false. For debugging and manual control, -Z use-ctors-section=yes/no will allow manual override. Fixes: #71233
2020-04-16 21:40:11 -05:00
bool RelaxELFRelocations,
llvm: update ffi bindings for split dwarf This commit modifies the FFI bindings to LLVM required for Split DWARF support in rustc. In particular: - `addPassesToEmitFile`'s wrapper, `LLVMRustWriteOutputFile` now takes a `DwoPath` `const char*`. When disabled, `nullptr` should be provided which will preserve existing behaviour. When enabled, the path to the `.dwo` file should be provided. - `createCompileUnit`'s wrapper, `LLVMRustDIBuilderCreateCompileUnit` now has two additional arguments, for the `DWOId` and to enable `SplitDebugInlining`. `DWOId` should always be zero. - `createTargetMachine`'s wrapper, `LLVMRustCreateTargetMachine` has an additional argument which should be provided the path to the `.dwo` when enabled. Signed-off-by: David Wood <david@davidtw.co>
2020-09-23 10:25:20 -05:00
bool UseInitArray,
const char *SplitDwarfFile) {
Use C type when passing value to LLVM pass Previously the C type LLVMRelocMode (available as RelocMode in Rust) was passed as is to the function. However createTargetMachine expects a Reloc::Model, which is an enum just one value short. Additionally, the function was marked as requiring Reloc::Model in the C code, but RelocMode on the Rust-side. We now use the correct C type LLVMRelocMode and convert it to an Optional<Reloc::Model> as expected by the createTargetMachine call the same the original LLVMCreateTargetMachine function does. See https://github.com/llvm-mirror/llvm/blob/c9b262bfbd5b9fb6f10749dba1465569f39bd625/lib/Target/TargetMachineC.cpp#L104-L121 This was found by @eddyb.
2016-07-24 15:31:16 -05:00
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
auto OptLevel = fromRust(RustOptLevel);
Added LLVMRustRelocMode Replaces the llvm-c exposed LLVMRelocMode, which does not include all relocation model variants, with a LLVMRustRelocMode modeled after LLVMRustCodeMode.
2017-04-28 17:21:59 -05:00
auto RM = fromRust(RustReloc);
rustc_target: Stop using "string typing" for code models Introduce `enum CodeModel` instead.
2020-05-06 19:34:27 -05:00
auto CM = fromRust(RustCM);
Use C type when passing value to LLVM pass Previously the C type LLVMRelocMode (available as RelocMode in Rust) was passed as is to the function. However createTargetMachine expects a Reloc::Model, which is an enum just one value short. Additionally, the function was marked as requiring Reloc::Model in the C code, but RelocMode on the Rust-side. We now use the correct C type LLVMRelocMode and convert it to an Optional<Reloc::Model> as expected by the createTargetMachine call the same the original LLVMCreateTargetMachine function does. See https://github.com/llvm-mirror/llvm/blob/c9b262bfbd5b9fb6f10749dba1465569f39bd625/lib/Target/TargetMachineC.cpp#L104-L121 This was found by @eddyb.
2016-07-24 15:31:16 -05:00
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
std::string Error;
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
Triple Trip(Triple::normalize(TripleStr));
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
const llvm::Target *TheTarget =
TargetRegistry::lookupTarget(Trip.getTriple(), Error);
Switching from NULL to nullptr in src/rustllvm.
2016-12-30 06:21:21 -06:00
if (TheTarget == nullptr) {
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
LLVMRustSetLastError(Error.c_str());
Switching from NULL to nullptr in src/rustllvm.
2016-12-30 06:21:21 -06:00
return nullptr;
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
}
Rewrite pass management with LLVM Beforehand, it was unclear whether rust was performing the "recommended set" of optimizations provided by LLVM for code. This commit changes the way we run passes to closely mirror that of clang, which in theory does it correctly. The notable changes include: * Passes are no longer explicitly added one by one. This would be difficult to keep up with as LLVM changes and we don't guaranteed always know the best order in which to run passes * Passes are now managed by LLVM's PassManagerBuilder object. This is then used to populate the various pass managers run. * We now run both a FunctionPassManager and a module-wide PassManager. This is what clang does, and I presume that we *may* see a speed boost from the module-wide passes just having to do less work. I have no measured this. * The codegen pass manager has been extracted to its own separate pass manager to not get mixed up with the other passes * All pass managers now include passes for target-specific data layout and analysis passes Some new features include: * You can now print all passes being run with `-Z print-llvm-passes` * When specifying passes via `--passes`, the passes are now appended to the default list of passes instead of overwriting them. * The output of `--passes list` is now generated by LLVM instead of maintaining a list of passes ourselves * Loop vectorization is turned on by default as an optimization pass and can be disabled with `-Z no-vectorize-loops`
2013-08-22 22:58:42 -05:00
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
TargetOptions Options;
[LLVM-3.9] Configure PIE at the module level instead of compilation unit level This was deleted here[1] which appears to be replaced by this[2] which is a new setPIELevel function on the LLVM module itself. [1]: http://reviews.llvm.org/D19753 [2]: http://reviews.llvm.org/D19671
2016-07-12 17:41:40 -05:00
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
Options.FloatABIType = FloatABI::Default;
if (UseSoftFloat) {
Options.FloatABIType = FloatABI::Soft;
}
Options.DataSections = DataSections;
Options.FunctionSections = FunctionSections;
Add -Z no-unique-section-names to reduce ELF header bloat. This change adds a new compiler flag that can help reduce the size of ELF binaries that contain many functions. By default, when enabling function sections (which is the default for most targets), the LLVM backend will generate different section names for each function. For example, a function "func" would generate a section called ".text.func". Normally this is fine because the linker will merge all those sections into a single one in the binary. However, starting with LLVM 12 (llvm/llvm-project@ee5d1a0), the backend will also generate unique section names for exception handling, resulting in thousands of ".gcc_except_table.*" sections ending up in the final binary because some linkers don't currently merge or strip these EH sections. This can bloat the ELF headers and string table significantly in binaries that contain many functions. The new option is analogous to Clang's -fno-unique-section-names, and instructs LLVM to generate the same ".text" and ".gcc_except_table" section for each function, resulting in smaller object files and potentially a smaller final binary.
2021-10-11 14:09:32 -05:00
Options.UniqueSectionNames = UniqueSectionNames;
Make LLVM emit assembly comments with -Z asm-comments. Fixes #35741.
2018-08-12 12:59:18 -05:00
Options.MCOptions.AsmVerbose = AsmComments;
Options.MCOptions.PreserveAsmComments = AsmComments;
Allow specifying key "llvm-abiname" in target specification This addresses #65024, as it allows RISC-V target specification files to set "llvm-abiname": "lp64d". In general, it is useful for the programmer to be able to set this codegen parameter, which other languages usually expose under a compiler argument like "-mabi=<XYZ>".
2019-10-29 23:12:05 -05:00
Options.MCOptions.ABIName = ABIStr;
llvm: update ffi bindings for split dwarf This commit modifies the FFI bindings to LLVM required for Split DWARF support in rustc. In particular: - `addPassesToEmitFile`'s wrapper, `LLVMRustWriteOutputFile` now takes a `DwoPath` `const char*`. When disabled, `nullptr` should be provided which will preserve existing behaviour. When enabled, the path to the `.dwo` file should be provided. - `createCompileUnit`'s wrapper, `LLVMRustDIBuilderCreateCompileUnit` now has two additional arguments, for the `DWOId` and to enable `SplitDebugInlining`. `DWOId` should always be zero. - `createTargetMachine`'s wrapper, `LLVMRustCreateTargetMachine` has an additional argument which should be provided the path to the `.dwo` when enabled. Signed-off-by: David Wood <david@davidtw.co>
2020-09-23 10:25:20 -05:00
if (SplitDwarfFile) {
Options.MCOptions.SplitDwarfFile = SplitDwarfFile;
}
Change linker for x86_64-fortanix-unknown-sgx to rust-lld For SGX, the relocation using the relocation table is done by the code in rust/src/libstd/sys/sgx/abi/reloc.rs and this code should not require relocation. Setting RelaxELFRelocations flag if allows this to happen, hence adding a Target Option for it.
2019-12-02 06:22:45 -06:00
Options.RelaxELFRelocations = RelaxELFRelocations;
Use .init_array rather than .ctors LLVM TargetMachines default to using the (now-legacy) .ctors representation of init functions. Mixing .ctors and .init_array representations can cause issues when linking with lld. This happens in practice for: * Our profiling runtime which is currently implicitly built with .init_array since it is built by clang, which sets this field. * External C/C++ code that may be linked into the same process. To support legacy systems which may use .ctors, targets may now specify that they use .ctors via the use_ctors attribute which defaults to false. For debugging and manual control, -Z use-ctors-section=yes/no will allow manual override. Fixes: #71233
2020-04-16 21:40:11 -05:00
Options.UseInitArray = UseInitArray;
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
Control LLVM's TrapUnreachable feature through rustc's TargetOptions.
2017-11-11 09:08:00 -06:00
if (TrapUnreachable) {
Rename trans to codegen everywhere.
2018-05-08 08:10:16 -05:00
// Tell LLVM to codegen `unreachable` into an explicit trap instruction.
Control LLVM's TrapUnreachable feature through rustc's TargetOptions.
2017-11-11 09:08:00 -06:00
// This limits the extent of possible undefined behavior in some cases, as
// it prevents control flow from "falling through" into whatever code
// happens to be laid out next in memory.
Options.TrapUnreachable = true;
}
Enable TrapUnreachable in LLVM. Enable LLVM's TrapUnreachable flag, which tells it to translate `unreachable` instructions into hardware trap instructions, rather than allowing control flow to "fall through" into whatever code happens to follow it in memory.
2017-11-10 13:00:52 -06:00
std: Add a new wasm32-unknown-unknown target This commit adds a new target to the compiler: wasm32-unknown-unknown. This target is a reimagining of what it looks like to generate WebAssembly code from Rust. Instead of using Emscripten which can bring with it a weighty runtime this instead is a target which uses only the LLVM backend for WebAssembly and a "custom linker" for now which will hopefully one day be direct calls to lld. Notable features of this target include: * There is zero runtime footprint. The target assumes nothing exists other than the wasm32 instruction set. * There is zero toolchain footprint beyond adding the target. No custom linker is needed, rustc contains everything. * Very small wasm modules can be generated directly from Rust code using this target. * Most of the standard library is stubbed out to return an error, but anything related to allocation works (aka `HashMap`, `Vec`, etc). * Naturally, any `#[no_std]` crate should be 100% compatible with this new target. This target is currently somewhat janky due to how linking works. The "linking" is currently unconditional whole program LTO (aka LLVM is being used as a linker). Naturally that means compiling programs is pretty slow! Eventually though this target should have a linker. This target is also intended to be quite experimental. I'm hoping that this can act as a catalyst for further experimentation in Rust with WebAssembly. Breaking changes are very likely to land to this target, so it's not recommended to rely on it in any critical capacity yet. We'll let you know when it's "production ready". --- Currently testing-wise this target is looking pretty good but isn't complete. I've got almost the entire `run-pass` test suite working with this target (lots of tests ignored, but many passing as well). The `core` test suite is still getting LLVM bugs fixed to get that working and will take some time. Relatively simple programs all seem to work though! --- It's worth nothing that you may not immediately see the "smallest possible wasm module" for the input you feed to rustc. For various reasons it's very difficult to get rid of the final "bloat" in vanilla rustc (again, a real linker should fix all this). For now what you'll have to do is: cargo install --git https://github.com/alexcrichton/wasm-gc wasm-gc foo.wasm bar.wasm And then `bar.wasm` should be the smallest we can get it! --- In any case for now I'd love feedback on this, particularly on the various integration points if you've got better ideas of how to approach them!
2017-10-22 22:01:00 -05:00
if (Singlethread) {
Options.ThreadModel = ThreadModel::Single;
}
add -Z emit-stack-sizes
2018-09-13 12:43:15 -05:00
Options.EmitStackSizeSection = EmitStackSizeSection;
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
TargetMachine *TM = TheTarget->createTargetMachine(
Fix warnings about the `native` target-cpu This fixes a regression from #53031 where specifying `-C target-cpu=native` is printing a lot of warnings from LLVM about `native` being an unknown CPU. It turns out that `native` is indeed an unknown CPU and we have to perform a mapping to an actual CPU name, but this mapping is only performed in one location rather than all locations we inform LLVM about the target CPU. This commit centralizes the mapping of `native` to LLVM's value of the native CPU, ensuring that all locations we inform LLVM about the `target-cpu` it's never `native`. Closes #53322
2018-08-23 13:03:22 -05:00
Trip.getTriple(), CPU, Feature, Options, RM, CM, OptLevel);
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
return wrap(TM);
Rewrite pass management with LLVM Beforehand, it was unclear whether rust was performing the "recommended set" of optimizations provided by LLVM for code. This commit changes the way we run passes to closely mirror that of clang, which in theory does it correctly. The notable changes include: * Passes are no longer explicitly added one by one. This would be difficult to keep up with as LLVM changes and we don't guaranteed always know the best order in which to run passes * Passes are now managed by LLVM's PassManagerBuilder object. This is then used to populate the various pass managers run. * We now run both a FunctionPassManager and a module-wide PassManager. This is what clang does, and I presume that we *may* see a speed boost from the module-wide passes just having to do less work. I have no measured this. * The codegen pass manager has been extracted to its own separate pass manager to not get mixed up with the other passes * All pass managers now include passes for target-specific data layout and analysis passes Some new features include: * You can now print all passes being run with `-Z print-llvm-passes` * When specifying passes via `--passes`, the passes are now appended to the default list of passes instead of overwriting them. * The output of `--passes list` is now generated by LLVM instead of maintaining a list of passes ourselves * Loop vectorization is turned on by default as an optimization pass and can be disabled with `-Z no-vectorize-loops`
2013-08-22 22:58:42 -05:00
}
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
extern "C" void LLVMRustDisposeTargetMachine(LLVMTargetMachineRef TM) {
delete unwrap(TM);
Rewrite pass management with LLVM Beforehand, it was unclear whether rust was performing the "recommended set" of optimizations provided by LLVM for code. This commit changes the way we run passes to closely mirror that of clang, which in theory does it correctly. The notable changes include: * Passes are no longer explicitly added one by one. This would be difficult to keep up with as LLVM changes and we don't guaranteed always know the best order in which to run passes * Passes are now managed by LLVM's PassManagerBuilder object. This is then used to populate the various pass managers run. * We now run both a FunctionPassManager and a module-wide PassManager. This is what clang does, and I presume that we *may* see a speed boost from the module-wide passes just having to do less work. I have no measured this. * The codegen pass manager has been extracted to its own separate pass manager to not get mixed up with the other passes * All pass managers now include passes for target-specific data layout and analysis passes Some new features include: * You can now print all passes being run with `-Z print-llvm-passes` * When specifying passes via `--passes`, the passes are now appended to the default list of passes instead of overwriting them. * The output of `--passes list` is now generated by LLVM instead of maintaining a list of passes ourselves * Loop vectorization is turned on by default as an optimization pass and can be disabled with `-Z no-vectorize-loops`
2013-08-22 22:58:42 -05:00
}
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
extern "C" void LLVMRustConfigurePassManagerBuilder(
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
LLVMPassManagerBuilderRef PMBR, LLVMRustCodeGenOptLevel OptLevel,
Set PrepareForThinLTO flag when using ThinLTO The LLVM PassManager has a PrepareForThinLTO flag, which is intended when compilation occurs in conjunction with linking by ThinLTO. The flag has two effects: * The NameAnonGlobal pass is run after all other passes, which ensures that all globals have a name. * In optimized builds, a number of late passes (mainly related to vectorization and unrolling) are disabled, on the rationale that these a) will increase codesize of the intermediate artifacts and b) will be run by ThinLTO again anyway. This patch enables the use of PrepareForThinLTO if Thin or ThinLocal linking is used. The background for this change is the CI failure in #49479, which we assume to be caused by the NameAnonGlobal pass not being run. As this changes which passes LLVM runs, this might have performance (or other) impact, so we want to land this separately.
2018-05-12 07:07:20 -05:00
bool MergeFunctions, bool SLPVectorize, bool LoopVectorize, bool PrepareForThinLTO,
Stub out more PassManagerBuilder functions
2022-04-20 02:25:13 -05:00
const char* PGOGenPath, const char* PGOUsePath, const char* PGOSampleUsePath,
int SizeLevel) {
#if LLVM_VERSION_LT(15, 0)
Reenable the MergeFunctions pass The crash that happened in #23566 doesn't happen anymore with the LLVM mergefunc pass enabled and it hugely reduces code size (for example it shaves off 10% of the final Servo executable). This patch reenables it.
2018-03-29 02:41:41 -05:00
unwrap(PMBR)->MergeFunctions = MergeFunctions;
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
unwrap(PMBR)->SLPVectorize = SLPVectorize;
unwrap(PMBR)->OptLevel = fromRust(OptLevel);
unwrap(PMBR)->LoopVectorize = LoopVectorize;
Set PrepareForThinLTO flag when using ThinLTO The LLVM PassManager has a PrepareForThinLTO flag, which is intended when compilation occurs in conjunction with linking by ThinLTO. The flag has two effects: * The NameAnonGlobal pass is run after all other passes, which ensures that all globals have a name. * In optimized builds, a number of late passes (mainly related to vectorization and unrolling) are disabled, on the rationale that these a) will increase codesize of the intermediate artifacts and b) will be run by ThinLTO again anyway. This patch enables the use of PrepareForThinLTO if Thin or ThinLocal linking is used. The background for this change is the CI failure in #49479, which we assume to be caused by the NameAnonGlobal pass not being run. As this changes which passes LLVM runs, this might have performance (or other) impact, so we want to land this separately.
2018-05-12 07:07:20 -05:00
unwrap(PMBR)->PrepareForThinLTO = PrepareForThinLTO;
Stub out more PassManagerBuilder functions
2022-04-20 02:25:13 -05:00
unwrap(PMBR)->SizeLevel = SizeLevel;
unwrap(PMBR)->DisableUnrollLoops = SizeLevel != 0;
try to fix the build on older LLVM versions.
2018-03-15 10:56:45 -05:00
rustc_llvm: rustc_trans: Thread the PGO config down to the pass manager builder. Signed-off-by: Emilio Cobos Álvarez <emilio@crisal.io>
2018-02-18 18:57:12 -06:00
if (PGOGenPath) {
Enable AutoFDO. This largely involves implementing the options debug-info-for-profiling and profile-sample-use and forwarding them on to LLVM. AutoFDO can be used on x86-64 Linux like this: rustc -O -Cdebug-info-for-profiling main.rs -o main perf record -b ./main create_llvm_prof --binary=main --out=code.prof rustc -O -Cprofile-sample-use=code.prof main.rs -o main2 Now `main2` will have feedback directed optimization applied to it. The create_llvm_prof tool can be obtained from this github repository: https://github.com/google/autofdo Fixes #64892.
2021-05-07 02:41:37 -05:00
assert(!PGOUsePath && !PGOSampleUsePath);
rustc_llvm: rustc_trans: Thread the PGO config down to the pass manager builder. Signed-off-by: Emilio Cobos Álvarez <emilio@crisal.io>
2018-02-18 18:57:12 -06:00
unwrap(PMBR)->EnablePGOInstrGen = true;
unwrap(PMBR)->PGOInstrGen = PGOGenPath;
Enable AutoFDO. This largely involves implementing the options debug-info-for-profiling and profile-sample-use and forwarding them on to LLVM. AutoFDO can be used on x86-64 Linux like this: rustc -O -Cdebug-info-for-profiling main.rs -o main perf record -b ./main create_llvm_prof --binary=main --out=code.prof rustc -O -Cprofile-sample-use=code.prof main.rs -o main2 Now `main2` will have feedback directed optimization applied to it. The create_llvm_prof tool can be obtained from this github repository: https://github.com/google/autofdo Fixes #64892.
2021-05-07 02:41:37 -05:00
} else if (PGOUsePath) {
assert(!PGOSampleUsePath);
rustc_llvm: rustc_trans: Thread the PGO config down to the pass manager builder. Signed-off-by: Emilio Cobos Álvarez <emilio@crisal.io>
2018-02-18 18:57:12 -06:00
unwrap(PMBR)->PGOInstrUse = PGOUsePath;
Enable AutoFDO. This largely involves implementing the options debug-info-for-profiling and profile-sample-use and forwarding them on to LLVM. AutoFDO can be used on x86-64 Linux like this: rustc -O -Cdebug-info-for-profiling main.rs -o main perf record -b ./main create_llvm_prof --binary=main --out=code.prof rustc -O -Cprofile-sample-use=code.prof main.rs -o main2 Now `main2` will have feedback directed optimization applied to it. The create_llvm_prof tool can be obtained from this github repository: https://github.com/google/autofdo Fixes #64892.
2021-05-07 02:41:37 -05:00
} else if (PGOSampleUsePath) {
unwrap(PMBR)->PGOSampleUse = PGOSampleUsePath;
rustc_llvm: rustc_trans: Thread the PGO config down to the pass manager builder. Signed-off-by: Emilio Cobos Álvarez <emilio@crisal.io>
2018-02-18 18:57:12 -06:00
}
Stub out more PassManagerBuilder functions
2022-04-20 02:25:13 -05:00
#else
report_fatal_error("Legacy PM not supported with LLVM 15");
#endif
Rewrite pass management with LLVM Beforehand, it was unclear whether rust was performing the "recommended set" of optimizations provided by LLVM for code. This commit changes the way we run passes to closely mirror that of clang, which in theory does it correctly. The notable changes include: * Passes are no longer explicitly added one by one. This would be difficult to keep up with as LLVM changes and we don't guaranteed always know the best order in which to run passes * Passes are now managed by LLVM's PassManagerBuilder object. This is then used to populate the various pass managers run. * We now run both a FunctionPassManager and a module-wide PassManager. This is what clang does, and I presume that we *may* see a speed boost from the module-wide passes just having to do less work. I have no measured this. * The codegen pass manager has been extracted to its own separate pass manager to not get mixed up with the other passes * All pass managers now include passes for target-specific data layout and analysis passes Some new features include: * You can now print all passes being run with `-Z print-llvm-passes` * When specifying passes via `--passes`, the passes are now appended to the default list of passes instead of overwriting them. * The output of `--passes list` is now generated by LLVM instead of maintaining a list of passes ourselves * Loop vectorization is turned on by default as an optimization pass and can be disabled with `-Z no-vectorize-loops`
2013-08-22 22:58:42 -05:00
}
// Unfortunately, the LLVM C API doesn't provide a way to set the `LibraryInfo`
// field of a PassManagerBuilder, we expose our own method of doing so.
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
extern "C" void LLVMRustAddBuilderLibraryInfo(LLVMPassManagerBuilderRef PMBR,
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
LLVMModuleRef M,
bool DisableSimplifyLibCalls) {
Triple TargetTriple(unwrap(M)->getTargetTriple());
TargetLibraryInfoImpl *TLI = new TargetLibraryInfoImpl(TargetTriple);
if (DisableSimplifyLibCalls)
TLI->disableAllFunctions();
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
unwrap(PMBR)->LibraryInfo = TLI;
Rewrite pass management with LLVM Beforehand, it was unclear whether rust was performing the "recommended set" of optimizations provided by LLVM for code. This commit changes the way we run passes to closely mirror that of clang, which in theory does it correctly. The notable changes include: * Passes are no longer explicitly added one by one. This would be difficult to keep up with as LLVM changes and we don't guaranteed always know the best order in which to run passes * Passes are now managed by LLVM's PassManagerBuilder object. This is then used to populate the various pass managers run. * We now run both a FunctionPassManager and a module-wide PassManager. This is what clang does, and I presume that we *may* see a speed boost from the module-wide passes just having to do less work. I have no measured this. * The codegen pass manager has been extracted to its own separate pass manager to not get mixed up with the other passes * All pass managers now include passes for target-specific data layout and analysis passes Some new features include: * You can now print all passes being run with `-Z print-llvm-passes` * When specifying passes via `--passes`, the passes are now appended to the default list of passes instead of overwriting them. * The output of `--passes list` is now generated by LLVM instead of maintaining a list of passes ourselves * Loop vectorization is turned on by default as an optimization pass and can be disabled with `-Z no-vectorize-loops`
2013-08-22 22:58:42 -05:00
}
// Unfortunately, the LLVM C API doesn't provide a way to create the
// TargetLibraryInfo pass, so we use this method to do so.
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
extern "C" void LLVMRustAddLibraryInfo(LLVMPassManagerRef PMR, LLVMModuleRef M,
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
bool DisableSimplifyLibCalls) {
Triple TargetTriple(unwrap(M)->getTargetTriple());
TargetLibraryInfoImpl TLII(TargetTriple);
if (DisableSimplifyLibCalls)
TLII.disableAllFunctions();
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
unwrap(PMR)->add(new TargetLibraryInfoWrapperPass(TLII));
Rewrite pass management with LLVM Beforehand, it was unclear whether rust was performing the "recommended set" of optimizations provided by LLVM for code. This commit changes the way we run passes to closely mirror that of clang, which in theory does it correctly. The notable changes include: * Passes are no longer explicitly added one by one. This would be difficult to keep up with as LLVM changes and we don't guaranteed always know the best order in which to run passes * Passes are now managed by LLVM's PassManagerBuilder object. This is then used to populate the various pass managers run. * We now run both a FunctionPassManager and a module-wide PassManager. This is what clang does, and I presume that we *may* see a speed boost from the module-wide passes just having to do less work. I have no measured this. * The codegen pass manager has been extracted to its own separate pass manager to not get mixed up with the other passes * All pass managers now include passes for target-specific data layout and analysis passes Some new features include: * You can now print all passes being run with `-Z print-llvm-passes` * When specifying passes via `--passes`, the passes are now appended to the default list of passes instead of overwriting them. * The output of `--passes list` is now generated by LLVM instead of maintaining a list of passes ourselves * Loop vectorization is turned on by default as an optimization pass and can be disabled with `-Z no-vectorize-loops`
2013-08-22 22:58:42 -05:00
}
// Unfortunately, the LLVM C API doesn't provide an easy way of iterating over
// all the functions in a module, so we do that manually here. You'll find
// similar code in clang's BackendUtil.cpp file.
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
extern "C" void LLVMRustRunFunctionPassManager(LLVMPassManagerRef PMR,
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
LLVMModuleRef M) {
llvm::legacy::FunctionPassManager *P =
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
unwrap<llvm::legacy::FunctionPassManager>(PMR);
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
P->doInitialization();
// Upgrade all calls to old intrinsics first.
for (Module::iterator I = unwrap(M)->begin(), E = unwrap(M)->end(); I != E;)
UpgradeCallsToIntrinsic(&*I++); // must be post-increment, as we remove
for (Module::iterator I = unwrap(M)->begin(), E = unwrap(M)->end(); I != E;
++I)
if (!I->isDeclaration())
P->run(*I);
P->doFinalization();
Rewrite pass management with LLVM Beforehand, it was unclear whether rust was performing the "recommended set" of optimizations provided by LLVM for code. This commit changes the way we run passes to closely mirror that of clang, which in theory does it correctly. The notable changes include: * Passes are no longer explicitly added one by one. This would be difficult to keep up with as LLVM changes and we don't guaranteed always know the best order in which to run passes * Passes are now managed by LLVM's PassManagerBuilder object. This is then used to populate the various pass managers run. * We now run both a FunctionPassManager and a module-wide PassManager. This is what clang does, and I presume that we *may* see a speed boost from the module-wide passes just having to do less work. I have no measured this. * The codegen pass manager has been extracted to its own separate pass manager to not get mixed up with the other passes * All pass managers now include passes for target-specific data layout and analysis passes Some new features include: * You can now print all passes being run with `-Z print-llvm-passes` * When specifying passes via `--passes`, the passes are now appended to the default list of passes instead of overwriting them. * The output of `--passes list` is now generated by LLVM instead of maintaining a list of passes ourselves * Loop vectorization is turned on by default as an optimization pass and can be disabled with `-Z no-vectorize-loops`
2013-08-22 22:58:42 -05:00
}
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
extern "C" void LLVMRustSetLLVMOptions(int Argc, char **Argv) {
// Initializing the command-line options more than once is not allowed. So,
// check if they've already been initialized. (This could happen if we're
// being called from rustpkg, for example). If the arguments change, then
// that's just kinda unfortunate.
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
static bool Initialized = false;
if (Initialized)
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
return;
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
Initialized = true;
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
cl::ParseCommandLineOptions(Argc, Argv);
Rewrite pass management with LLVM Beforehand, it was unclear whether rust was performing the "recommended set" of optimizations provided by LLVM for code. This commit changes the way we run passes to closely mirror that of clang, which in theory does it correctly. The notable changes include: * Passes are no longer explicitly added one by one. This would be difficult to keep up with as LLVM changes and we don't guaranteed always know the best order in which to run passes * Passes are now managed by LLVM's PassManagerBuilder object. This is then used to populate the various pass managers run. * We now run both a FunctionPassManager and a module-wide PassManager. This is what clang does, and I presume that we *may* see a speed boost from the module-wide passes just having to do less work. I have no measured this. * The codegen pass manager has been extracted to its own separate pass manager to not get mixed up with the other passes * All pass managers now include passes for target-specific data layout and analysis passes Some new features include: * You can now print all passes being run with `-Z print-llvm-passes` * When specifying passes via `--passes`, the passes are now appended to the default list of passes instead of overwriting them. * The output of `--passes list` is now generated by LLVM instead of maintaining a list of passes ourselves * Loop vectorization is turned on by default as an optimization pass and can be disabled with `-Z no-vectorize-loops`
2013-08-22 22:58:42 -05:00
}
audit LLVM C++ types in ArchiveWrapper and PassWrapper
2016-08-01 16:16:16 -05:00
enum class LLVMRustFileType {
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
AssemblyFile,
ObjectFile,
audit LLVM C++ types in ArchiveWrapper and PassWrapper
2016-08-01 16:16:16 -05:00
};
CodeGenFileType moved outside TargetMachine
2019-12-31 07:05:34 -06:00
static CodeGenFileType fromRust(LLVMRustFileType Type) {
switch (Type) {
case LLVMRustFileType::AssemblyFile:
return CGFT_AssemblyFile;
case LLVMRustFileType::ObjectFile:
return CGFT_ObjectFile;
default:
report_fatal_error("Bad FileType.");
}
}
audit LLVM C++ types in ArchiveWrapper and PassWrapper
2016-08-01 16:16:16 -05:00
extern "C" LLVMRustResult
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
LLVMRustWriteOutputFile(LLVMTargetMachineRef Target, LLVMPassManagerRef PMR,
llvm: update ffi bindings for split dwarf This commit modifies the FFI bindings to LLVM required for Split DWARF support in rustc. In particular: - `addPassesToEmitFile`'s wrapper, `LLVMRustWriteOutputFile` now takes a `DwoPath` `const char*`. When disabled, `nullptr` should be provided which will preserve existing behaviour. When enabled, the path to the `.dwo` file should be provided. - `createCompileUnit`'s wrapper, `LLVMRustDIBuilderCreateCompileUnit` now has two additional arguments, for the `DWOId` and to enable `SplitDebugInlining`. `DWOId` should always be zero. - `createTargetMachine`'s wrapper, `LLVMRustCreateTargetMachine` has an additional argument which should be provided the path to the `.dwo` when enabled. Signed-off-by: David Wood <david@davidtw.co>
2020-09-23 10:25:20 -05:00
LLVMModuleRef M, const char *Path, const char *DwoPath,
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
LLVMRustFileType RustFileType) {
[LLVM-3.9] Specify that we are using the legacy interface LLVM pass manager infrastructure is currently getting rewritten to be more flexible, but the rewrite isn't complete yet.
2016-07-12 17:42:20 -05:00
llvm::legacy::PassManager *PM = unwrap<llvm::legacy::PassManager>(PMR);
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
auto FileType = fromRust(RustFileType);
Rewrite pass management with LLVM Beforehand, it was unclear whether rust was performing the "recommended set" of optimizations provided by LLVM for code. This commit changes the way we run passes to closely mirror that of clang, which in theory does it correctly. The notable changes include: * Passes are no longer explicitly added one by one. This would be difficult to keep up with as LLVM changes and we don't guaranteed always know the best order in which to run passes * Passes are now managed by LLVM's PassManagerBuilder object. This is then used to populate the various pass managers run. * We now run both a FunctionPassManager and a module-wide PassManager. This is what clang does, and I presume that we *may* see a speed boost from the module-wide passes just having to do less work. I have no measured this. * The codegen pass manager has been extracted to its own separate pass manager to not get mixed up with the other passes * All pass managers now include passes for target-specific data layout and analysis passes Some new features include: * You can now print all passes being run with `-Z print-llvm-passes` * When specifying passes via `--passes`, the passes are now appended to the default list of passes instead of overwriting them. * The output of `--passes list` is now generated by LLVM instead of maintaining a list of passes ourselves * Loop vectorization is turned on by default as an optimization pass and can be disabled with `-Z no-vectorize-loops`
2013-08-22 22:58:42 -05:00
std::string ErrorInfo;
Update LLVM.
2014-09-30 16:20:22 -05:00
std::error_code EC;
Replace llvm::sys::fs::F_None with llvm::sys::fs::OF_None The former is deprecated. OF_None has been available in LLVM since 2018-06.
2021-04-29 17:25:17 -05:00
raw_fd_ostream OS(Path, EC, sys::fs::OF_None);
Update LLVM.
2014-09-30 16:20:22 -05:00
if (EC)
ErrorInfo = EC.message();
Rewrite pass management with LLVM Beforehand, it was unclear whether rust was performing the "recommended set" of optimizations provided by LLVM for code. This commit changes the way we run passes to closely mirror that of clang, which in theory does it correctly. The notable changes include: * Passes are no longer explicitly added one by one. This would be difficult to keep up with as LLVM changes and we don't guaranteed always know the best order in which to run passes * Passes are now managed by LLVM's PassManagerBuilder object. This is then used to populate the various pass managers run. * We now run both a FunctionPassManager and a module-wide PassManager. This is what clang does, and I presume that we *may* see a speed boost from the module-wide passes just having to do less work. I have no measured this. * The codegen pass manager has been extracted to its own separate pass manager to not get mixed up with the other passes * All pass managers now include passes for target-specific data layout and analysis passes Some new features include: * You can now print all passes being run with `-Z print-llvm-passes` * When specifying passes via `--passes`, the passes are now appended to the default list of passes instead of overwriting them. * The output of `--passes list` is now generated by LLVM instead of maintaining a list of passes ourselves * Loop vectorization is turned on by default as an optimization pass and can be disabled with `-Z no-vectorize-loops`
2013-08-22 22:58:42 -05:00
if (ErrorInfo != "") {
rustc: Fix passing errors from LLVM to rustc Many of the instances of setting a global error variable ended up leaving a dangling pointer into free'd memory. This changes the method of error transmission to strdup any error and "relinquish ownership" to rustc when it gets an error. The corresponding Rust code will then free the error as necessary. Closes #12865
2014-04-15 09:25:22 -05:00
LLVMRustSetLastError(ErrorInfo.c_str());
audit LLVM C++ types in ArchiveWrapper and PassWrapper
2016-08-01 16:16:16 -05:00
return LLVMRustResult::Failure;
Rewrite pass management with LLVM Beforehand, it was unclear whether rust was performing the "recommended set" of optimizations provided by LLVM for code. This commit changes the way we run passes to closely mirror that of clang, which in theory does it correctly. The notable changes include: * Passes are no longer explicitly added one by one. This would be difficult to keep up with as LLVM changes and we don't guaranteed always know the best order in which to run passes * Passes are now managed by LLVM's PassManagerBuilder object. This is then used to populate the various pass managers run. * We now run both a FunctionPassManager and a module-wide PassManager. This is what clang does, and I presume that we *may* see a speed boost from the module-wide passes just having to do less work. I have no measured this. * The codegen pass manager has been extracted to its own separate pass manager to not get mixed up with the other passes * All pass managers now include passes for target-specific data layout and analysis passes Some new features include: * You can now print all passes being run with `-Z print-llvm-passes` * When specifying passes via `--passes`, the passes are now appended to the default list of passes instead of overwriting them. * The output of `--passes list` is now generated by LLVM instead of maintaining a list of passes ourselves * Loop vectorization is turned on by default as an optimization pass and can be disabled with `-Z no-vectorize-loops`
2013-08-22 22:58:42 -05:00
}
Buffer LLVM's object output stream In some profiling on OSX I saw the `write` syscall as quite high up on the profiling graph, which is definitely not good! It looks like we're setting the output stream of an object file as directly to a file descriptor which means that we run the risk of doing lots of little writes rather than a few large writes. This commit fixes this issue by adding a buffered stream on the output, causing the `write` syscall to disappear from the profiles on OSX.
2018-08-20 01:44:25 -05:00
buffer_ostream BOS(OS);
llvm: update ffi bindings for split dwarf This commit modifies the FFI bindings to LLVM required for Split DWARF support in rustc. In particular: - `addPassesToEmitFile`'s wrapper, `LLVMRustWriteOutputFile` now takes a `DwoPath` `const char*`. When disabled, `nullptr` should be provided which will preserve existing behaviour. When enabled, the path to the `.dwo` file should be provided. - `createCompileUnit`'s wrapper, `LLVMRustDIBuilderCreateCompileUnit` now has two additional arguments, for the `DWOId` and to enable `SplitDebugInlining`. `DWOId` should always be zero. - `createTargetMachine`'s wrapper, `LLVMRustCreateTargetMachine` has an additional argument which should be provided the path to the `.dwo` when enabled. Signed-off-by: David Wood <david@davidtw.co>
2020-09-23 10:25:20 -05:00
if (DwoPath) {
Replace llvm::sys::fs::F_None with llvm::sys::fs::OF_None The former is deprecated. OF_None has been available in LLVM since 2018-06.
2021-04-29 17:25:17 -05:00
raw_fd_ostream DOS(DwoPath, EC, sys::fs::OF_None);
llvm: update ffi bindings for split dwarf This commit modifies the FFI bindings to LLVM required for Split DWARF support in rustc. In particular: - `addPassesToEmitFile`'s wrapper, `LLVMRustWriteOutputFile` now takes a `DwoPath` `const char*`. When disabled, `nullptr` should be provided which will preserve existing behaviour. When enabled, the path to the `.dwo` file should be provided. - `createCompileUnit`'s wrapper, `LLVMRustDIBuilderCreateCompileUnit` now has two additional arguments, for the `DWOId` and to enable `SplitDebugInlining`. `DWOId` should always be zero. - `createTargetMachine`'s wrapper, `LLVMRustCreateTargetMachine` has an additional argument which should be provided the path to the `.dwo` when enabled. Signed-off-by: David Wood <david@davidtw.co>
2020-09-23 10:25:20 -05:00
EC.clear();
if (EC)
ErrorInfo = EC.message();
if (ErrorInfo != "") {
LLVMRustSetLastError(ErrorInfo.c_str());
return LLVMRustResult::Failure;
}
buffer_ostream DBOS(DOS);
unwrap(Target)->addPassesToEmitFile(*PM, BOS, &DBOS, FileType, false);
PM->run(*unwrap(M));
} else {
unwrap(Target)->addPassesToEmitFile(*PM, BOS, nullptr, FileType, false);
PM->run(*unwrap(M));
}
rustc: Update LLVM This commit updates the LLVM submodule in use to the current HEAD of the LLVM repository. This is primarily being done to start picking up unwinding support for MSVC, which is currently unimplemented in the revision of LLVM we are using. Along the way a few changes had to be made: * As usual, lots of C++ debuginfo bindings in LLVM changed, so there were some significant changes to our RustWrapper.cpp * As usual, some pass management changed in LLVM, so clang was re-scrutinized to ensure that we're doing the same thing as clang. * Some optimization options are now passed directly into the `PassManagerBuilder` instead of through CLI switches to LLVM. * The `NoFramePointerElim` option was removed from LLVM, favoring instead the `no-frame-pointer-elim` function attribute instead. Additionally, LLVM has picked up some new optimizations which required fixing an existing soundness hole in the IR we generate. It appears that the current LLVM we use does not expose this hole. When an enum is moved, the previous slot in memory is overwritten with a bit pattern corresponding to "dropped". When the drop glue for this slot is run, however, the switch on the discriminant can often start executing the `unreachable` block of the switch due to the discriminant now being outside the normal range. This was patched over locally for now by having the `unreachable` block just change to a `ret void`.
2015-05-14 14:10:43 -05:00
typos: fix a grabbag of typos all over the place
2015-10-07 17:11:25 -05:00
// Apparently `addPassesToEmitFile` adds a pointer to our on-the-stack output
rustc: Update LLVM This commit updates the LLVM submodule in use to the current HEAD of the LLVM repository. This is primarily being done to start picking up unwinding support for MSVC, which is currently unimplemented in the revision of LLVM we are using. Along the way a few changes had to be made: * As usual, lots of C++ debuginfo bindings in LLVM changed, so there were some significant changes to our RustWrapper.cpp * As usual, some pass management changed in LLVM, so clang was re-scrutinized to ensure that we're doing the same thing as clang. * Some optimization options are now passed directly into the `PassManagerBuilder` instead of through CLI switches to LLVM. * The `NoFramePointerElim` option was removed from LLVM, favoring instead the `no-frame-pointer-elim` function attribute instead. Additionally, LLVM has picked up some new optimizations which required fixing an existing soundness hole in the IR we generate. It appears that the current LLVM we use does not expose this hole. When an enum is moved, the previous slot in memory is overwritten with a bit pattern corresponding to "dropped". When the drop glue for this slot is run, however, the switch on the discriminant can often start executing the `unreachable` block of the switch due to the discriminant now being outside the normal range. This was patched over locally for now by having the `unreachable` block just change to a `ret void`.
2015-05-14 14:10:43 -05:00
// stream (OS), so the only real safe place to delete this is here? Don't we
// wish this was written in Rust?
Use LLVMDisposePassManager instead of raw delete in rustllvm
2019-11-25 14:44:26 -06:00
LLVMDisposePassManager(PMR);
audit LLVM C++ types in ArchiveWrapper and PassWrapper
2016-08-01 16:16:16 -05:00
return LLVMRustResult::Success;
Rewrite pass management with LLVM Beforehand, it was unclear whether rust was performing the "recommended set" of optimizations provided by LLVM for code. This commit changes the way we run passes to closely mirror that of clang, which in theory does it correctly. The notable changes include: * Passes are no longer explicitly added one by one. This would be difficult to keep up with as LLVM changes and we don't guaranteed always know the best order in which to run passes * Passes are now managed by LLVM's PassManagerBuilder object. This is then used to populate the various pass managers run. * We now run both a FunctionPassManager and a module-wide PassManager. This is what clang does, and I presume that we *may* see a speed boost from the module-wide passes just having to do less work. I have no measured this. * The codegen pass manager has been extracted to its own separate pass manager to not get mixed up with the other passes * All pass managers now include passes for target-specific data layout and analysis passes Some new features include: * You can now print all passes being run with `-Z print-llvm-passes` * When specifying passes via `--passes`, the passes are now appended to the default list of passes instead of overwriting them. * The output of `--passes list` is now generated by LLVM instead of maintaining a list of passes ourselves * Loop vectorization is turned on by default as an optimization pass and can be disabled with `-Z no-vectorize-loops`
2013-08-22 22:58:42 -05:00
}
add selfprofiling for new llvm passmanager
2020-02-11 15:37:16 -06:00
extern "C" typedef void (*LLVMRustSelfProfileBeforePassCallback)(void*, // LlvmSelfProfiler
const char*, // pass name
const char*); // IR name
extern "C" typedef void (*LLVMRustSelfProfileAfterPassCallback)(void*); // LlvmSelfProfiler
std::string LLVMRustwrappedIrGetName(const llvm::Any &WrappedIr) {
if (any_isa<const Module *>(WrappedIr))
return any_cast<const Module *>(WrappedIr)->getName().str();
if (any_isa<const Function *>(WrappedIr))
return any_cast<const Function *>(WrappedIr)->getName().str();
if (any_isa<const Loop *>(WrappedIr))
return any_cast<const Loop *>(WrappedIr)->getName().str();
if (any_isa<const LazyCallGraph::SCC *>(WrappedIr))
return any_cast<const LazyCallGraph::SCC *>(WrappedIr)->getName();
return "<UNKNOWN>";
}
void LLVMSelfProfileInitializeCallbacks(
PassInstrumentationCallbacks& PIC, void* LlvmSelfProfiler,
LLVMRustSelfProfileBeforePassCallback BeforePassCallback,
LLVMRustSelfProfileAfterPassCallback AfterPassCallback) {
Support LLVM 12 in rustc
2020-11-03 15:47:16 -06:00
PIC.registerBeforeNonSkippedPassCallback([LlvmSelfProfiler, BeforePassCallback](
StringRef Pass, llvm::Any Ir) {
std::string PassName = Pass.str();
std::string IrName = LLVMRustwrappedIrGetName(Ir);
BeforePassCallback(LlvmSelfProfiler, PassName.c_str(), IrName.c_str());
});
PIC.registerAfterPassCallback(
[LlvmSelfProfiler, AfterPassCallback](StringRef Pass, llvm::Any IR,
const PreservedAnalyses &Preserved) {
AfterPassCallback(LlvmSelfProfiler);
});
PIC.registerAfterPassInvalidatedCallback(
[LlvmSelfProfiler, AfterPassCallback](StringRef Pass, const PreservedAnalyses &Preserved) {
AfterPassCallback(LlvmSelfProfiler);
});
add selfprofiling for new llvm passmanager
2020-02-11 15:37:16 -06:00
PIC.registerBeforeAnalysisCallback([LlvmSelfProfiler, BeforePassCallback](
StringRef Pass, llvm::Any Ir) {
std::string PassName = Pass.str();
std::string IrName = LLVMRustwrappedIrGetName(Ir);
BeforePassCallback(LlvmSelfProfiler, PassName.c_str(), IrName.c_str());
});
PIC.registerAfterAnalysisCallback(
[LlvmSelfProfiler, AfterPassCallback](StringRef Pass, llvm::Any Ir) {
AfterPassCallback(LlvmSelfProfiler);
});
}
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
enum class LLVMRustOptStage {
PreLinkNoLTO,
PreLinkThinLTO,
PreLinkFatLTO,
ThinLTO,
FatLTO,
};
struct LLVMRustSanitizerOptions {
Diagnose use of incompatible sanitizers Emit an error when incompatible sanitizer are configured through command line options. Previously the last one configured prevailed and others were silently ignored. Additionally use a set to represent configured sanitizers, making it possible to enable multiple sanitizers at once. At least in principle, since currently all of them are considered to be incompatible with others.
2020-06-13 19:00:00 -05:00
bool SanitizeAddress;
bool SanitizeAddressRecover;
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
bool SanitizeMemory;
Diagnose use of incompatible sanitizers Emit an error when incompatible sanitizer are configured through command line options. Previously the last one configured prevailed and others were silently ignored. Additionally use a set to represent configured sanitizers, making it possible to enable multiple sanitizers at once. At least in principle, since currently all of them are considered to be incompatible with others.
2020-06-13 19:00:00 -05:00
bool SanitizeMemoryRecover;
int SanitizeMemoryTrackOrigins;
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
bool SanitizeThread;
HWASan support
2021-01-22 20:32:38 -06:00
bool SanitizeHWAddress;
bool SanitizeHWAddressRecover;
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
};
Support -C passes in NewPM And report an error if parsing the additional pass pipeline fails. Threading through the error accounts for most of the changes here.
2021-04-05 08:37:11 -05:00
extern "C" LLVMRustResult
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
LLVMRustOptimizeWithNewPassManager(
LLVMModuleRef ModuleRef,
LLVMTargetMachineRef TMRef,
LLVMRustPassBuilderOptLevel OptLevelRust,
LLVMRustOptStage OptStage,
bool NoPrepopulatePasses, bool VerifyIR, bool UseThinLTOBuffers,
bool MergeFunctions, bool UnrollLoops, bool SLPVectorize, bool LoopVectorize,
Consistently use LLVM lifetime markers during codegen Ensure that inliner inserts lifetime markers if they have been emitted during codegen. Otherwise if allocas from inlined functions are merged together, lifetime markers from one function might invalidate load & stores performed by the other one.
2020-05-12 19:00:00 -05:00
bool DisableSimplifyLibCalls, bool EmitLifetimeMarkers,
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
LLVMRustSanitizerOptions *SanitizerOptions,
Explicitly register GCOV profiling pass as well
2021-04-05 04:26:48 -05:00
const char *PGOGenPath, const char *PGOUsePath,
bool InstrumentCoverage, bool InstrumentGCOV,
Enable AutoFDO. This largely involves implementing the options debug-info-for-profiling and profile-sample-use and forwarding them on to LLVM. AutoFDO can be used on x86-64 Linux like this: rustc -O -Cdebug-info-for-profiling main.rs -o main perf record -b ./main create_llvm_prof --binary=main --out=code.prof rustc -O -Cprofile-sample-use=code.prof main.rs -o main2 Now `main2` will have feedback directed optimization applied to it. The create_llvm_prof tool can be obtained from this github repository: https://github.com/google/autofdo Fixes #64892.
2021-05-07 02:41:37 -05:00
const char *PGOSampleUsePath, bool DebugInfoForProfiling,
add selfprofiling for new llvm passmanager
2020-02-11 15:37:16 -06:00
void* LlvmSelfProfiler,
LLVMRustSelfProfileBeforePassCallback BeforePassCallback,
Support -C passes in NewPM And report an error if parsing the additional pass pipeline fails. Threading through the error accounts for most of the changes here.
2021-04-05 08:37:11 -05:00
LLVMRustSelfProfileAfterPassCallback AfterPassCallback,
Add a codegen option to allow loading LLVM pass plugins
2021-11-19 10:01:41 -06:00
const char *ExtraPasses, size_t ExtraPassesLen,
Use the existing llvm-plugins option for both legacy and new pm registration
2021-11-24 04:43:40 -06:00
const char *LLVMPlugins, size_t LLVMPluginsLen) {
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
Module *TheModule = unwrap(ModuleRef);
TargetMachine *TM = unwrap(TMRef);
PassWrapper: handle move of OptimizationLevel class out of PassBuilder This is the first build break of the LLVM 14 cycle, and was caused by https://reviews.llvm.org/D107025. Mercifully an easy fix.
2021-08-05 11:13:58 -05:00
OptimizationLevel OptLevel = fromRust(OptLevelRust);
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
PipelineTuningOptions PTO;
PTO.LoopUnrolling = UnrollLoops;
PTO.LoopInterleaving = UnrollLoops;
PTO.LoopVectorization = LoopVectorize;
PTO.SLPVectorization = SLPVectorize;
Support merge_functions option in NewPM since LLVM >= 12 now we can pass this flag since https://reviews.llvm.org/D93002 has been merged.
2021-03-11 15:30:49 -06:00
PTO.MergeFunctions = MergeFunctions;
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
Support LLVM 12 in rustc
2020-11-03 15:47:16 -06:00
// FIXME: We may want to expose this as an option.
bool DebugPassManager = false;
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
PassInstrumentationCallbacks PIC;
Support LLVM 12 in rustc
2020-11-03 15:47:16 -06:00
StandardInstrumentations SI(DebugPassManager);
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
SI.registerCallbacks(PIC);
add selfprofiling for new llvm passmanager
2020-02-11 15:37:16 -06:00
if (LlvmSelfProfiler){
LLVMSelfProfileInitializeCallbacks(PIC,LlvmSelfProfiler,BeforePassCallback,AfterPassCallback);
}
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
Optional<PGOOptions> PGOOpt;
if (PGOGenPath) {
Enable AutoFDO. This largely involves implementing the options debug-info-for-profiling and profile-sample-use and forwarding them on to LLVM. AutoFDO can be used on x86-64 Linux like this: rustc -O -Cdebug-info-for-profiling main.rs -o main perf record -b ./main create_llvm_prof --binary=main --out=code.prof rustc -O -Cprofile-sample-use=code.prof main.rs -o main2 Now `main2` will have feedback directed optimization applied to it. The create_llvm_prof tool can be obtained from this github repository: https://github.com/google/autofdo Fixes #64892.
2021-05-07 02:41:37 -05:00
assert(!PGOUsePath && !PGOSampleUsePath);
PGOOpt = PGOOptions(PGOGenPath, "", "", PGOOptions::IRInstr,
PGOOptions::NoCSAction, DebugInfoForProfiling);
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
} else if (PGOUsePath) {
Enable AutoFDO. This largely involves implementing the options debug-info-for-profiling and profile-sample-use and forwarding them on to LLVM. AutoFDO can be used on x86-64 Linux like this: rustc -O -Cdebug-info-for-profiling main.rs -o main perf record -b ./main create_llvm_prof --binary=main --out=code.prof rustc -O -Cprofile-sample-use=code.prof main.rs -o main2 Now `main2` will have feedback directed optimization applied to it. The create_llvm_prof tool can be obtained from this github repository: https://github.com/google/autofdo Fixes #64892.
2021-05-07 02:41:37 -05:00
assert(!PGOSampleUsePath);
PGOOpt = PGOOptions(PGOUsePath, "", "", PGOOptions::IRUse,
PGOOptions::NoCSAction, DebugInfoForProfiling);
} else if (PGOSampleUsePath) {
PGOOpt = PGOOptions(PGOSampleUsePath, "", "", PGOOptions::SampleUse,
PGOOptions::NoCSAction, DebugInfoForProfiling);
} else if (DebugInfoForProfiling) {
PGOOpt = PGOOptions("", "", "", PGOOptions::NoAction,
PGOOptions::NoCSAction, DebugInfoForProfiling);
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
}
PassWrapper: update for LLVM change D102093 In https://reviews.llvm.org/D102093 lots of things stopped taking the DebugLogging boolean parameter. Mercifully we appear to always set DebugPassManager to false, so I don't think we're losing anything by not passing this parameter.
2021-05-17 13:12:38 -05:00
#if LLVM_VERSION_GE(13, 0)
Update the minimum external LLVM to 12
2021-10-22 12:50:07 -05:00
PassBuilder PB(TM, PTO, PGOOpt, &PIC);
PassWrapper: update for LLVM change D102093 In https://reviews.llvm.org/D102093 lots of things stopped taking the DebugLogging boolean parameter. Mercifully we appear to always set DebugPassManager to false, so I don't think we're losing anything by not passing this parameter.
2021-05-17 13:12:38 -05:00
LoopAnalysisManager LAM;
FunctionAnalysisManager FAM;
CGSCCAnalysisManager CGAM;
ModuleAnalysisManager MAM;
#else
Update the minimum external LLVM to 12
2021-10-22 12:50:07 -05:00
PassBuilder PB(DebugPassManager, TM, PTO, PGOOpt, &PIC);
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
LoopAnalysisManager LAM(DebugPassManager);
FunctionAnalysisManager FAM(DebugPassManager);
CGSCCAnalysisManager CGAM(DebugPassManager);
ModuleAnalysisManager MAM(DebugPassManager);
PassWrapper: update for LLVM change D102093 In https://reviews.llvm.org/D102093 lots of things stopped taking the DebugLogging boolean parameter. Mercifully we appear to always set DebugPassManager to false, so I don't think we're losing anything by not passing this parameter.
2021-05-17 13:12:38 -05:00
#endif
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
FAM.registerPass([&] { return PB.buildDefaultAAPipeline(); });
Triple TargetTriple(TheModule->getTargetTriple());
std::unique_ptr<TargetLibraryInfoImpl> TLII(new TargetLibraryInfoImpl(TargetTriple));
if (DisableSimplifyLibCalls)
TLII->disableAllFunctions();
FAM.registerPass([&] { return TargetLibraryAnalysis(*TLII); });
PB.registerModuleAnalyses(MAM);
PB.registerCGSCCAnalyses(CGAM);
PB.registerFunctionAnalyses(FAM);
PB.registerLoopAnalyses(LAM);
PB.crossRegisterProxies(LAM, FAM, CGAM, MAM);
// We manually collect pipeline callbacks so we can apply them at O0, where the
// PassBuilder does not create a pipeline.
PassWrapper: handle move of OptimizationLevel class out of PassBuilder This is the first build break of the LLVM 14 cycle, and was caused by https://reviews.llvm.org/D107025. Mercifully an easy fix.
2021-08-05 11:13:58 -05:00
std::vector<std::function<void(ModulePassManager &, OptimizationLevel)>>
Support LLVM 12 in rustc
2020-11-03 15:47:16 -06:00
PipelineStartEPCallbacks;
PassWrapper: handle move of OptimizationLevel class out of PassBuilder This is the first build break of the LLVM 14 cycle, and was caused by https://reviews.llvm.org/D107025. Mercifully an easy fix.
2021-08-05 11:13:58 -05:00
std::vector<std::function<void(ModulePassManager &, OptimizationLevel)>>
Prepare for LLVM 11
2020-06-25 20:52:41 -05:00
OptimizerLastEPCallbacks;
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
if (VerifyIR) {
Support LLVM 12 in rustc
2020-11-03 15:47:16 -06:00
PipelineStartEPCallbacks.push_back(
PassWrapper: handle move of OptimizationLevel class out of PassBuilder This is the first build break of the LLVM 14 cycle, and was caused by https://reviews.llvm.org/D107025. Mercifully an easy fix.
2021-08-05 11:13:58 -05:00
[VerifyIR](ModulePassManager &MPM, OptimizationLevel Level) {
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
MPM.addPass(VerifierPass());
Support LLVM 12 in rustc
2020-11-03 15:47:16 -06:00
}
);
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
}
Explicitly register GCOV profiling pass as well
2021-04-05 04:26:48 -05:00
if (InstrumentGCOV) {
PipelineStartEPCallbacks.push_back(
PassWrapper: handle move of OptimizationLevel class out of PassBuilder This is the first build break of the LLVM 14 cycle, and was caused by https://reviews.llvm.org/D107025. Mercifully an easy fix.
2021-08-05 11:13:58 -05:00
[](ModulePassManager &MPM, OptimizationLevel Level) {
Explicitly register GCOV profiling pass as well
2021-04-05 04:26:48 -05:00
MPM.addPass(GCOVProfilerPass(GCOVOptions::getDefault()));
}
);
}
Explicitly register instrprof pass Don't use "passes" for this purpose, explicitly insert it into the correct place in the pipeline instead.
2021-04-05 03:45:04 -05:00
if (InstrumentCoverage) {
PipelineStartEPCallbacks.push_back(
PassWrapper: handle move of OptimizationLevel class out of PassBuilder This is the first build break of the LLVM 14 cycle, and was caused by https://reviews.llvm.org/D107025. Mercifully an easy fix.
2021-08-05 11:13:58 -05:00
[](ModulePassManager &MPM, OptimizationLevel Level) {
Explicitly register instrprof pass Don't use "passes" for this purpose, explicitly insert it into the correct place in the pipeline instead.
2021-04-05 03:45:04 -05:00
InstrProfOptions Options;
MPM.addPass(InstrProfiling(Options, false));
}
);
}
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
if (SanitizerOptions) {
if (SanitizerOptions->SanitizeMemory) {
MemorySanitizerOptions Options(
SanitizerOptions->SanitizeMemoryTrackOrigins,
Diagnose use of incompatible sanitizers Emit an error when incompatible sanitizer are configured through command line options. Previously the last one configured prevailed and others were silently ignored. Additionally use a set to represent configured sanitizers, making it possible to enable multiple sanitizers at once. At least in principle, since currently all of them are considered to be incompatible with others.
2020-06-13 19:00:00 -05:00
SanitizerOptions->SanitizeMemoryRecover,
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
/*CompileKernel=*/false);
Prepare for LLVM 11
2020-06-25 20:52:41 -05:00
OptimizerLastEPCallbacks.push_back(
PassWrapper: handle move of OptimizationLevel class out of PassBuilder This is the first build break of the LLVM 14 cycle, and was caused by https://reviews.llvm.org/D107025. Mercifully an easy fix.
2021-08-05 11:13:58 -05:00
[Options](ModulePassManager &MPM, OptimizationLevel Level) {
PassWrapper: handle separate Module*SanitizerPass Change ab41eef9aca3 in LLVM split MemorySanitizerPass into MemorySanitizerPass for functions and ModuleMemorySanitizerPass for modules. There's a related change for ThreadSanitizerPass, and in here since we're using a ModulePassManager I only add the module flavor of the pass on LLVM 14. r? @nikic cc @nagisa
2021-09-16 10:45:38 -05:00
#if LLVM_VERSION_GE(14, 0)
MPM.addPass(ModuleMemorySanitizerPass(Options));
#else
Prepare for LLVM 11
2020-06-25 20:52:41 -05:00
MPM.addPass(MemorySanitizerPass(Options));
PassWrapper: handle separate Module*SanitizerPass Change ab41eef9aca3 in LLVM split MemorySanitizerPass into MemorySanitizerPass for functions and ModuleMemorySanitizerPass for modules. There's a related change for ThreadSanitizerPass, and in here since we're using a ModulePassManager I only add the module flavor of the pass on LLVM 14. r? @nikic cc @nagisa
2021-09-16 10:45:38 -05:00
#endif
PassWrapper: these two lines shouldn't have been ifdef'd
2021-09-16 14:04:18 -05:00
MPM.addPass(createModuleToFunctionPassAdaptor(MemorySanitizerPass(Options)));
Prepare for LLVM 11
2020-06-25 20:52:41 -05:00
}
);
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
}
if (SanitizerOptions->SanitizeThread) {
Prepare for LLVM 11
2020-06-25 20:52:41 -05:00
OptimizerLastEPCallbacks.push_back(
PassWrapper: handle move of OptimizationLevel class out of PassBuilder This is the first build break of the LLVM 14 cycle, and was caused by https://reviews.llvm.org/D107025. Mercifully an easy fix.
2021-08-05 11:13:58 -05:00
[](ModulePassManager &MPM, OptimizationLevel Level) {
PassWrapper: handle separate Module*SanitizerPass Change ab41eef9aca3 in LLVM split MemorySanitizerPass into MemorySanitizerPass for functions and ModuleMemorySanitizerPass for modules. There's a related change for ThreadSanitizerPass, and in here since we're using a ModulePassManager I only add the module flavor of the pass on LLVM 14. r? @nikic cc @nagisa
2021-09-16 10:45:38 -05:00
#if LLVM_VERSION_GE(14, 0)
MPM.addPass(ModuleThreadSanitizerPass());
#else
Prepare for LLVM 11
2020-06-25 20:52:41 -05:00
MPM.addPass(ThreadSanitizerPass());
PassWrapper: handle separate Module*SanitizerPass Change ab41eef9aca3 in LLVM split MemorySanitizerPass into MemorySanitizerPass for functions and ModuleMemorySanitizerPass for modules. There's a related change for ThreadSanitizerPass, and in here since we're using a ModulePassManager I only add the module flavor of the pass on LLVM 14. r? @nikic cc @nagisa
2021-09-16 10:45:38 -05:00
#endif
PassWrapper: these two lines shouldn't have been ifdef'd
2021-09-16 14:04:18 -05:00
MPM.addPass(createModuleToFunctionPassAdaptor(ThreadSanitizerPass()));
Prepare for LLVM 11
2020-06-25 20:52:41 -05:00
}
);
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
}
if (SanitizerOptions->SanitizeAddress) {
Prepare for LLVM 11
2020-06-25 20:52:41 -05:00
OptimizerLastEPCallbacks.push_back(
PassWrapper: handle move of OptimizationLevel class out of PassBuilder This is the first build break of the LLVM 14 cycle, and was caused by https://reviews.llvm.org/D107025. Mercifully an easy fix.
2021-08-05 11:13:58 -05:00
[SanitizerOptions](ModulePassManager &MPM, OptimizationLevel Level) {
Prepare for LLVM 11
2020-06-25 20:52:41 -05:00
MPM.addPass(RequireAnalysisPass<ASanGlobalsMetadataAnalysis, Module>());
PassWrapper: adapt for LLVM 14 changes These API changes appear to have all taken place in https://reviews.llvm.org/D105007, which moved HWAddressSanitizerPass and AddressSanitizerPass to only accept their options type as a ctor argument instead of the sequence of bools etc. This required a couple of parameter additions, which I made match the default prior to the mentioned upstream LLVM change. This patch restores rustc to building (though not quite passing all tests, I've mailed other patches for those issues) against LLVM HEAD.
2021-08-19 11:40:37 -05:00
#if LLVM_VERSION_GE(14, 0)
rustc_llvm: update PassWrapper for recent LLVM Now AddressSanitizerOptions is a struct, but at least the change was tiny. r? nikic
2021-11-04 16:16:16 -05:00
AddressSanitizerOptions opts = AddressSanitizerOptions{
/*CompileKernel=*/false,
SanitizerOptions->SanitizeAddressRecover,
Didn't mean to invert this boolean.
2021-11-04 17:08:32 -05:00
/*UseAfterScope=*/true,
rustc_llvm: update PassWrapper for recent LLVM Now AddressSanitizerOptions is a struct, but at least the change was tiny. r? nikic
2021-11-04 16:16:16 -05:00
AsanDetectStackUseAfterReturnMode::Runtime,
};
MPM.addPass(ModuleAddressSanitizerPass(opts));
PassWrapper: adapt for LLVM 14 changes These API changes appear to have all taken place in https://reviews.llvm.org/D105007, which moved HWAddressSanitizerPass and AddressSanitizerPass to only accept their options type as a ctor argument instead of the sequence of bools etc. This required a couple of parameter additions, which I made match the default prior to the mentioned upstream LLVM change. This patch restores rustc to building (though not quite passing all tests, I've mailed other patches for those issues) against LLVM HEAD.
2021-08-19 11:40:37 -05:00
#else
rustc_llvm: update PassWrapper for recent LLVM Now AddressSanitizerOptions is a struct, but at least the change was tiny. r? nikic
2021-11-04 16:16:16 -05:00
MPM.addPass(ModuleAddressSanitizerPass(
/*CompileKernel=*/false, SanitizerOptions->SanitizeAddressRecover));
Prepare for LLVM 11
2020-06-25 20:52:41 -05:00
MPM.addPass(createModuleToFunctionPassAdaptor(AddressSanitizerPass(
/*CompileKernel=*/false, SanitizerOptions->SanitizeAddressRecover,
/*UseAfterScope=*/true)));
PassWrapper: adapt for LLVM 14 changes These API changes appear to have all taken place in https://reviews.llvm.org/D105007, which moved HWAddressSanitizerPass and AddressSanitizerPass to only accept their options type as a ctor argument instead of the sequence of bools etc. This required a couple of parameter additions, which I made match the default prior to the mentioned upstream LLVM change. This patch restores rustc to building (though not quite passing all tests, I've mailed other patches for those issues) against LLVM HEAD.
2021-08-19 11:40:37 -05:00
#endif
Prepare for LLVM 11
2020-06-25 20:52:41 -05:00
}
);
HWASan support
2021-01-22 20:32:38 -06:00
}
if (SanitizerOptions->SanitizeHWAddress) {
OptimizerLastEPCallbacks.push_back(
PassWrapper: handle move of OptimizationLevel class out of PassBuilder This is the first build break of the LLVM 14 cycle, and was caused by https://reviews.llvm.org/D107025. Mercifully an easy fix.
2021-08-05 11:13:58 -05:00
[SanitizerOptions](ModulePassManager &MPM, OptimizationLevel Level) {
PassWrapper: adapt for LLVM 14 changes These API changes appear to have all taken place in https://reviews.llvm.org/D105007, which moved HWAddressSanitizerPass and AddressSanitizerPass to only accept their options type as a ctor argument instead of the sequence of bools etc. This required a couple of parameter additions, which I made match the default prior to the mentioned upstream LLVM change. This patch restores rustc to building (though not quite passing all tests, I've mailed other patches for those issues) against LLVM HEAD.
2021-08-19 11:40:37 -05:00
#if LLVM_VERSION_GE(14, 0)
HWAddressSanitizerOptions opts(
/*CompileKernel=*/false, SanitizerOptions->SanitizeHWAddressRecover,
/*DisableOptimization=*/false);
MPM.addPass(HWAddressSanitizerPass(opts));
#else
HWASan support
2021-01-22 20:32:38 -06:00
MPM.addPass(HWAddressSanitizerPass(
/*CompileKernel=*/false, SanitizerOptions->SanitizeHWAddressRecover));
PassWrapper: adapt for LLVM 14 changes These API changes appear to have all taken place in https://reviews.llvm.org/D105007, which moved HWAddressSanitizerPass and AddressSanitizerPass to only accept their options type as a ctor argument instead of the sequence of bools etc. This required a couple of parameter additions, which I made match the default prior to the mentioned upstream LLVM change. This patch restores rustc to building (though not quite passing all tests, I've mailed other patches for those issues) against LLVM HEAD.
2021-08-19 11:40:37 -05:00
#endif
HWASan support
2021-01-22 20:32:38 -06:00
}
);
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
}
}
Use the existing llvm-plugins option for both legacy and new pm registration
2021-11-24 04:43:40 -06:00
if (LLVMPluginsLen) {
auto PluginsStr = StringRef(LLVMPlugins, LLVMPluginsLen);
Add a codegen option to allow loading LLVM pass plugins
2021-11-19 10:01:41 -06:00
SmallVector<StringRef> Plugins;
Use the existing llvm-plugins option for both legacy and new pm registration
2021-11-24 04:43:40 -06:00
PluginsStr.split(Plugins, ',', -1, false);
Add a codegen option to allow loading LLVM pass plugins
2021-11-19 10:01:41 -06:00
for (auto PluginPath: Plugins) {
auto Plugin = PassPlugin::Load(PluginPath.str());
if (!Plugin) {
LLVMRustSetLastError(("Failed to load pass plugin" + PluginPath.str()).c_str());
continue;
}
Plugin->registerPassBuilderCallbacks(PB);
}
}
PassWrapper: update for LLVM change D102093 In https://reviews.llvm.org/D102093 lots of things stopped taking the DebugLogging boolean parameter. Mercifully we appear to always set DebugPassManager to false, so I don't think we're losing anything by not passing this parameter.
2021-05-17 13:12:38 -05:00
#if LLVM_VERSION_GE(13, 0)
ModulePassManager MPM;
#else
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
ModulePassManager MPM(DebugPassManager);
PassWrapper: update for LLVM change D102093 In https://reviews.llvm.org/D102093 lots of things stopped taking the DebugLogging boolean parameter. Mercifully we appear to always set DebugPassManager to false, so I don't think we're losing anything by not passing this parameter.
2021-05-17 13:12:38 -05:00
#endif
Support LLVM 12 in rustc
2020-11-03 15:47:16 -06:00
bool NeedThinLTOBufferPasses = UseThinLTOBuffers;
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
if (!NoPrepopulatePasses) {
Use correct pipeline for LTO at O0 Unlike the pre-link piplines, the LTO pipelines do support O0, and using them is required to avoid leaving behind undefined references for the linker.
2021-09-13 11:18:59 -05:00
// The pre-link pipelines don't support O0 and require using budilO0DefaultPipeline() instead.
// At the same time, the LTO pipelines do support O0 and using them is required.
bool IsLTO = OptStage == LLVMRustOptStage::ThinLTO || OptStage == LLVMRustOptStage::FatLTO;
if (OptLevel == OptimizationLevel::O0 && !IsLTO) {
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
for (const auto &C : PipelineStartEPCallbacks)
Support LLVM 12 in rustc
2020-11-03 15:47:16 -06:00
PB.registerPipelineStartEPCallback(C);
for (const auto &C : OptimizerLastEPCallbacks)
PB.registerOptimizerLastEPCallback(C);
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
Support LLVM 12 in rustc
2020-11-03 15:47:16 -06:00
// Pass false as we manually schedule ThinLTOBufferPasses below.
MPM = PB.buildO0DefaultPipeline(OptLevel, /* PreLinkLTO */ false);
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
} else {
for (const auto &C : PipelineStartEPCallbacks)
PB.registerPipelineStartEPCallback(C);
Invoke OptimizerLastEPCallbacks in PreLinkThinLTO The default ThinLTO pre-link pipeline does not include optimizer last extension points. Thus, when using the new LLVM pass manager & ThinLTO & sanitizers on any opt-level different from zero, the sanitizer function passes would be omitted from the pipeline. Add optimizer last extensions points manually to the pipeline, but guard registration with stage check in the case this behaviour changes in the future.
2020-03-02 18:00:00 -06:00
if (OptStage != LLVMRustOptStage::PreLinkThinLTO) {
for (const auto &C : OptimizerLastEPCallbacks)
PB.registerOptimizerLastEPCallback(C);
}
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
switch (OptStage) {
case LLVMRustOptStage::PreLinkNoLTO:
MPM = PB.buildPerModuleDefaultPipeline(OptLevel, DebugPassManager);
break;
case LLVMRustOptStage::PreLinkThinLTO:
Support LLVM 12 in rustc
2020-11-03 15:47:16 -06:00
MPM = PB.buildThinLTOPreLinkDefaultPipeline(OptLevel);
Schedule ThinLTOBuffer passes again after sanitizer passes This works around a design defect in the LLVM 12 pass builder implementation. In LLVM 13, the PreLink ThinLTO pipeline properly respects the OptimizerLastEPCallbacks.
2021-03-02 14:54:03 -06:00
// The ThinLTOPreLink pipeline already includes ThinLTOBuffer passes. However, callback
// passes may still run afterwards. This means we need to run the buffer passes again.
// FIXME: In LLVM 13, the ThinLTOPreLink pipeline also runs OptimizerLastEPCallbacks
// before the RequiredLTOPreLinkPasses, in which case we can remove these hacks.
if (OptimizerLastEPCallbacks.empty())
NeedThinLTOBufferPasses = false;
Prepare for LLVM 11
2020-06-25 20:52:41 -05:00
for (const auto &C : OptimizerLastEPCallbacks)
C(MPM, OptLevel);
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
break;
case LLVMRustOptStage::PreLinkFatLTO:
Support LLVM 12 in rustc
2020-11-03 15:47:16 -06:00
MPM = PB.buildLTOPreLinkDefaultPipeline(OptLevel);
NeedThinLTOBufferPasses = false;
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
break;
case LLVMRustOptStage::ThinLTO:
// FIXME: Does it make sense to pass the ModuleSummaryIndex?
// It only seems to be needed for C++ specific optimizations.
Support LLVM 12 in rustc
2020-11-03 15:47:16 -06:00
MPM = PB.buildThinLTODefaultPipeline(OptLevel, nullptr);
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
break;
case LLVMRustOptStage::FatLTO:
Support LLVM 12 in rustc
2020-11-03 15:47:16 -06:00
MPM = PB.buildLTODefaultPipeline(OptLevel, nullptr);
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
break;
}
}
Respect -Z verify-llvm-ir and other flags that add extra passes when combined with -C no-prepopulate-passes in the new LLVM Pass Manager.
2022-04-10 14:30:35 -05:00
} else {
// We're not building any of the default pipelines but we still want to
// add the verifier, instrumentation, etc passes if they were requested
for (const auto &C : PipelineStartEPCallbacks)
C(MPM, OptLevel);
for (const auto &C : OptimizerLastEPCallbacks)
C(MPM, OptLevel);
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
}
Support -C passes in NewPM And report an error if parsing the additional pass pipeline fails. Threading through the error accounts for most of the changes here.
2021-04-05 08:37:11 -05:00
if (ExtraPassesLen) {
if (auto Err = PB.parsePassPipeline(MPM, StringRef(ExtraPasses, ExtraPassesLen))) {
std::string ErrMsg = toString(std::move(Err));
LLVMRustSetLastError(ErrMsg.c_str());
return LLVMRustResult::Failure;
}
}
Support LLVM 12 in rustc
2020-11-03 15:47:16 -06:00
if (NeedThinLTOBufferPasses) {
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
MPM.addPass(CanonicalizeAliasesPass());
MPM.addPass(NameAnonGlobalPass());
}
// Upgrade all calls to old intrinsics first.
for (Module::iterator I = TheModule->begin(), E = TheModule->end(); I != E;)
UpgradeCallsToIntrinsic(&*I++); // must be post-increment, as we remove
MPM.run(*TheModule, MAM);
Support -C passes in NewPM And report an error if parsing the additional pass pipeline fails. Threading through the error accounts for most of the changes here.
2021-04-05 08:37:11 -05:00
return LLVMRustResult::Success;
Add support for new pass manager The new pass manager can be enabled using -Z new-llvm-pass-manager=on.
2020-01-05 12:16:58 -06:00
}
When writing LLVM IR output demangled fn name in comments `--emit=llvm-ir` looks like this now: ``` ; <alloc::vec::Vec<T> as core::ops::index::IndexMut<core::ops::range::RangeFull>>::index_mut ; Function Attrs: inlinehint uwtable define internal { i8*, i64 } @"_ZN106_$LT$alloc..vec..Vec$LT$T$GT$$u20$as$u20$core..ops..index..IndexMut$LT$core..ops..range..RangeFull$GT$$GT$9index_mut17h7f7b576609f30262E"(%"alloc::vec::Vec<u8>"* dereferenceable(24)) unnamed_addr #0 { start: ... ``` cc https://github.com/integer32llc/rust-playground/issues/15
2017-06-29 09:52:43 -05:00
// Callback to demangle function name
// Parameters:
// * name to be demangled
// * name len
// * output buffer
// * output buffer len
// Returns len of demangled string, or 0 if demangle failed.
typedef size_t (*DemangleFn)(const char*, size_t, char*, size_t);
namespace {
class RustAssemblyAnnotationWriter : public AssemblyAnnotationWriter {
DemangleFn Demangle;
std::vector<char> Buf;
public:
RustAssemblyAnnotationWriter(DemangleFn Demangle) : Demangle(Demangle) {}
// Return empty string if demangle failed
// or if name does not need to be demangled
StringRef CallDemangle(StringRef name) {
if (!Demangle) {
return StringRef();
}
if (Buf.size() < name.size() * 2) {
// Semangled name usually shorter than mangled,
// but allocate twice as much memory just in case
Buf.resize(name.size() * 2);
}
auto R = Demangle(name.data(), name.size(), Buf.data(), Buf.size());
if (!R) {
// Demangle failed.
return StringRef();
}
auto Demangled = StringRef(Buf.data(), R);
if (Demangled == name) {
// Do not print anything if demangled name is equal to mangled.
return StringRef();
}
return Demangled;
}
void emitFunctionAnnot(const Function *F,
formatted_raw_ostream &OS) override {
StringRef Demangled = CallDemangle(F->getName());
if (Demangled.empty()) {
return;
}
OS << "; " << Demangled << "\n";
}
void emitInstructionAnnot(const Instruction *I,
formatted_raw_ostream &OS) override {
const char *Name;
const Value *Value;
if (const CallInst *CI = dyn_cast<CallInst>(I)) {
Name = "call";
Prepare for LLVM 11
2020-06-25 20:52:41 -05:00
Value = CI->getCalledOperand();
When writing LLVM IR output demangled fn name in comments `--emit=llvm-ir` looks like this now: ``` ; <alloc::vec::Vec<T> as core::ops::index::IndexMut<core::ops::range::RangeFull>>::index_mut ; Function Attrs: inlinehint uwtable define internal { i8*, i64 } @"_ZN106_$LT$alloc..vec..Vec$LT$T$GT$$u20$as$u20$core..ops..index..IndexMut$LT$core..ops..range..RangeFull$GT$$GT$9index_mut17h7f7b576609f30262E"(%"alloc::vec::Vec<u8>"* dereferenceable(24)) unnamed_addr #0 { start: ... ``` cc https://github.com/integer32llc/rust-playground/issues/15
2017-06-29 09:52:43 -05:00
} else if (const InvokeInst* II = dyn_cast<InvokeInst>(I)) {
Name = "invoke";
Prepare for LLVM 11
2020-06-25 20:52:41 -05:00
Value = II->getCalledOperand();
When writing LLVM IR output demangled fn name in comments `--emit=llvm-ir` looks like this now: ``` ; <alloc::vec::Vec<T> as core::ops::index::IndexMut<core::ops::range::RangeFull>>::index_mut ; Function Attrs: inlinehint uwtable define internal { i8*, i64 } @"_ZN106_$LT$alloc..vec..Vec$LT$T$GT$$u20$as$u20$core..ops..index..IndexMut$LT$core..ops..range..RangeFull$GT$$GT$9index_mut17h7f7b576609f30262E"(%"alloc::vec::Vec<u8>"* dereferenceable(24)) unnamed_addr #0 { start: ... ``` cc https://github.com/integer32llc/rust-playground/issues/15
2017-06-29 09:52:43 -05:00
} else {
// Could demangle more operations, e. g.
// `store %place, @function`.
return;
}
if (!Value->hasName()) {
return;
}
StringRef Demangled = CallDemangle(Value->getName());
if (Demangled.empty()) {
return;
}
OS << "; " << Name << " " << Demangled << "\n";
}
};
} // namespace
Show better errors for LLVM IR output I was trying to output LLVM IR directly to the console: $ rustc hello.rs --emit=llvm-ir -o /dev/stdout LLVM ERROR: IO failure on output stream: Bad file descriptor Now `LLVMRustPrintModule` returns an error, and we print: error: failed to write LLVM IR to /dev/stdout.hello.7rcbfp3g-cgu.0.rcgu.ll: Permission denied ... which is more informative.
2019-04-05 19:48:23 -05:00
extern "C" LLVMRustResult
Use Module::print() instead of a PrintModulePass llvm::Module has a print() method. It is unnecessary to create a pass just for the purpose of printing LLVM IR.
2019-12-02 03:53:01 -06:00
LLVMRustPrintModule(LLVMModuleRef M, const char *Path, DemangleFn Demangle) {
Rewrite pass management with LLVM Beforehand, it was unclear whether rust was performing the "recommended set" of optimizations provided by LLVM for code. This commit changes the way we run passes to closely mirror that of clang, which in theory does it correctly. The notable changes include: * Passes are no longer explicitly added one by one. This would be difficult to keep up with as LLVM changes and we don't guaranteed always know the best order in which to run passes * Passes are now managed by LLVM's PassManagerBuilder object. This is then used to populate the various pass managers run. * We now run both a FunctionPassManager and a module-wide PassManager. This is what clang does, and I presume that we *may* see a speed boost from the module-wide passes just having to do less work. I have no measured this. * The codegen pass manager has been extracted to its own separate pass manager to not get mixed up with the other passes * All pass managers now include passes for target-specific data layout and analysis passes Some new features include: * You can now print all passes being run with `-Z print-llvm-passes` * When specifying passes via `--passes`, the passes are now appended to the default list of passes instead of overwriting them. * The output of `--passes list` is now generated by LLVM instead of maintaining a list of passes ourselves * Loop vectorization is turned on by default as an optimization pass and can be disabled with `-Z no-vectorize-loops`
2013-08-22 22:58:42 -05:00
std::string ErrorInfo;
Update LLVM.
2014-09-30 16:20:22 -05:00
std::error_code EC;
Replace llvm::sys::fs::F_None with llvm::sys::fs::OF_None The former is deprecated. OF_None has been available in LLVM since 2018-06.
2021-04-29 17:25:17 -05:00
raw_fd_ostream OS(Path, EC, sys::fs::OF_None);
Update LLVM.
2014-09-30 16:20:22 -05:00
if (EC)
ErrorInfo = EC.message();
Show better errors for LLVM IR output I was trying to output LLVM IR directly to the console: $ rustc hello.rs --emit=llvm-ir -o /dev/stdout LLVM ERROR: IO failure on output stream: Bad file descriptor Now `LLVMRustPrintModule` returns an error, and we print: error: failed to write LLVM IR to /dev/stdout.hello.7rcbfp3g-cgu.0.rcgu.ll: Permission denied ... which is more informative.
2019-04-05 19:48:23 -05:00
if (ErrorInfo != "") {
LLVMRustSetLastError(ErrorInfo.c_str());
return LLVMRustResult::Failure;
}
rustc: Get rustc compiling with LLVM 3.{3,4} again The travis builds have been breaking recently because LLVM 3.5 upstream is changing. This looks like it's likely to continue, so it would be more useful for us if we could lock ourselves to a system LLVM version that is not changing. This commit has the support to bring our C++ glue to LLVM back in line with what was possible back in LLVM 3.{3,4}. I don't think we're going to be able to reasonably protect against regressions in the future, but this kind of code is a good sign that we can continue to use the system LLVM for simple-ish things. Codegen for ARM won't work and it won't have some of the perf improvements we have, but using the system LLVM should work well enough for development.
2014-02-26 16:06:27 -06:00
Use Module::print() instead of a PrintModulePass llvm::Module has a print() method. It is unnecessary to create a pass just for the purpose of printing LLVM IR.
2019-12-02 03:53:01 -06:00
RustAssemblyAnnotationWriter AAW(Demangle);
Rewrite pass management with LLVM Beforehand, it was unclear whether rust was performing the "recommended set" of optimizations provided by LLVM for code. This commit changes the way we run passes to closely mirror that of clang, which in theory does it correctly. The notable changes include: * Passes are no longer explicitly added one by one. This would be difficult to keep up with as LLVM changes and we don't guaranteed always know the best order in which to run passes * Passes are now managed by LLVM's PassManagerBuilder object. This is then used to populate the various pass managers run. * We now run both a FunctionPassManager and a module-wide PassManager. This is what clang does, and I presume that we *may* see a speed boost from the module-wide passes just having to do less work. I have no measured this. * The codegen pass manager has been extracted to its own separate pass manager to not get mixed up with the other passes * All pass managers now include passes for target-specific data layout and analysis passes Some new features include: * You can now print all passes being run with `-Z print-llvm-passes` * When specifying passes via `--passes`, the passes are now appended to the default list of passes instead of overwriting them. * The output of `--passes list` is now generated by LLVM instead of maintaining a list of passes ourselves * Loop vectorization is turned on by default as an optimization pass and can be disabled with `-Z no-vectorize-loops`
2013-08-22 22:58:42 -05:00
formatted_raw_ostream FOS(OS);
Use Module::print() instead of a PrintModulePass llvm::Module has a print() method. It is unnecessary to create a pass just for the purpose of printing LLVM IR.
2019-12-02 03:53:01 -06:00
unwrap(M)->print(FOS, &AAW);
Show better errors for LLVM IR output I was trying to output LLVM IR directly to the console: $ rustc hello.rs --emit=llvm-ir -o /dev/stdout LLVM ERROR: IO failure on output stream: Bad file descriptor Now `LLVMRustPrintModule` returns an error, and we print: error: failed to write LLVM IR to /dev/stdout.hello.7rcbfp3g-cgu.0.rcgu.ll: Permission denied ... which is more informative.
2019-04-05 19:48:23 -05:00
return LLVMRustResult::Success;
Rewrite pass management with LLVM Beforehand, it was unclear whether rust was performing the "recommended set" of optimizations provided by LLVM for code. This commit changes the way we run passes to closely mirror that of clang, which in theory does it correctly. The notable changes include: * Passes are no longer explicitly added one by one. This would be difficult to keep up with as LLVM changes and we don't guaranteed always know the best order in which to run passes * Passes are now managed by LLVM's PassManagerBuilder object. This is then used to populate the various pass managers run. * We now run both a FunctionPassManager and a module-wide PassManager. This is what clang does, and I presume that we *may* see a speed boost from the module-wide passes just having to do less work. I have no measured this. * The codegen pass manager has been extracted to its own separate pass manager to not get mixed up with the other passes * All pass managers now include passes for target-specific data layout and analysis passes Some new features include: * You can now print all passes being run with `-Z print-llvm-passes` * When specifying passes via `--passes`, the passes are now appended to the default list of passes instead of overwriting them. * The output of `--passes list` is now generated by LLVM instead of maintaining a list of passes ourselves * Loop vectorization is turned on by default as an optimization pass and can be disabled with `-Z no-vectorize-loops`
2013-08-22 22:58:42 -05:00
}
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
extern "C" void LLVMRustPrintPasses() {
LLVMInitializePasses();
struct MyListener : PassRegistrationListener {
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
void passEnumerate(const PassInfo *Info) {
StringRef PassArg = Info->getPassArgument();
StringRef PassName = Info->getPassName();
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
if (!PassArg.empty()) {
// These unsigned->signed casts could theoretically overflow, but
// realistically never will (and even if, the result is implementation
// defined rather plain UB).
printf("%15.*s - %.*s\n", (int)PassArg.size(), PassArg.data(),
(int)PassName.size(), PassName.data());
}
}
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
} Listener;
Rewrite pass management with LLVM Beforehand, it was unclear whether rust was performing the "recommended set" of optimizations provided by LLVM for code. This commit changes the way we run passes to closely mirror that of clang, which in theory does it correctly. The notable changes include: * Passes are no longer explicitly added one by one. This would be difficult to keep up with as LLVM changes and we don't guaranteed always know the best order in which to run passes * Passes are now managed by LLVM's PassManagerBuilder object. This is then used to populate the various pass managers run. * We now run both a FunctionPassManager and a module-wide PassManager. This is what clang does, and I presume that we *may* see a speed boost from the module-wide passes just having to do less work. I have no measured this. * The codegen pass manager has been extracted to its own separate pass manager to not get mixed up with the other passes * All pass managers now include passes for target-specific data layout and analysis passes Some new features include: * You can now print all passes being run with `-Z print-llvm-passes` * When specifying passes via `--passes`, the passes are now appended to the default list of passes instead of overwriting them. * The output of `--passes list` is now generated by LLVM instead of maintaining a list of passes ourselves * Loop vectorization is turned on by default as an optimization pass and can be disabled with `-Z no-vectorize-loops`
2013-08-22 22:58:42 -05:00
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
PassRegistry *PR = PassRegistry::getPassRegistry();
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
PR->enumerateWith(&Listener);
Refactor optimization pass handling. Refactor the optimization passes to explicitly use the passes. This commit just re-implements the same passes as were already being run. It also adds an option (behind `-Z`) to run the LLVM lint pass on the unoptimized IR.
2013-05-27 18:15:31 -05:00
}
rustc: Dispose of LLVM passes in test cases
2013-06-19 17:18:25 -05:00
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
extern "C" void LLVMRustAddAlwaysInlinePass(LLVMPassManagerBuilderRef PMBR,
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
bool AddLifetimes) {
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
unwrap(PMBR)->Inliner = llvm::createAlwaysInlinerLegacyPass(AddLifetimes);
rustc: Dispose of LLVM passes in test cases
2013-06-19 17:18:25 -05:00
}
Implement LTO This commit implements LTO for rust leveraging LLVM's passes. What this means is: * When compiling an rlib, in addition to insdering foo.o into the archive, also insert foo.bc (the LLVM bytecode) of the optimized module. * When the compiler detects the -Z lto option, it will attempt to perform LTO on a staticlib or binary output. The compiler will emit an error if a dylib or rlib output is being generated. * The actual act of performing LTO is as follows: 1. Force all upstream libraries to have an rlib version available. 2. Load the bytecode of each upstream library from the rlib. 3. Link all this bytecode into the current LLVM module (just using llvm apis) 4. Run an internalization pass which internalizes all symbols except those found reachable for the local crate of compilation. 5. Run the LLVM LTO pass manager over this entire module 6a. If assembling an archive, then add all upstream rlibs into the output archive. This ignores all of the object/bitcode/metadata files rust generated and placed inside the rlibs. 6b. If linking a binary, create copies of all upstream rlibs, remove the rust-generated object-file, and then link everything as usual. As I have explained in #10741, this process is excruciatingly slow, so this is *not* turned on by default, and it is also why I have decided to hide it behind a -Z flag for now. The good news is that the binary sizes are about as small as they can be as a result of LTO, so it's definitely working. Closes #10741 Closes #10740
2013-12-03 01:19:29 -06:00
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
extern "C" void LLVMRustRunRestrictionPass(LLVMModuleRef M, char **Symbols,
size_t Len) {
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
llvm::legacy::PassManager passes;
[LLVM-3.9] Preserve certain functions when internalizing This makes sure to still use the old way for older LLVM versions.
2016-07-12 17:42:44 -05:00
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
auto PreserveFunctions = [=](const GlobalValue &GV) {
Improve naming style in rustllvm. As per the LLVM style guide, use CamelCase for all locals and classes, and camelCase for all non-FFI functions. Also, make names of variables of commonly used types more consistent. Fixes #38688.
2016-12-31 11:01:23 -06:00
for (size_t I = 0; I < Len; I++) {
if (GV.getName() == Symbols[I]) {
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
return true;
}
}
return false;
};
[LLVM-3.9] Preserve certain functions when internalizing This makes sure to still use the old way for older LLVM versions.
2016-07-12 17:42:44 -05:00
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
passes.add(llvm::createInternalizePass(PreserveFunctions));
[LLVM-3.9] Preserve certain functions when internalizing This makes sure to still use the old way for older LLVM versions.
2016-07-12 17:42:44 -05:00
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
passes.run(*unwrap(M));
Implement LTO This commit implements LTO for rust leveraging LLVM's passes. What this means is: * When compiling an rlib, in addition to insdering foo.o into the archive, also insert foo.bc (the LLVM bytecode) of the optimized module. * When the compiler detects the -Z lto option, it will attempt to perform LTO on a staticlib or binary output. The compiler will emit an error if a dylib or rlib output is being generated. * The actual act of performing LTO is as follows: 1. Force all upstream libraries to have an rlib version available. 2. Load the bytecode of each upstream library from the rlib. 3. Link all this bytecode into the current LLVM module (just using llvm apis) 4. Run an internalization pass which internalizes all symbols except those found reachable for the local crate of compilation. 5. Run the LLVM LTO pass manager over this entire module 6a. If assembling an archive, then add all upstream rlibs into the output archive. This ignores all of the object/bitcode/metadata files rust generated and placed inside the rlibs. 6b. If linking a binary, create copies of all upstream rlibs, remove the rust-generated object-file, and then link everything as usual. As I have explained in #10741, this process is excruciatingly slow, so this is *not* turned on by default, and it is also why I have decided to hide it behind a -Z flag for now. The good news is that the binary sizes are about as small as they can be as a result of LTO, so it's definitely working. Closes #10741 Closes #10740
2013-12-03 01:19:29 -06:00
}
Disable all unwinding on -Z no-landing-pads LTO When performing LTO, the rust compiler has an opportunity to completely strip all landing pads in all dependent libraries. I've modified the LTO pass to recognize the -Z no-landing-pads option when also running an LTO pass to flag everything in LLVM as nothrow. I've verified that this prevents any and all invoke instructions from being emitted. I believe that this is one of our best options for moving forward with accomodating use-cases where unwinding doesn't really make sense. This will allow libraries to be built with landing pads by default but allow usage of them in contexts where landing pads aren't necessary. cc #10780
2013-12-11 01:27:15 -06:00
trans: Clean up handling the LLVM data layout Turns out for OSX our data layout was subtly wrong and the LLVM update must have exposed this. Instead of fixing this I've removed all data layouts from the compiler to just use the defaults that LLVM provides for all targets. All data layouts (and a number of dead modules) are removed from the compiler here. Custom target specifications can still provide a custom data layout, but it is now an optional key as the default will be used if one isn't specified.
2015-07-16 17:48:16 -05:00
extern "C" void
LLVMRustSetDataLayoutFromTargetMachine(LLVMModuleRef Module,
LLVMTargetMachineRef TMR) {
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
TargetMachine *Target = unwrap(TMR);
unwrap(Module)->setDataLayout(Target->createDataLayout());
trans: Clean up handling the LLVM data layout Turns out for OSX our data layout was subtly wrong and the LLVM update must have exposed this. Instead of fixing this I've removed all data layouts from the compiler to just use the defaults that LLVM provides for all targets. All data layouts (and a number of dead modules) are removed from the compiler here. Custom target specifications can still provide a custom data layout, but it is now an optional key as the default will be used if one isn't specified.
2015-07-16 17:48:16 -05:00
}
Configure LLVM module PIC level As of LLVM 9, this is required for 32-bit PowerPC to properly generate PLT references. Previously, only BigPIC was supported; now LLVM supports both BigPIC and SmallPIC, and there is no default value provided.
2019-11-03 10:52:00 -06:00
extern "C" void LLVMRustSetModulePICLevel(LLVMModuleRef M) {
unwrap(M)->setPICLevel(PICLevel::Level::BigPIC);
}
Ran clang-format on src/rustllvm with llvm as the coding style.
2016-12-30 05:22:11 -06:00
extern "C" void LLVMRustSetModulePIELevel(LLVMModuleRef M) {
unwrap(M)->setPIELevel(PIELevel::Level::Large);
[LLVM-3.9] Configure PIE at the module level instead of compilation unit level This was deleted here[1] which appears to be replaced by this[2] which is a new setPIELevel function on the LLVM module itself. [1]: http://reviews.llvm.org/D19753 [2]: http://reviews.llvm.org/D19671
2016-07-12 17:41:40 -05:00
}
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
Add support for storing code model to LLVM module IR This patch avoids undefined behavior by linking different object files. Also this would it could be propagated properly to LTO. See https://reviews.llvm.org/D52322 and https://reviews.llvm.org/D52323. This patch is based on https://github.com/rust-lang/rust/pull/74002
2021-03-11 17:35:13 -06:00
extern "C" void LLVMRustSetModuleCodeModel(LLVMModuleRef M,
LLVMRustCodeModel Model) {
auto CM = fromRust(Model);
if (!CM.hasValue())
return;
unwrap(M)->setCodeModel(*CM);
}
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
// Here you'll find an implementation of ThinLTO as used by the Rust compiler
// right now. This ThinLTO support is only enabled on "recent ish" versions of
// LLVM, and otherwise it's just blanket rejected from other compilers.
//
// Most of this implementation is straight copied from LLVM. At the time of
// this writing it wasn't *quite* suitable to reuse more code from upstream
// for our purposes, but we should strive to upstream this support once it's
// ready to go! I figure we may want a bit of testing locally first before
// sending this upstream to LLVM. I hear though they're quite eager to receive
// feedback like this!
//
// If you're reading this code and wondering "what in the world" or you're
// working "good lord by LLVM upgrade is *still* failing due to these bindings"
// then fear not! (ok maybe fear a little). All code here is mostly based
// on `lib/LTO/ThinLTOCodeGenerator.cpp` in LLVM.
//
// You'll find that the general layout here roughly corresponds to the `run`
// method in that file as well as `ProcessThinLTOModule`. Functions are
// specifically commented below as well, but if you're updating this code
// or otherwise trying to understand it, the LLVM source will be useful in
// interpreting the mysteries within.
//
// Otherwise I'll apologize in advance, it probably requires a relatively
// significant investment on your part to "truly understand" what's going on
// here. Not saying I do myself, but it took me awhile staring at LLVM's source
// and various online resources about ThinLTO to make heads or tails of all
// this.
// This is a shared data structure which *must* be threadsafe to share
// read-only amongst threads. This also corresponds basically to the arguments
// of the `ProcessThinLTOModule` function in the LLVM source.
struct LLVMRustThinLTOData {
// The combined index that is the global analysis over all modules we're
// performing ThinLTO for. This is mostly managed by LLVM.
ModuleSummaryIndex Index;
// All modules we may look at, stored as in-memory serialized versions. This
// is later used when inlining to ensure we can extract any module to inline
// from.
StringMap<MemoryBufferRef> ModuleMap;
// A set that we manage of everything we *don't* want internalized. Note that
// this includes all transitive references right now as well, but it may not
// always!
DenseSet<GlobalValue::GUID> GUIDPreservedSymbols;
// Not 100% sure what these are, but they impact what's internalized and
// what's inlined across modules, I believe.
StringMap<FunctionImporter::ImportMapTy> ImportLists;
StringMap<FunctionImporter::ExportSetTy> ExportLists;
StringMap<GVSummaryMapTy> ModuleToDefinedGVSummaries;
Use llvm::computeLTOCacheKey to determine post-ThinLTO CGU reuse During incremental ThinLTO compilation, we attempt to re-use the optimized (post-ThinLTO) bitcode file for a module if it is 'safe' to do so. Up until now, 'safe' has meant that the set of modules that our current modules imports from/exports to is unchanged from the previous compilation session. See PR #67020 and PR #71131 for more details. However, this turns out be insufficient to guarantee that it's safe to reuse the post-LTO module (i.e. that optimizing the pre-LTO module would produce the same result). When LLVM optimizes a module during ThinLTO, it may look at other information from the 'module index', such as whether a (non-imported!) global variable is used. If this information changes between compilation runs, we may end up re-using an optimized module that (for example) had dead-code elimination run on a function that is now used by another module. Fortunately, LLVM implements its own ThinLTO module cache, which is used when ThinLTO is performed by a linker plugin (e.g. when clang is used to compile a C proect). Using this cache directly would require extensive refactoring of our code - but fortunately for us, LLVM provides a function that does exactly what we need. The function `llvm::computeLTOCacheKey` is used to compute a SHA-1 hash from all data that might influence the result of ThinLTO on a module. In addition to the module imports/exports that we manually track, it also hashes information about global variables (e.g. their liveness) which might be used during optimization. By using this function, we shouldn't have to worry about new LLVM passes breaking our module re-use behavior. In LLVM, the output of this function forms part of the filename used to store the post-ThinLTO module. To keep our current filename structure intact, this PR just writes out the mapping 'CGU name -> Hash' to a file. To determine if a post-LTO module should be reused, we compare hashes from the previous session. This should unblock PR #75199 - by sheer chance, it seems to have hit this issue due to the particular CGU partitioning and optimization decisions that end up getting made.
2020-09-17 16:36:13 -05:00
StringMap<std::map<GlobalValue::GUID, GlobalValue::LinkageTypes>> ResolvedODR;
rustc: Add some defines for LLVM 7 compat I was testing out the tip support to see what's going on with wasm, and this was I believe the only issue encountered with LLVM 7 support so far.
2018-01-30 13:53:18 -06:00
Rebase to the llvm-project monorepo The new git submodule src/llvm-project is a monorepo replacing src/llvm and src/tools/{clang,lld,lldb}. This also serves as a rebase for these projects to the new 8.x branch from trunk. The src/llvm-emscripten fork is unchanged for now.
2019-01-16 11:59:03 -06:00
LLVMRustThinLTOData() : Index(/* HaveGVs = */ false) {}
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
};
// Just an argument to the `LLVMRustCreateThinLTOData` function below.
struct LLVMRustThinLTOModule {
const char *identifier;
const char *data;
size_t len;
};
// This is copied from `lib/LTO/ThinLTOCodeGenerator.cpp`, not sure what it
// does.
static const GlobalValueSummary *
getFirstDefinitionForLinker(const GlobalValueSummaryList &GVSummaryList) {
auto StrongDefForLinker = llvm::find_if(
GVSummaryList, [](const std::unique_ptr<GlobalValueSummary> &Summary) {
auto Linkage = Summary->linkage();
return !GlobalValue::isAvailableExternallyLinkage(Linkage) &&
!GlobalValue::isWeakForLinker(Linkage);
});
if (StrongDefForLinker != GVSummaryList.end())
return StrongDefForLinker->get();
auto FirstDefForLinker = llvm::find_if(
GVSummaryList, [](const std::unique_ptr<GlobalValueSummary> &Summary) {
auto Linkage = Summary->linkage();
return !GlobalValue::isAvailableExternallyLinkage(Linkage);
});
if (FirstDefForLinker == GVSummaryList.end())
return nullptr;
return FirstDefForLinker->get();
}
// The main entry point for creating the global ThinLTO analysis. The structure
// here is basically the same as before threads are spawned in the `run`
// function of `lib/LTO/ThinLTOCodeGenerator.cpp`.
extern "C" LLVMRustThinLTOData*
LLVMRustCreateThinLTOData(LLVMRustThinLTOModule *modules,
int num_modules,
const char **preserved_symbols,
int num_symbols) {
Handle removal of llvm::make_unique()
2019-12-31 07:08:25 -06:00
auto Ret = std::make_unique<LLVMRustThinLTOData>();
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
// Load each module's summary and merge it into one combined index
for (int i = 0; i < num_modules; i++) {
auto module = &modules[i];
StringRef buffer(module->data, module->len);
MemoryBufferRef mem_buffer(buffer, module->identifier);
Ret->ModuleMap[module->identifier] = mem_buffer;
Update ThinLTO for LLVM 5
2017-10-10 14:29:14 -05:00
if (Error Err = readModuleSummaryIndex(mem_buffer, Ret->Index, i)) {
LLVMRustSetLastError(toString(std::move(Err)).c_str());
return nullptr;
}
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
}
// Collect for each module the list of function it defines (GUID -> Summary)
Ret->Index.collectDefinedGVSummariesPerModule(Ret->ModuleToDefinedGVSummaries);
// Convert the preserved symbols set from string to GUID, this is then needed
rustc: Tweak the `isExported` callback for ThinLTO Previously we were too eagerly exporting almost all symbols used in ThinLTO which can cause a whole host of problems downstream! This commit instead fixes this error by aligning more closely with `lib/LTO/LTO.cpp` in LLVM's codebase which is to only change the linkage of summaries which are computed as dead. Closes #46374
2017-11-29 14:00:10 -06:00
// for internalization.
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
for (int i = 0; i < num_symbols; i++) {
rustc: Tweak the `isExported` callback for ThinLTO Previously we were too eagerly exporting almost all symbols used in ThinLTO which can cause a whole host of problems downstream! This commit instead fixes this error by aligning more closely with `lib/LTO/LTO.cpp` in LLVM's codebase which is to only change the linkage of summaries which are computed as dead. Closes #46374
2017-11-29 14:00:10 -06:00
auto GUID = GlobalValue::getGUID(preserved_symbols[i]);
Ret->GUIDPreservedSymbols.insert(GUID);
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
}
// Collect the import/export lists for all modules from the call-graph in the
// combined index
//
// This is copied from `lib/LTO/ThinLTOCodeGenerator.cpp`
rustc: Add some defines for LLVM 7 compat I was testing out the tip support to see what's going on with wasm, and this was I believe the only issue encountered with LLVM 7 support so far.
2018-01-30 13:53:18 -06:00
auto deadIsPrevailing = [&](GlobalValue::GUID G) {
return PrevailingType::Unknown;
};
Limit internalization in LLVM 8 ThinLTO
2019-04-26 10:58:14 -05:00
// We don't have a complete picture in our use of ThinLTO, just our immediate
// crate, so we need `ImportEnabled = false` to limit internalization.
// Otherwise, we sometimes lose `static` values -- see #60184.
Rebase to the llvm-project monorepo The new git submodule src/llvm-project is a monorepo replacing src/llvm and src/tools/{clang,lld,lldb}. This also serves as a rebase for these projects to the new 8.x branch from trunk. The src/llvm-emscripten fork is unchanged for now.
2019-01-16 11:59:03 -06:00
computeDeadSymbolsWithConstProp(Ret->Index, Ret->GUIDPreservedSymbols,
Limit internalization in LLVM 8 ThinLTO
2019-04-26 10:58:14 -05:00
deadIsPrevailing, /* ImportEnabled = */ false);
Update ThinLTO (internalization) for LLVM 5 Ref: https://github.com/llvm-mirror/llvm/commit/ccb80b9c0f60f33780e5e29bf66a87bb56968b99
2017-10-17 00:41:56 -05:00
ComputeCrossModuleImport(
Ret->Index,
Ret->ModuleToDefinedGVSummaries,
Ret->ImportLists,
Ret->ExportLists
);
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
// Resolve LinkOnce/Weak symbols, this has to be computed early be cause it
// impacts the caching.
//
rustc: Tweak the `isExported` callback for ThinLTO Previously we were too eagerly exporting almost all symbols used in ThinLTO which can cause a whole host of problems downstream! This commit instead fixes this error by aligning more closely with `lib/LTO/LTO.cpp` in LLVM's codebase which is to only change the linkage of summaries which are computed as dead. Closes #46374
2017-11-29 14:00:10 -06:00
// This is copied from `lib/LTO/ThinLTOCodeGenerator.cpp` with some of this
// being lifted from `lib/LTO/LTO.cpp` as well
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
DenseMap<GlobalValue::GUID, const GlobalValueSummary *> PrevailingCopy;
for (auto &I : Ret->Index) {
Update ThinLTO for LLVM 5
2017-10-10 14:29:14 -05:00
if (I.second.SummaryList.size() > 1)
PrevailingCopy[I.first] = getFirstDefinitionForLinker(I.second.SummaryList);
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
}
auto isPrevailing = [&](GlobalValue::GUID GUID, const GlobalValueSummary *S) {
const auto &Prevailing = PrevailingCopy.find(GUID);
if (Prevailing == PrevailingCopy.end())
return true;
return Prevailing->second == S;
};
auto recordNewLinkage = [&](StringRef ModuleIdentifier,
GlobalValue::GUID GUID,
GlobalValue::LinkageTypes NewLinkage) {
Use llvm::computeLTOCacheKey to determine post-ThinLTO CGU reuse During incremental ThinLTO compilation, we attempt to re-use the optimized (post-ThinLTO) bitcode file for a module if it is 'safe' to do so. Up until now, 'safe' has meant that the set of modules that our current modules imports from/exports to is unchanged from the previous compilation session. See PR #67020 and PR #71131 for more details. However, this turns out be insufficient to guarantee that it's safe to reuse the post-LTO module (i.e. that optimizing the pre-LTO module would produce the same result). When LLVM optimizes a module during ThinLTO, it may look at other information from the 'module index', such as whether a (non-imported!) global variable is used. If this information changes between compilation runs, we may end up re-using an optimized module that (for example) had dead-code elimination run on a function that is now used by another module. Fortunately, LLVM implements its own ThinLTO module cache, which is used when ThinLTO is performed by a linker plugin (e.g. when clang is used to compile a C proect). Using this cache directly would require extensive refactoring of our code - but fortunately for us, LLVM provides a function that does exactly what we need. The function `llvm::computeLTOCacheKey` is used to compute a SHA-1 hash from all data that might influence the result of ThinLTO on a module. In addition to the module imports/exports that we manually track, it also hashes information about global variables (e.g. their liveness) which might be used during optimization. By using this function, we shouldn't have to worry about new LLVM passes breaking our module re-use behavior. In LLVM, the output of this function forms part of the filename used to store the post-ThinLTO module. To keep our current filename structure intact, this PR just writes out the mapping 'CGU name -> Hash' to a file. To determine if a post-LTO module should be reused, we compare hashes from the previous session. This should unblock PR #75199 - by sheer chance, it seems to have hit this issue due to the particular CGU partitioning and optimization decisions that end up getting made.
2020-09-17 16:36:13 -05:00
Ret->ResolvedODR[ModuleIdentifier][GUID] = NewLinkage;
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
};
fully exploited the dropped support of LLVM 8 This commit grepped for LLVM_VERSION_GE, LLVM_VERSION_LT, get_major_version and min-llvm-version and statically evaluated every expression possible (and sensible) assuming that the LLVM version is >=9 now
2020-11-07 16:25:45 -06:00
fix: I meant LLVM version 13, not 12
2021-03-19 15:46:40 -05:00
#if LLVM_VERSION_GE(13,0)
cleanup: add some comments per review feedback
2021-03-22 13:37:49 -05:00
// Uses FromPrevailing visibility scheme which works for many binary
// formats. We probably could and should use ELF visibility scheme for many of
// our targets, however.
llvm-wrapper: adapt to function signature change of thinLTOResolvePrevailingInIndex This changed in 54fb3ca96e261f7107cb1b5778c34cb0e0808be6 - I'm not entirely sure it's correct that we're leaving config empty, but the one case in LLVM that looked similar did that.
2021-03-16 15:45:21 -05:00
lto::Config conf;
thinLTOResolvePrevailingInIndex(conf, Ret->Index, isPrevailing, recordNewLinkage,
Ret->GUIDPreservedSymbols);
#else
Pass GUIDPreservedSymbols to thinLTOResolvePrevailingInIndex()
2019-07-06 12:02:48 -05:00
thinLTOResolvePrevailingInIndex(Ret->Index, isPrevailing, recordNewLinkage,
Ret->GUIDPreservedSymbols);
llvm-wrapper: adapt to function signature change of thinLTOResolvePrevailingInIndex This changed in 54fb3ca96e261f7107cb1b5778c34cb0e0808be6 - I'm not entirely sure it's correct that we're leaving config empty, but the one case in LLVM that looked similar did that.
2021-03-16 15:45:21 -05:00
#endif
rustc: Tweak the `isExported` callback for ThinLTO Previously we were too eagerly exporting almost all symbols used in ThinLTO which can cause a whole host of problems downstream! This commit instead fixes this error by aligning more closely with `lib/LTO/LTO.cpp` in LLVM's codebase which is to only change the linkage of summaries which are computed as dead. Closes #46374
2017-11-29 14:00:10 -06:00
// Here we calculate an `ExportedGUIDs` set for use in the `isExported`
// callback below. This callback below will dictate the linkage for all
// summaries in the index, and we basically just only want to ensure that dead
// symbols are internalized. Otherwise everything that's already external
// linkage will stay as external, and internal will stay as internal.
std::set<GlobalValue::GUID> ExportedGUIDs;
for (auto &List : Ret->Index) {
ThinLTO: updates for LLVM 5 refs: https://github.com/llvm-mirror/llvm/commit/ccb80b9c0f60f33780e5e29bf66a87bb56968b99 https://github.com/llvm-mirror/llvm/commit/e611018a3f1237c9328763027db4a616ed7be04a
2017-12-11 00:52:58 -06:00
for (auto &GVS: List.second.SummaryList) {
rustc: Further tweak linkage in ThinLTO In #46382 the logic around linkage preservation with ThinLTO ws tweaked but the loop that registered all otherwise exported GUID values as "don't internalize me please" was erroneously too conservative and only asking "external" linkage items to not be internalized. Instead we actually want the inversion of that condition, everything *without* "local" linkage to be internalized. This commit updates the condition there, adds a test, and... Closes #46543
2017-12-06 19:59:33 -06:00
if (GlobalValue::isLocalLinkage(GVS->linkage()))
rustc: Tweak the `isExported` callback for ThinLTO Previously we were too eagerly exporting almost all symbols used in ThinLTO which can cause a whole host of problems downstream! This commit instead fixes this error by aligning more closely with `lib/LTO/LTO.cpp` in LLVM's codebase which is to only change the linkage of summaries which are computed as dead. Closes #46374
2017-11-29 14:00:10 -06:00
continue;
auto GUID = GVS->getOriginalName();
ThinLTO: updates for LLVM 5 refs: https://github.com/llvm-mirror/llvm/commit/ccb80b9c0f60f33780e5e29bf66a87bb56968b99 https://github.com/llvm-mirror/llvm/commit/e611018a3f1237c9328763027db4a616ed7be04a
2017-12-11 00:52:58 -06:00
if (GVS->flags().Live)
rustc: Tweak the `isExported` callback for ThinLTO Previously we were too eagerly exporting almost all symbols used in ThinLTO which can cause a whole host of problems downstream! This commit instead fixes this error by aligning more closely with `lib/LTO/LTO.cpp` in LLVM's codebase which is to only change the linkage of summaries which are computed as dead. Closes #46374
2017-11-29 14:00:10 -06:00
ExportedGUIDs.insert(GUID);
}
}
Update thinLTOInternalizeAndPromoteInIndex() usage
2019-12-31 07:14:31 -06:00
auto isExported = [&](StringRef ModuleIdentifier, ValueInfo VI) {
const auto &ExportList = Ret->ExportLists.find(ModuleIdentifier);
return (ExportList != Ret->ExportLists.end() &&
ExportList->second.count(VI)) ||
ExportedGUIDs.count(VI.getGUID());
};
thinLTOInternalizeAndPromoteInIndex(Ret->Index, isExported, isPrevailing);
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
return Ret.release();
}
extern "C" void
LLVMRustFreeThinLTOData(LLVMRustThinLTOData *Data) {
delete Data;
}
// Below are the various passes that happen *per module* when doing ThinLTO.
//
// In other words, these are the functions that are all run concurrently
// with one another, one per module. The passes here correspond to the analysis
// passes in `lib/LTO/ThinLTOCodeGenerator.cpp`, currently found in the
// `ProcessThinLTOModule` function. Here they're split up into separate steps
// so rustc can save off the intermediate bytecode between each step.
Prepare for LLVM 11
2020-06-25 20:52:41 -05:00
static bool
clearDSOLocalOnDeclarations(Module &Mod, TargetMachine &TM) {
// When linking an ELF shared object, dso_local should be dropped. We
// conservatively do this for -fpic.
bool ClearDSOLocalOnDeclarations =
TM.getTargetTriple().isOSBinFormatELF() &&
TM.getRelocationModel() != Reloc::Static &&
Mod.getPIELevel() == PIELevel::Default;
return ClearDSOLocalOnDeclarations;
}
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
extern "C" bool
Prepare for LLVM 11
2020-06-25 20:52:41 -05:00
LLVMRustPrepareThinLTORename(const LLVMRustThinLTOData *Data, LLVMModuleRef M,
LLVMTargetMachineRef TM) {
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
Module &Mod = *unwrap(M);
Prepare for LLVM 11
2020-06-25 20:52:41 -05:00
TargetMachine &Target = *unwrap(TM);
bool ClearDSOLocal = clearDSOLocalOnDeclarations(Mod, Target);
bool error = renameModuleForThinLTO(Mod, Data->Index, ClearDSOLocal);
if (error) {
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
LLVMRustSetLastError("renameModuleForThinLTO failed");
return false;
}
return true;
}
extern "C" bool
LLVMRustPrepareThinLTOResolveWeak(const LLVMRustThinLTOData *Data, LLVMModuleRef M) {
Module &Mod = *unwrap(M);
const auto &DefinedGlobals = Data->ModuleToDefinedGVSummaries.lookup(Mod.getModuleIdentifier());
PassWrapper: handle function rename from upstream D36850 thinLTOResolvePrevailingInModule became thinLTOFinalizeInModule and gained the ability to propagate noRecurse and noUnwind function attributes. I ran codegen tests with it both on and off, as the upstream patch uses it in both modes, and the tests pass both ways. Given that, it seemed reasonable to go ahead and let the propagation be enabled in rustc, and see what happens. See https://reviews.llvm.org/D36850 for more examples of how the new version of the function gets used.
2021-09-27 16:03:08 -05:00
#if LLVM_VERSION_GE(14, 0)
thinLTOFinalizeInModule(Mod, DefinedGlobals, /*PropagateAttrs=*/true);
#else
Handle some renamed ThinLTO functions
2018-11-09 15:09:32 -06:00
thinLTOResolvePrevailingInModule(Mod, DefinedGlobals);
PassWrapper: handle function rename from upstream D36850 thinLTOResolvePrevailingInModule became thinLTOFinalizeInModule and gained the ability to propagate noRecurse and noUnwind function attributes. I ran codegen tests with it both on and off, as the upstream patch uses it in both modes, and the tests pass both ways. Given that, it seemed reasonable to go ahead and let the propagation be enabled in rustc, and see what happens. See https://reviews.llvm.org/D36850 for more examples of how the new version of the function gets used.
2021-09-27 16:03:08 -05:00
#endif
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
return true;
}
extern "C" bool
LLVMRustPrepareThinLTOInternalize(const LLVMRustThinLTOData *Data, LLVMModuleRef M) {
Module &Mod = *unwrap(M);
const auto &DefinedGlobals = Data->ModuleToDefinedGVSummaries.lookup(Mod.getModuleIdentifier());
thinLTOInternalizeModule(Mod, DefinedGlobals);
return true;
}
extern "C" bool
Prepare for LLVM 11
2020-06-25 20:52:41 -05:00
LLVMRustPrepareThinLTOImport(const LLVMRustThinLTOData *Data, LLVMModuleRef M,
LLVMTargetMachineRef TM) {
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
Module &Mod = *unwrap(M);
Prepare for LLVM 11
2020-06-25 20:52:41 -05:00
TargetMachine &Target = *unwrap(TM);
rustc: Work around an upstream wasm ThinLTO bug This commit implements a workaround for an [upstream LLVM bug][1] where custom sections were accidentally duplicated amongst codegen units when ThinLTO passes were performed. This is due to the fact that custom sections for wasm are stored as metadata nodes which are automatically imported into modules when ThinLTO happens. The fix here is to forcibly delete the metadata node from imported modules before LLVM has a chance to try to copy it over. [1]: https://bugs.llvm.org/show_bug.cgi?id=38184
2018-07-18 14:12:53 -05:00
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
const auto &ImportList = Data->ImportLists.lookup(Mod.getModuleIdentifier());
auto Loader = [&](StringRef Identifier) {
const auto &Memory = Data->ModuleMap.lookup(Identifier);
auto &Context = Mod.getContext();
rustc: Work around an upstream wasm ThinLTO bug This commit implements a workaround for an [upstream LLVM bug][1] where custom sections were accidentally duplicated amongst codegen units when ThinLTO passes were performed. This is due to the fact that custom sections for wasm are stored as metadata nodes which are automatically imported into modules when ThinLTO happens. The fix here is to forcibly delete the metadata node from imported modules before LLVM has a chance to try to copy it over. [1]: https://bugs.llvm.org/show_bug.cgi?id=38184
2018-07-18 14:12:53 -05:00
auto MOrErr = getLazyBitcodeModule(Memory, Context, true, true);
if (!MOrErr)
Fix -Wpessimizing-move warnings in rustllvm/PassWrapper
2018-07-29 06:20:06 -05:00
return MOrErr;
rustc: Work around an upstream wasm ThinLTO bug This commit implements a workaround for an [upstream LLVM bug][1] where custom sections were accidentally duplicated amongst codegen units when ThinLTO passes were performed. This is due to the fact that custom sections for wasm are stored as metadata nodes which are automatically imported into modules when ThinLTO happens. The fix here is to forcibly delete the metadata node from imported modules before LLVM has a chance to try to copy it over. [1]: https://bugs.llvm.org/show_bug.cgi?id=38184
2018-07-18 14:12:53 -05:00
// The rest of this closure is a workaround for
// https://bugs.llvm.org/show_bug.cgi?id=38184 where during ThinLTO imports
// we accidentally import wasm custom sections into different modules,
// duplicating them by in the final output artifact.
//
// The issue is worked around here by manually removing the
// `wasm.custom_sections` named metadata node from any imported module. This
// we know isn't used by any optimization pass so there's no need for it to
// be imported.
//
// Note that the metadata is currently lazily loaded, so we materialize it
// here before looking up if there's metadata inside. The `FunctionImporter`
// will immediately materialize metadata anyway after an import, so this
// shouldn't be a perf hit.
if (Error Err = (*MOrErr)->materializeMetadata()) {
Expected<std::unique_ptr<Module>> Ret(std::move(Err));
Fix -Wpessimizing-move warnings in rustllvm/PassWrapper
2018-07-29 06:20:06 -05:00
return Ret;
rustc: Work around an upstream wasm ThinLTO bug This commit implements a workaround for an [upstream LLVM bug][1] where custom sections were accidentally duplicated amongst codegen units when ThinLTO passes were performed. This is due to the fact that custom sections for wasm are stored as metadata nodes which are automatically imported into modules when ThinLTO happens. The fix here is to forcibly delete the metadata node from imported modules before LLVM has a chance to try to copy it over. [1]: https://bugs.llvm.org/show_bug.cgi?id=38184
2018-07-18 14:12:53 -05:00
}
auto *WasmCustomSections = (*MOrErr)->getNamedMetadata("wasm.custom_sections");
if (WasmCustomSections)
WasmCustomSections->eraseFromParent();
Fix -Wpessimizing-move warnings in rustllvm/PassWrapper
2018-07-29 06:20:06 -05:00
return MOrErr;
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
};
Prepare for LLVM 11
2020-06-25 20:52:41 -05:00
bool ClearDSOLocal = clearDSOLocalOnDeclarations(Mod, Target);
FunctionImporter Importer(Data->Index, Loader, ClearDSOLocal);
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
Expected<bool> Result = Importer.importFunctions(Mod, ImportList);
if (!Result) {
LLVMRustSetLastError(toString(Result.takeError()).c_str());
return false;
}
return true;
}
Provide a way of accessing the ThinLTO module import map in rustc.
2018-08-17 08:25:46 -05:00
extern "C" typedef void (*LLVMRustModuleNameCallback)(void*, // payload
const char*, // importing module name
const char*); // imported module name
// Calls `module_name_callback` for each module import done by ThinLTO.
// The callback is provided with regular null-terminated C strings.
extern "C" void
Use llvm::computeLTOCacheKey to determine post-ThinLTO CGU reuse During incremental ThinLTO compilation, we attempt to re-use the optimized (post-ThinLTO) bitcode file for a module if it is 'safe' to do so. Up until now, 'safe' has meant that the set of modules that our current modules imports from/exports to is unchanged from the previous compilation session. See PR #67020 and PR #71131 for more details. However, this turns out be insufficient to guarantee that it's safe to reuse the post-LTO module (i.e. that optimizing the pre-LTO module would produce the same result). When LLVM optimizes a module during ThinLTO, it may look at other information from the 'module index', such as whether a (non-imported!) global variable is used. If this information changes between compilation runs, we may end up re-using an optimized module that (for example) had dead-code elimination run on a function that is now used by another module. Fortunately, LLVM implements its own ThinLTO module cache, which is used when ThinLTO is performed by a linker plugin (e.g. when clang is used to compile a C proect). Using this cache directly would require extensive refactoring of our code - but fortunately for us, LLVM provides a function that does exactly what we need. The function `llvm::computeLTOCacheKey` is used to compute a SHA-1 hash from all data that might influence the result of ThinLTO on a module. In addition to the module imports/exports that we manually track, it also hashes information about global variables (e.g. their liveness) which might be used during optimization. By using this function, we shouldn't have to worry about new LLVM passes breaking our module re-use behavior. In LLVM, the output of this function forms part of the filename used to store the post-ThinLTO module. To keep our current filename structure intact, this PR just writes out the mapping 'CGU name -> Hash' to a file. To determine if a post-LTO module should be reused, we compare hashes from the previous session. This should unblock PR #75199 - by sheer chance, it seems to have hit this issue due to the particular CGU partitioning and optimization decisions that end up getting made.
2020-09-17 16:36:13 -05:00
LLVMRustGetThinLTOModules(const LLVMRustThinLTOData *data,
Provide a way of accessing the ThinLTO module import map in rustc.
2018-08-17 08:25:46 -05:00
LLVMRustModuleNameCallback module_name_callback,
void* callback_payload) {
for (const auto& importing_module : data->ImportLists) {
const std::string importing_module_id = importing_module.getKey().str();
const auto& imports = importing_module.getValue();
for (const auto& imported_module : imports) {
const std::string imported_module_id = imported_module.getKey().str();
module_name_callback(callback_payload,
importing_module_id.c_str(),
imported_module_id.c_str());
}
}
}
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
// This struct and various functions are sort of a hack right now, but the
// problem is that we've got in-memory LLVM modules after we generate and
// optimize all codegen-units for one compilation in rustc. To be compatible
// with the LTO support above we need to serialize the modules plus their
// ThinLTO summary into memory.
//
// This structure is basically an owned version of a serialize module, with
// a ThinLTO summary attached.
struct LLVMRustThinLTOBuffer {
std::string data;
};
extern "C" LLVMRustThinLTOBuffer*
LLVMRustThinLTOBufferCreate(LLVMModuleRef M) {
Handle removal of llvm::make_unique()
2019-12-31 07:08:25 -06:00
auto Ret = std::make_unique<LLVMRustThinLTOBuffer>();
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
{
raw_string_ostream OS(Ret->data);
{
legacy::PassManager PM;
PM.add(createWriteThinLTOBitcodePass(OS));
PM.run(*unwrap(M));
}
}
return Ret.release();
}
extern "C" void
LLVMRustThinLTOBufferFree(LLVMRustThinLTOBuffer *Buffer) {
delete Buffer;
}
extern "C" const void*
LLVMRustThinLTOBufferPtr(const LLVMRustThinLTOBuffer *Buffer) {
return Buffer->data.data();
}
extern "C" size_t
LLVMRustThinLTOBufferLen(const LLVMRustThinLTOBuffer *Buffer) {
return Buffer->data.length();
}
// This is what we used to parse upstream bitcode for actual ThinLTO
// processing. We'll call this once per module optimized through ThinLTO, and
// it'll be called concurrently on many threads.
extern "C" LLVMModuleRef
rustc: Implement incremental "fat" LTO Currently the compiler will produce an error if both incremental compilation and full fat LTO is requested. With recent changes and the advent of incremental ThinLTO, however, all the hard work is already done for us and it's actually not too bad to remove this error! This commit updates the codegen backend to allow incremental full fat LTO. The semantics are that the input modules to LTO are all produce incrementally, but the final LTO step is always done unconditionally regardless of whether the inputs changed or not. The only real incremental win we could have here is if zero of the input modules changed, but that's so rare it's unlikely to be worthwhile to implement such a code path. cc #57968 cc rust-lang/cargo#6643
2019-02-11 09:46:04 -06:00
LLVMRustParseBitcodeForLTO(LLVMContextRef Context,
const char *data,
size_t len,
const char *identifier) {
rustc: Implement ThinLTO This commit is an implementation of LLVM's ThinLTO for consumption in rustc itself. Currently today LTO works by merging all relevant LLVM modules into one and then running optimization passes. "Thin" LTO operates differently by having more sharded work and allowing parallelism opportunities between optimizing codegen units. Further down the road Thin LTO also allows *incremental* LTO which should enable even faster release builds without compromising on the performance we have today. This commit uses a `-Z thinlto` flag to gate whether ThinLTO is enabled. It then also implements two forms of ThinLTO: * In one mode we'll *only* perform ThinLTO over the codegen units produced in a single compilation. That is, we won't load upstream rlibs, but we'll instead just perform ThinLTO amongst all codegen units produced by the compiler for the local crate. This is intended to emulate a desired end point where we have codegen units turned on by default for all crates and ThinLTO allows us to do this without performance loss. * In anther mode, like full LTO today, we'll optimize all upstream dependencies in "thin" mode. Unlike today, however, this LTO step is fully parallelized so should finish much more quickly. There's a good bit of comments about what the implementation is doing and where it came from, but the tl;dr; is that currently most of the support here is copied from upstream LLVM. This code duplication is done for a number of reasons: * Controlling parallelism means we can use the existing jobserver support to avoid overloading machines. * We will likely want a slightly different form of incremental caching which integrates with our own incremental strategy, but this is yet to be determined. * This buys us some flexibility about when/where we run ThinLTO, as well as having it tailored to fit our needs for the time being. * Finally this allows us to reuse some artifacts such as our `TargetMachine` creation, where all our options we used today aren't necessarily supported by upstream LLVM yet. My hope is that we can get some experience with this copy/paste in tree and then eventually upstream some work to LLVM itself to avoid the duplication while still ensuring our needs are met. Otherwise I fear that maintaining these bindings may be quite costly over the years with LLVM updates!
2017-07-23 10:14:38 -05:00
StringRef Data(data, len);
MemoryBufferRef Buffer(Data, identifier);
unwrap(Context)->enableDebugTypeODRUniquing();
Expected<std::unique_ptr<Module>> SrcOrError =
parseBitcodeFile(Buffer, *unwrap(Context));
if (!SrcOrError) {
LLVMRustSetLastError(toString(SrcOrError.takeError()).c_str());
return nullptr;
}
return wrap(std::move(*SrcOrError).release());
}
Store LLVM bitcode in object files, not compressed This commit is an attempted resurrection of #70458 where LLVM bitcode emitted by rustc into rlibs is stored into object file sections rather than in a separate file. The main rationale for doing this is that when rustc emits bitcode it will no longer use a custom compression scheme which makes it both easier to interoperate with existing tools and also cuts down on compile time since this compression isn't happening. The blocker for this in #70458 turned out to be that native linkers didn't handle the new sections well, causing the sections to either trigger bugs in the linker or actually end up in the final linked artifact. This commit attempts to address these issues by ensuring that native linkers ignore the new sections by inserting custom flags with module-level inline assembly. Note that this does not currently change the API of the compiler at all. The pre-existing `-C bitcode-in-rlib` flag is co-opted to indicate whether the bitcode should be present in the object file or not. Finally, note that an important consequence of this commit, which is also one of its primary purposes, is to enable rustc's `-Clto` bitcode loading to load rlibs produced with `-Clinker-plugin-lto`. The goal here is that when you're building with LTO Cargo will tell rustc to skip codegen of all intermediate crates and only generate LLVM IR. Today rustc will generate both object code and LLVM IR, but the object code is later simply thrown away, wastefully.
2020-04-23 13:45:55 -05:00
// Find the bitcode section in the object file data and return it as a slice.
// Fail if the bitcode section is present but empty.
//
// On success, the return value is the pointer to the start of the slice and
// `out_len` is filled with the (non-zero) length. On failure, the return value
// is `nullptr` and `out_len` is set to zero.
extern "C" const char*
LLVMRustGetBitcodeSliceFromObjectData(const char *data,
size_t len,
size_t *out_len) {
*out_len = 0;
StringRef Data(data, len);
MemoryBufferRef Buffer(Data, ""); // The id is unused.
Expected<MemoryBufferRef> BitcodeOrError =
object::IRObjectFile::findBitcodeInMemBuffer(Buffer);
if (!BitcodeOrError) {
LLVMRustSetLastError(toString(BitcodeOrError.takeError()).c_str());
return nullptr;
}
*out_len = BitcodeOrError->getBufferSize();
return BitcodeOrError->getBufferStart();
}
rustc: Work around `DICompileUnit` bugs in LLVM This commit implements a workaround for #46346 which basically just avoids triggering the situation that LLVM's bug https://bugs.llvm.org/show_bug.cgi?id=35562 arises. More details can be found in the code itself but this commit is also intended to ... Closes #46346
2017-12-16 10:20:54 -06:00
// Rewrite all `DICompileUnit` pointers to the `DICompileUnit` specified. See
// the comment in `back/lto.rs` for why this exists.
extern "C" void
Work around invalid DWARF bugs for fat LTO Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2021-09-17 07:21:33 -05:00
LLVMRustLTOGetDICompileUnit(LLVMModuleRef Mod,
rustc: Work around `DICompileUnit` bugs in LLVM This commit implements a workaround for #46346 which basically just avoids triggering the situation that LLVM's bug https://bugs.llvm.org/show_bug.cgi?id=35562 arises. More details can be found in the code itself but this commit is also intended to ... Closes #46346
2017-12-16 10:20:54 -06:00
DICompileUnit **A,
DICompileUnit **B) {
Module *M = unwrap(Mod);
DICompileUnit **Cur = A;
DICompileUnit **Next = B;
for (DICompileUnit *CU : M->debug_compile_units()) {
*Cur = CU;
Cur = Next;
Next = nullptr;
if (Cur == nullptr)
break;
}
}
// Rewrite all `DICompileUnit` pointers to the `DICompileUnit` specified. See
// the comment in `back/lto.rs` for why this exists.
extern "C" void
Work around invalid DWARF bugs for fat LTO Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2021-09-17 07:21:33 -05:00
LLVMRustLTOPatchDICompileUnit(LLVMModuleRef Mod, DICompileUnit *Unit) {
rustc: Work around `DICompileUnit` bugs in LLVM This commit implements a workaround for #46346 which basically just avoids triggering the situation that LLVM's bug https://bugs.llvm.org/show_bug.cgi?id=35562 arises. More details can be found in the code itself but this commit is also intended to ... Closes #46346
2017-12-16 10:20:54 -06:00
Module *M = unwrap(Mod);
// If the original source module didn't have a `DICompileUnit` then try to
// merge all the existing compile units. If there aren't actually any though
// then there's not much for us to do so return.
if (Unit == nullptr) {
for (DICompileUnit *CU : M->debug_compile_units()) {
Unit = CU;
break;
}
if (Unit == nullptr)
return;
}
// Use LLVM's built-in `DebugInfoFinder` to find a bunch of debuginfo and
Prepare for LLVM 11
2020-06-25 20:52:41 -05:00
// process it recursively. Note that we used to specifically iterate over
// instructions to ensure we feed everything into it, but `processModule`
// started doing this the same way in LLVM 7 (commit d769eb36ab2b8).
rustc: Work around `DICompileUnit` bugs in LLVM This commit implements a workaround for #46346 which basically just avoids triggering the situation that LLVM's bug https://bugs.llvm.org/show_bug.cgi?id=35562 arises. More details can be found in the code itself but this commit is also intended to ... Closes #46346
2017-12-16 10:20:54 -06:00
DebugInfoFinder Finder;
Finder.processModule(*M);
// After we've found all our debuginfo, rewrite all subprograms to point to
// the same `DICompileUnit`.
for (auto &F : Finder.subprograms()) {
F->replaceUnit(Unit);
}
// Erase any other references to other `DICompileUnit` instances, the verifier
// will later ensure that we don't actually have any other stale references to
// worry about.
auto *MD = M->getNamedMetadata("llvm.dbg.cu");
MD->clearOperands();
MD->addOperand(Unit);
}
Use llvm::computeLTOCacheKey to determine post-ThinLTO CGU reuse During incremental ThinLTO compilation, we attempt to re-use the optimized (post-ThinLTO) bitcode file for a module if it is 'safe' to do so. Up until now, 'safe' has meant that the set of modules that our current modules imports from/exports to is unchanged from the previous compilation session. See PR #67020 and PR #71131 for more details. However, this turns out be insufficient to guarantee that it's safe to reuse the post-LTO module (i.e. that optimizing the pre-LTO module would produce the same result). When LLVM optimizes a module during ThinLTO, it may look at other information from the 'module index', such as whether a (non-imported!) global variable is used. If this information changes between compilation runs, we may end up re-using an optimized module that (for example) had dead-code elimination run on a function that is now used by another module. Fortunately, LLVM implements its own ThinLTO module cache, which is used when ThinLTO is performed by a linker plugin (e.g. when clang is used to compile a C proect). Using this cache directly would require extensive refactoring of our code - but fortunately for us, LLVM provides a function that does exactly what we need. The function `llvm::computeLTOCacheKey` is used to compute a SHA-1 hash from all data that might influence the result of ThinLTO on a module. In addition to the module imports/exports that we manually track, it also hashes information about global variables (e.g. their liveness) which might be used during optimization. By using this function, we shouldn't have to worry about new LLVM passes breaking our module re-use behavior. In LLVM, the output of this function forms part of the filename used to store the post-ThinLTO module. To keep our current filename structure intact, this PR just writes out the mapping 'CGU name -> Hash' to a file. To determine if a post-LTO module should be reused, we compare hashes from the previous session. This should unblock PR #75199 - by sheer chance, it seems to have hit this issue due to the particular CGU partitioning and optimization decisions that end up getting made.
2020-09-17 16:36:13 -05:00
// Computes the LTO cache key for the provided 'ModId' in the given 'Data',
// storing the result in 'KeyOut'.
// Currently, this cache key is a SHA-1 hash of anything that could affect
// the result of optimizing this module (e.g. module imports, exports, liveness
// of access globals, etc).
// The precise details are determined by LLVM in `computeLTOCacheKey`, which is
// used during the normal linker-plugin incremental thin-LTO process.
extern "C" void
LLVMRustComputeLTOCacheKey(RustStringRef KeyOut, const char *ModId, LLVMRustThinLTOData *Data) {
SmallString<40> Key;
llvm::lto::Config conf;
const auto &ImportList = Data->ImportLists.lookup(ModId);
const auto &ExportList = Data->ExportLists.lookup(ModId);
const auto &ResolvedODR = Data->ResolvedODR.lookup(ModId);
const auto &DefinedGlobals = Data->ModuleToDefinedGVSummaries.lookup(ModId);
std::set<GlobalValue::GUID> CfiFunctionDefs;
std::set<GlobalValue::GUID> CfiFunctionDecls;
// Based on the 'InProcessThinBackend' constructor in LLVM
for (auto &Name : Data->Index.cfiFunctionDefs())
CfiFunctionDefs.insert(
GlobalValue::getGUID(GlobalValue::dropLLVMManglingEscape(Name)));
for (auto &Name : Data->Index.cfiFunctionDecls())
CfiFunctionDecls.insert(
GlobalValue::getGUID(GlobalValue::dropLLVMManglingEscape(Name)));
llvm::computeLTOCacheKey(Key, conf, Data->Index, ModId,
ImportList, ExportList, ResolvedODR, DefinedGlobals, CfiFunctionDefs, CfiFunctionDecls
);
LLVMRustStringWriteImpl(KeyOut, Key.c_str(), Key.size());
}
Reference in New Issue Copy Permalink