Add support for SHA256 source file hashing
Adds support for `-Z src-hash-algorithm sha256`, which became available in LLVM 11.
Using an older version of LLVM will cause an error `invalid checksum kind` if the hash algorithm is set to sha256.
r? `@eddyb`
cc #70401 `@est31`
rustc_llvm: unwrap LLVMMetadataRef before casting
Directly casting the opaque pointer was [reported] to cause an
"incomplete type" error with GCC 9.3:
```
llvm-wrapper/RustWrapper.cpp:939:31: required from here
/usr/include/c++/9.3/type_traits:1301:12: error: invalid use of incomplete type 'struct LLVMOpaqueMetadata'
1301 | struct is_base_of
| ^~~~~~~~~~
In file included from [...]/rust/src/llvm-project/llvm/include/llvm-c/BitReader.h:23,
from llvm-wrapper/LLVMWrapper.h:1,
from llvm-wrapper/RustWrapper.cpp:1:
[...]/rust/src/llvm-project/llvm/include/llvm-c/Types.h:89:16: note: forward declaration of 'struct LLVMOpaqueMetadata'
89 | typedef struct LLVMOpaqueMetadata *LLVMMetadataRef;
| ^~~~~~~~~~~~~~~~~~
```
[reported]: https://zulip-archive.rust-lang.org/182449tcompilerhelp/12215halprustcllvmbuildfail.html#214915124
A simple `unwrap` fixes the issue.
r? `@eddyb`
Directly casting the opaque pointer was [reported] to cause an
"incomplete type" error with GCC 9.3:
```
llvm-wrapper/RustWrapper.cpp:939:31: required from here
/usr/include/c++/9.3/type_traits:1301:12: error: invalid use of incomplete type 'struct LLVMOpaqueMetadata'
1301 | struct is_base_of
| ^~~~~~~~~~
In file included from [...]/rust/src/llvm-project/llvm/include/llvm-c/BitReader.h:23,
from llvm-wrapper/LLVMWrapper.h:1,
from llvm-wrapper/RustWrapper.cpp:1:
[...]/rust/src/llvm-project/llvm/include/llvm-c/Types.h:89:16: note: forward declaration of 'struct LLVMOpaqueMetadata'
89 | typedef struct LLVMOpaqueMetadata *LLVMMetadataRef;
| ^~~~~~~~~~~~~~~~~~
```
[reported]: https://zulip-archive.rust-lang.org/182449tcompilerhelp/12215halprustcllvmbuildfail.html#214915124
A simple `unwrap` fixes the issue.
Use unwrapDIPtr because the Scope may be null.
I ran into an assertion when using debug information on Windows with LLVM assertions enabled.
It seems like we are using unwrap here (which in turn calls isa and requires the pointer to be non-null) but we expect the value to be null because that is what we are passing from rustc.
This change uses unwrapDIPtr which explicitly allows nullptr.
The FFI prototype for this method on the rust side has the `LLVMMetadataRef` parameter as `Scope: Option<&'a DIScope>`, and we always pass `None` when `msvc_like_names` is true.
Use llvm::computeLTOCacheKey to determine post-ThinLTO CGU reuse
During incremental ThinLTO compilation, we attempt to re-use the
optimized (post-ThinLTO) bitcode file for a module if it is 'safe' to do
so.
Up until now, 'safe' has meant that the set of modules that our current
modules imports from/exports to is unchanged from the previous
compilation session. See PR #67020 and PR #71131 for more details.
However, this turns out be insufficient to guarantee that it's safe
to reuse the post-LTO module (i.e. that optimizing the pre-LTO module
would produce the same result). When LLVM optimizes a module during
ThinLTO, it may look at other information from the 'module index', such
as whether a (non-imported!) global variable is used. If this
information changes between compilation runs, we may end up re-using an
optimized module that (for example) had dead-code elimination run on a
function that is now used by another module.
Fortunately, LLVM implements its own ThinLTO module cache, which is used
when ThinLTO is performed by a linker plugin (e.g. when clang is used to
compile a C proect). Using this cache directly would require extensive
refactoring of our code - but fortunately for us, LLVM provides a
function that does exactly what we need.
The function `llvm::computeLTOCacheKey` is used to compute a SHA-1 hash
from all data that might influence the result of ThinLTO on a module.
In addition to the module imports/exports that we manually track, it
also hashes information about global variables (e.g. their liveness)
which might be used during optimization. By using this function, we
shouldn't have to worry about new LLVM passes breaking our module re-use
behavior.
In LLVM, the output of this function forms part of the filename used to
store the post-ThinLTO module. To keep our current filename structure
intact, this PR just writes out the mapping 'CGU name -> Hash' to a
file. To determine if a post-LTO module should be reused, we compare
hashes from the previous session.
This should unblock PR #75199 - by sheer chance, it seems to have hit
this issue due to the particular CGU partitioning and optimization
decisions that end up getting made.
Secure entry functions do not support if arguments are passed on the
stack. An "unsupported" diagnostic will be emitted by LLVM if that is
the case.
This commits adds support in Rust for that diagnostic so that an error
will be output if that is the case!
Signed-off-by: Hugues de Valon <hugues.devalon@arm.com>
During incremental ThinLTO compilation, we attempt to re-use the
optimized (post-ThinLTO) bitcode file for a module if it is 'safe' to do
so.
Up until now, 'safe' has meant that the set of modules that our current
modules imports from/exports to is unchanged from the previous
compilation session. See PR #67020 and PR #71131 for more details.
However, this turns out be insufficient to guarantee that it's safe
to reuse the post-LTO module (i.e. that optimizing the pre-LTO module
would produce the same result). When LLVM optimizes a module during
ThinLTO, it may look at other information from the 'module index', such
as whether a (non-imported!) global variable is used. If this
information changes between compilation runs, we may end up re-using an
optimized module that (for example) had dead-code elimination run on a
function that is now used by another module.
Fortunately, LLVM implements its own ThinLTO module cache, which is used
when ThinLTO is performed by a linker plugin (e.g. when clang is used to
compile a C proect). Using this cache directly would require extensive
refactoring of our code - but fortunately for us, LLVM provides a
function that does exactly what we need.
The function `llvm::computeLTOCacheKey` is used to compute a SHA-1 hash
from all data that might influence the result of ThinLTO on a module.
In addition to the module imports/exports that we manually track, it
also hashes information about global variables (e.g. their liveness)
which might be used during optimization. By using this function, we
shouldn't have to worry about new LLVM passes breaking our module re-use
behavior.
In LLVM, the output of this function forms part of the filename used to
store the post-ThinLTO module. To keep our current filename structure
intact, this PR just writes out the mapping 'CGU name -> Hash' to a
file. To determine if a post-LTO module should be reused, we compare
hashes from the previous session.
This should unblock PR #75199 - by sheer chance, it seems to have hit
this issue due to the particular CGU partitioning and optimization
decisions that end up getting made.
