86 lines
3.3 KiB
Rust
Raw Normal View History

//! Reading of the rustc metadata for rlibs and dylibs
use std::fs::File;
use std::path::Path;
rustc: Store metadata-in-rlibs in object files This commit updates how rustc compiler metadata is stored in rlibs. Previously metadata was stored as a raw file that has the same format as `--emit metadata`. After this commit, however, the metadata is encoded into a small object file which has one section which is the contents of the metadata. The motivation for this commit is to fix a common case where #83730 arises. The problem is that when rustc crates a `dylib` crate type it needs to include entire rlib files into the dylib, so it passes `--whole-archive` (or the equivalent) to the linker. The problem with this, though, is that the linker will attempt to read all files in the archive. If the metadata file were left as-is (today) then the linker would generate an error saying it can't read the file. The previous solution was to alter the rlib just before linking, creating a new archive in a temporary directory which has the metadata file removed. This problem from before this commit is now removed if the metadata file is stored in an object file that the linker can read. The only caveat we have to take care of is to ensure that the linker never actually includes the contents of the object file into the final output. We apply similar tricks as the `.llvmbc` bytecode sections to do this. This involved changing the metadata loading code a bit, namely updating some of the LLVM C APIs used to use non-deprecated ones and fiddling with the lifetimes a bit to get everything to work out. Otherwise though this isn't intended to be a functional change really, only that metadata is stored differently in archives now. This should end up fixing #83730 because by default dylibs will no longer have their rlib dependencies "altered" meaning that split-debuginfo will continue to have valid paths pointing at the original rlibs. (note that we still "alter" rlibs if LTO is enabled to remove Rust object files and we also "alter" for the #[link(cfg)] feature, but that's rarely used). Closes #83730
2021-04-22 11:53:33 -07:00
use object::{Object, ObjectSection};
use rustc_data_structures::memmap::Mmap;
use rustc_data_structures::owning_ref::OwningRef;
use rustc_data_structures::rustc_erase_owner;
use rustc_data_structures::sync::MetadataRef;
use rustc_session::cstore::MetadataLoader;
use rustc_target::spec::Target;
use crate::METADATA_FILENAME;
/// The default metadata loader. This is used by cg_llvm and cg_clif.
///
/// # Metadata location
///
/// <dl>
/// <dt>rlib</dt>
/// <dd>The metadata can be found in the `lib.rmeta` file inside of the ar archive.</dd>
/// <dt>dylib</dt>
/// <dd>The metadata can be found in the `.rustc` section of the shared library.</dd>
/// </dl>
pub struct DefaultMetadataLoader;
fn load_metadata_with(
path: &Path,
f: impl for<'a> FnOnce(&'a [u8]) -> Result<&'a [u8], String>,
) -> Result<MetadataRef, String> {
2021-05-10 09:49:42 +02:00
let file =
File::open(path).map_err(|e| format!("failed to open file '{}': {}", path.display(), e))?;
let data = unsafe { Mmap::map(file) }
.map_err(|e| format!("failed to mmap file '{}': {}", path.display(), e))?;
let metadata = OwningRef::new(data).try_map(f)?;
return Ok(rustc_erase_owner!(metadata.map_owner_box()));
}
impl MetadataLoader for DefaultMetadataLoader {
fn get_rlib_metadata(&self, _target: &Target, path: &Path) -> Result<MetadataRef, String> {
load_metadata_with(path, |data| {
let archive = object::read::archive::ArchiveFile::parse(&*data)
2021-05-10 09:49:42 +02:00
.map_err(|e| format!("failed to parse rlib '{}': {}", path.display(), e))?;
for entry_result in archive.members() {
2021-05-10 09:49:42 +02:00
let entry = entry_result
.map_err(|e| format!("failed to parse rlib '{}': {}", path.display(), e))?;
if entry.name() == METADATA_FILENAME.as_bytes() {
rustc: Store metadata-in-rlibs in object files This commit updates how rustc compiler metadata is stored in rlibs. Previously metadata was stored as a raw file that has the same format as `--emit metadata`. After this commit, however, the metadata is encoded into a small object file which has one section which is the contents of the metadata. The motivation for this commit is to fix a common case where #83730 arises. The problem is that when rustc crates a `dylib` crate type it needs to include entire rlib files into the dylib, so it passes `--whole-archive` (or the equivalent) to the linker. The problem with this, though, is that the linker will attempt to read all files in the archive. If the metadata file were left as-is (today) then the linker would generate an error saying it can't read the file. The previous solution was to alter the rlib just before linking, creating a new archive in a temporary directory which has the metadata file removed. This problem from before this commit is now removed if the metadata file is stored in an object file that the linker can read. The only caveat we have to take care of is to ensure that the linker never actually includes the contents of the object file into the final output. We apply similar tricks as the `.llvmbc` bytecode sections to do this. This involved changing the metadata loading code a bit, namely updating some of the LLVM C APIs used to use non-deprecated ones and fiddling with the lifetimes a bit to get everything to work out. Otherwise though this isn't intended to be a functional change really, only that metadata is stored differently in archives now. This should end up fixing #83730 because by default dylibs will no longer have their rlib dependencies "altered" meaning that split-debuginfo will continue to have valid paths pointing at the original rlibs. (note that we still "alter" rlibs if LTO is enabled to remove Rust object files and we also "alter" for the #[link(cfg)] feature, but that's rarely used). Closes #83730
2021-04-22 11:53:33 -07:00
let data = entry
.data(data)
.map_err(|e| format!("failed to parse rlib '{}': {}", path.display(), e))?;
return search_for_metadata(path, data, ".rmeta");
}
}
2021-05-10 09:49:42 +02:00
Err(format!("metadata not found in rlib '{}'", path.display()))
})
}
fn get_dylib_metadata(&self, _target: &Target, path: &Path) -> Result<MetadataRef, String> {
rustc: Store metadata-in-rlibs in object files This commit updates how rustc compiler metadata is stored in rlibs. Previously metadata was stored as a raw file that has the same format as `--emit metadata`. After this commit, however, the metadata is encoded into a small object file which has one section which is the contents of the metadata. The motivation for this commit is to fix a common case where #83730 arises. The problem is that when rustc crates a `dylib` crate type it needs to include entire rlib files into the dylib, so it passes `--whole-archive` (or the equivalent) to the linker. The problem with this, though, is that the linker will attempt to read all files in the archive. If the metadata file were left as-is (today) then the linker would generate an error saying it can't read the file. The previous solution was to alter the rlib just before linking, creating a new archive in a temporary directory which has the metadata file removed. This problem from before this commit is now removed if the metadata file is stored in an object file that the linker can read. The only caveat we have to take care of is to ensure that the linker never actually includes the contents of the object file into the final output. We apply similar tricks as the `.llvmbc` bytecode sections to do this. This involved changing the metadata loading code a bit, namely updating some of the LLVM C APIs used to use non-deprecated ones and fiddling with the lifetimes a bit to get everything to work out. Otherwise though this isn't intended to be a functional change really, only that metadata is stored differently in archives now. This should end up fixing #83730 because by default dylibs will no longer have their rlib dependencies "altered" meaning that split-debuginfo will continue to have valid paths pointing at the original rlibs. (note that we still "alter" rlibs if LTO is enabled to remove Rust object files and we also "alter" for the #[link(cfg)] feature, but that's rarely used). Closes #83730
2021-04-22 11:53:33 -07:00
load_metadata_with(path, |data| search_for_metadata(path, data, ".rustc"))
}
}
rustc: Store metadata-in-rlibs in object files This commit updates how rustc compiler metadata is stored in rlibs. Previously metadata was stored as a raw file that has the same format as `--emit metadata`. After this commit, however, the metadata is encoded into a small object file which has one section which is the contents of the metadata. The motivation for this commit is to fix a common case where #83730 arises. The problem is that when rustc crates a `dylib` crate type it needs to include entire rlib files into the dylib, so it passes `--whole-archive` (or the equivalent) to the linker. The problem with this, though, is that the linker will attempt to read all files in the archive. If the metadata file were left as-is (today) then the linker would generate an error saying it can't read the file. The previous solution was to alter the rlib just before linking, creating a new archive in a temporary directory which has the metadata file removed. This problem from before this commit is now removed if the metadata file is stored in an object file that the linker can read. The only caveat we have to take care of is to ensure that the linker never actually includes the contents of the object file into the final output. We apply similar tricks as the `.llvmbc` bytecode sections to do this. This involved changing the metadata loading code a bit, namely updating some of the LLVM C APIs used to use non-deprecated ones and fiddling with the lifetimes a bit to get everything to work out. Otherwise though this isn't intended to be a functional change really, only that metadata is stored differently in archives now. This should end up fixing #83730 because by default dylibs will no longer have their rlib dependencies "altered" meaning that split-debuginfo will continue to have valid paths pointing at the original rlibs. (note that we still "alter" rlibs if LTO is enabled to remove Rust object files and we also "alter" for the #[link(cfg)] feature, but that's rarely used). Closes #83730
2021-04-22 11:53:33 -07:00
fn search_for_metadata<'a>(
path: &Path,
bytes: &'a [u8],
section: &str,
) -> Result<&'a [u8], String> {
let file = match object::File::parse(bytes) {
Ok(f) => f,
// The parse above could fail for odd reasons like corruption, but for
// now we just interpret it as this target doesn't support metadata
// emission in object files so the entire byte slice itself is probably
// a metadata file. Ideally though if necessary we could at least check
// the prefix of bytes to see if it's an actual metadata object and if
// not forward the error along here.
Err(_) => return Ok(bytes),
};
file.section_by_name(section)
.ok_or_else(|| format!("no `{}` section in '{}'", section, path.display()))?
.data()
.map_err(|e| format!("failed to read {} section in '{}': {}", section, path.display(), e))
}