Commit Graph

38 Commits

Author SHA1 Message Date
bors
4dbc89de3f Auto merge of - cuviper:hashbrown-0.12, r=Mark-Simulacrum
Upgrade indexmap and thorin-dwp to use hashbrown 0.12

This removes the last dependencies on hashbrown 0.11.

This also upgrades to hashbrown 0.12.3 to fix a double-free ().
2022-07-24 04:03:29 +00:00
Josh Stone
daa7c14a44 Upgrade indexmap and thorin-dwp to use hashbrown 0.12
This removes the last dependencies on hashbrown 0.11.
2022-07-17 07:05:58 -07:00
5225225
27412d1e3e Use constant eval to do strict validity checks 2022-07-14 22:55:17 +01:00
Nicholas Nethercote
7c40661ddb Update smallvec to 1.8.1.
This pulls in https://github.com/servo/rust-smallvec/pull/282, which
gives some small wins for rustc.
2022-06-27 08:48:55 +10:00
Yuki Okushi
0af99c99c4
Rollup merge of - mkroening:object-osabi, r=petrochenkov
Session object: Set OS/ABI

Closes https://github.com/rust-lang/rust/issues/97535.

This depends on
* https://github.com/gimli-rs/object/pull/438

This adapts LLVM's behavior of [`MCELFObjectTargetWriter::getOSABI`](8c8a2679a2/llvm/include/llvm/MC/MCELFObjectWriter.h (L72-L86)).
2022-06-24 16:43:43 +09:00
Martin Kröning
21625e5730 Session object: Set OS/ABI
This adapts LLVM's behavior of MCELFObjectTargetWriter::getOSABI [1].

[1]: 8c8a2679a2/llvm/include/llvm/MC/MCELFObjectWriter.h (L72-L86)
2022-06-23 11:20:52 +02:00
bjorn3
5cc3593c17 Fix emscripten linker invocation 2022-06-03 17:02:14 +00:00
Ayrton
3d5b1eeb75 Fix e_flags for 32-bit MIPS targets in generated object file
In  the compiler started generating a temporary symbols.o which is added
to the linker invocation. This object file has an `e_flags` which may be invalid
for 32-bit MIPS targets. Even though symbols.o doesn't contain code, linking
    with [lld fails](https://github.com/llvm/llvm-project/blob/main/lld/ELF/Arch/MipsArchTree.cpp#L79) with
```
rust-lld: error: foo-cgu.0.rcgu.o: ABI 'o32' is incompatible with target ABI 'n64'
```
because it omits the ABI bits (EF_MIPS_ABI_O32) so lld assumes it's using the
N64 ABI. This breaks linking on nightly for the out-of-tree [psx
target](https://github.com/ayrtonm/psx-sdk-rs/issues/9), the builtin
mipsel-sony-psp target (cc @overdrivenpotato) and any other 32-bit MIPS
target using lld.

This PR sets the ABI in `e_flags` to O32 since that's the only ABI for 32-bit
MIPS that LLVM supports. It also sets other `e_flags` bits based on the target.
I had to bump the object crate version since some of these constants were [added
recently](https://github.com/gimli-rs/object/pull/433). I'm not sure if this
PR needs a test, but I can confirm that it fixes the linking issue on both
targets I mentioned.
2022-05-10 22:48:19 -04:00
pierwill
f684acdd7e Update itertools
Update to 0.10.1
2022-03-04 11:54:28 -06:00
Rémy Rakic
820fd05e29 Update thorin-dwp to deduplicate object 2022-01-20 15:09:05 +01:00
Rémy Rakic
5928056af2 Update itertools to deduplicate it 2022-01-14 12:33:54 +01:00
Lain Yang
9a337b6fe0 update Cargo.lock and gimli-rs/object for rustc_codegen_ssa 2022-01-07 13:33:20 +08:00
Lain Yang
579e8bce7d Make rlib metadata strip works with MIPSr6 architecture
Because MIPSr6 has many differences with previous MIPSr2 arch, the previous rlib metadata stripping code in `rustc_codegen_ssa` is only for MIPSr2/r3/r5 (which share the same elf e_flags).
This commit fixed this problem. It makes `rustc_codegen_ssa` happy when compiling rustc for MIPSr6 target or hosts.
2022-01-07 10:47:27 +08:00
David Wood
2dc1a8a779 cg: use thorin instead of llvm-dwp
`thorin` is a Rust implementation of a DWARF packaging utility that
supports reading DWARF objects from archive files (i.e. rlibs) and
therefore is better suited for integration into rustc.

Signed-off-by: David Wood <david.wood@huawei.com>
2022-01-06 09:32:42 +00:00
Nikita Popov
9488cacc52 Use object crate for .rustc metadata generation
We already use the object crate for generating uncompressed .rmeta
metadata object files. This switches the generation of compressed
.rustc object files to use the object crate as well. These have
slightly different requirements in that .rmeta should be completely
excluded from any final compilation artifacts, while .rustc should
be part of shared objects, but not loaded into memory.

The primary motivation for this change is : In LLVM 14, the
current way of setting section flags (and in particular, preventing
the setting of SHF_ALLOC) will no longer work. There are other ways
we could work around this, but switching to the object crate seems
like the most elegant, as we already use it for .rmeta, and as it
makes this independent of the codegen backend. In particular, we
don't need separate handling in codegen_llvm and codegen_gcc.
codegen_cranelift should be able to reuse the implementation as
well, though I have omitted that here, as it is not based on
codegen_ssa.

This change mostly extracts the existing code for .rmeta handling
to allow using it for .rustc as well, and adjust the codegen
infrastructure to handle the metadata object file separately: We
no longer create a backend-specific module for it, and directly
produce the compiled module instead.

This does not fix  by itself yet, as .llvmbc will need to be
handled separately.
2021-12-07 09:39:05 +01:00
Camille GILLOT
02025d86ac Remove re-export. 2021-10-03 16:08:54 +02:00
Camille GILLOT
b244b98e7c Move EncodedMetadata to rustc_metadata. 2021-09-30 19:41:32 +02:00
Mark Rousskov
c746be2219 Migrate to 2021 2021-09-20 22:21:42 -04:00
Mark Rousskov
4c7c97a208 Fix loading large rlibs
Bumps object crate to permit parsing archives with 64-bit table entries. These
are primarily encountered when there's more than 4GB of archive data.
2021-08-30 16:22:53 -04:00
Alex Crichton
4a3e73643a Update the backtrace crate in libstd
This commit updates the backtrace crate in libstd now that dependencies
have been updated to use `memchr` from the standard library as well.
This is mostly just making sure deps are up-to-date and have all the
latest-and-greatest fixes and such.

Closes 
2021-08-19 07:31:49 -07:00
bors
d4ad1cfc63 Auto merge of - HackAttack:expand-unknown-option-message, r=wesleywiser
Allow more "unknown argument" strings from linker

Some toolchains emit slightly different errors, e.g.

    ppc-vle-gcc: error: unrecognized option '-no-pie'
2021-08-05 00:11:05 +00:00
Michael Hackner
32992357eb Allow more "unknown argument" strings from linker
Some toolchains emit slightly different errors, e.g.

    ppc-vle-gcc: error: unrecognized option '-no-pie'
2021-07-31 09:35:55 -07:00
Jade
3cf820e17d rfc3052: Remove authors field from Cargo manifests
Since RFC 3052 soft deprecated the authors field anyway, hiding it from
crates.io, docs.rs, and making Cargo not add it by default, and it is
not generally up to date/useful information, we should remove it from
crates in this repo.
2021-07-29 14:56:05 -07:00
Michael Woerister
e6e1e095ff [debuginfo] Emit associated type bindings in trait object type names. 2021-07-15 10:40:25 +02:00
Yuki Okushi
e457c2739b
Upgrade cc crate to 1.0.69 2021-07-13 17:58:50 +09:00
Alex Crichton
0e0338744d rustc: Store metadata-in-rlibs in object files
This commit updates how rustc compiler metadata is stored in rlibs.
Previously metadata was stored as a raw file that has the same format as
`--emit metadata`. After this commit, however, the metadata is encoded
into a small object file which has one section which is the contents of
the metadata.

The motivation for this commit is to fix a common case where 
arises. The problem is that when rustc crates a `dylib` crate type it
needs to include entire rlib files into the dylib, so it passes
`--whole-archive` (or the equivalent) to the linker. The problem with
this, though, is that the linker will attempt to read all files in the
archive. If the metadata file were left as-is (today) then the linker
would generate an error saying it can't read the file. The previous
solution was to alter the rlib just before linking, creating a new
archive in a temporary directory which has the metadata file removed.

This problem from before this commit is now removed if the metadata file
is stored in an object file that the linker can read. The only caveat we
have to take care of is to ensure that the linker never actually
includes the contents of the object file into the final output. We apply
similar tricks as the `.llvmbc` bytecode sections to do this.

This involved changing the metadata loading code a bit, namely updating
some of the LLVM C APIs used to use non-deprecated ones and fiddling
with the lifetimes a bit to get everything to work out. Otherwise though
this isn't intended to be a functional change really, only that metadata
is stored differently in archives now.

This should end up fixing  because by default dylibs will no
longer have their rlib dependencies "altered" meaning that
split-debuginfo will continue to have valid paths pointing at the
original rlibs. (note that we still "alter" rlibs if LTO is enabled to
remove Rust object files and we also "alter" for the #[link(cfg)]
feature, but that's rarely used).

Closes 
2021-06-04 10:05:20 -07:00
Chris Denton
e238ee31d4
Update cc
Recent commits to cc have helped to address  and 
2021-05-24 23:34:12 +01:00
bors
75da570d78 Auto merge of - bjorn3:shared_metadata_reader, r=nagisa
Use the object crate for metadata reading

This allows sharing the metadata reader between cg_llvm, cg_clif and other codegen backends.

This is not currently useful for rlib reading with cg_spirv ([rust-gpu](https://github.com/EmbarkStudios/rust-gpu/)) as it uses tar rather than ar as .rlib format, but it is useful for dylib reading required for loading proc macros. (cc `@eddyb)`

The object crate is already trusted as dependency of libstd through backtrace. As far as I know it supports reading all object file formats used by targets for which we support rust dylibs with crate metadata, but I am not certain. If this happens to not be the case, I could keep using LLVM for reading dylib metadata.

Marked as WIP for a perf run and as it is based on .
2021-05-14 12:58:58 +00:00
Mateusz Mikuła
b04fd78d66 update cc crate
To pull in this fix: 801a87bf2f
2021-05-12 00:55:03 +02:00
bjorn3
802fe1756b Disable wasm feature of object in cg_ssa
The version 1 resolver unifies enabled features across the
whole workspace. This includes libstd which isn't allowed
to depend on wasmparser.
2021-05-07 18:57:25 +02:00
bjorn3
267d55d44a Use the object crate for metadata reading 2021-05-07 18:48:58 +02:00
Jubilee Young
b2c1dbbd33 Use tempfile 2021-04-23 15:33:57 -07:00
The8472
7f45cdb090 bump jobserver dependency
the newest jobserver version should slightly reduce context switches
in highly parallel build environments on linux kernels >= 5.6
2021-04-21 22:02:54 +02:00
bjorn3
8331dbe6d0 Add an Mmap wrapper to rustc_data_structures
This wrapper implements StableAddress and falls back to directly reading
the file on wasm32
2021-03-30 18:57:03 +02:00
Camille GILLOT
458d044c5b Upgrade memmap to memmap2 in other crates. 2021-03-18 18:36:55 +01:00
Tyson Nottingham
5f243d3c2b rustc_codegen_ssa: tune codegen according to available concurrency
This change tunes ahead-of-time codegening according to the amount of
concurrency available, rather than according to the number of CPUs on
the system. This can lower memory usage by reducing the number of
compiled LLVM modules in memory at once, particularly across several
rustc instances.

Previously, each rustc instance would assume that it should codegen
ahead of time to meet the demand of number-of-CPUs workers. But often, a
rustc instance doesn't have nearly that much concurrency available to
it, because the concurrency availability is split, via the jobserver,
across all active rustc instances spawned by the driving cargo process,
and is further limited by the `-j` flag argument. Therefore, each rustc
might have had several times the number of LLVM modules in memory than
it really needed to meet demand. If the modules were large, the effect
on memory usage would be noticeable.

With this change, the required amount of ahead-of-time codegen scales up
with the actual number of workers running within a rustc instance. Note
that the number of workers running can be less than the actual
concurrency available to a rustc instance. However, if more concurrency
is actually available, workers are spun up quickly as job tokens are
acquired, and the ahead-of-time codegen scales up quickly as well.
2021-02-15 13:02:49 -08:00
Tyson Nottingham
29711d8c96 rustc_codegen_ssa: tune codegen scheduling to reduce memory usage
For better throughput during parallel processing by LLVM, we used to sort
CGUs largest to smallest. This would lead to better thread utilization
by, for example, preventing a large CGU from being processed last and
having only one LLVM thread working while the rest remained idle.

However, this strategy would lead to high memory usage, as it meant the
LLVM-IR for all of the largest CGUs would be resident in memory at once.

Instead, we can compromise by ordering CGUs such that the largest and
smallest are first, second largest and smallest are next, etc. If there
are large size variations, this can reduce memory usage significantly.
2021-02-03 18:55:05 -08:00
mark
9e5f7d5631 mv compiler to compiler/ 2020-08-30 18:45:07 +03:00