This is a follow-up to f189d7a693 and 9d11b089ad. While `-z ignore`
is what needs to be passed to the Solaris linker, because gcc is used as
the default linker, both that form and `-Wl,-z -Wl,ignore` (including
extra double quotes) need to be taken into account, which explains the
more complex regular expression.
Run translation and LLVM in parallel when compiling with multiple CGUs
This is still a work in progress but the bulk of the implementation is done, so I thought it would be good to get it in front of more eyes.
This PR makes the compiler start running LLVM while translation is still in progress, effectively allowing for more parallelism towards the end of the compilation pipeline. It also allows the main thread to switch between either translation or running LLVM, which allows to reduce peak memory usage since not all LLVM module have to be kept in memory until linking. This is especially good for incr. comp. but it works just as well when running with `-Ccodegen-units=N`.
In order to help tuning and debugging the work scheduler, the PR adds the `-Ztrans-time-graph` flag which spits out html files that show how work packages where scheduled:
(red is translation, green is llvm)
One side effect here is that `-Ztime-passes` might show something not quite correct because trans and LLVM are not strictly separated anymore. I plan to have some special handling there that will try to produce useful output.
One open question is how to determine whether the trans-thread should switch to intermediate LLVM processing.
TODO:
- [x] Restore `-Z time-passes` output for LLVM.
- [x] Update documentation, esp. for work package scheduling.
- [x] Tune the scheduling algorithm.
cc @alexcrichton @rust-lang/compiler
The function should accept feature strings that old LLVM might not
support.
Simplify the code using the same approach used by
LLVMRustPrintTargetFeatures.
Dummify the function for non 4.0 LLVM and update the tests accordingly.
Add support for dylibs with Address Sanitizer
Many applications use address sanitizer to assert correct behaviour of their programs. When using Rust with C, it's much more important to assert correct programs with tools like asan/lsan due to the unsafe nature of the access across an ffi boundary. However, previously only rust bin types could use asan. This posed a challenge for existing C applications that link or dlopen .so when the C application is compiled with asan.
This PR enables asan to be linked to the dylib and cdylib crate type. We alter the test to check the proc-macro crate does not work with -Z sanitizer=address. Finally, we add a test that compiles a shared object in rust, then another rust program links it and demonstrates a crash through the call to the library.
This PR is nearly complete, but I do require advice on the change to fix the -lasan that currently exists in the dylib test. This is required because the link statement is not being added correctly to the rustc build when -Z sanitizer=address is added (and I'm not 100% sure why)
Thanks,
Note different versions of same crate when absolute paths of different types match.
The current check to address #22750 only works when the paths of the mismatched types relative to the current crate are equal, but this does not always work if one of the types is only included through an indirect dependency. If reexports are involved, the indirectly included path can e.g. [contain private modules](https://github.com/rust-lang/rust/issues/22750#issuecomment-302755516).
This PR takes care of these cases by also comparing the *absolute* path, which is equal if the type hasn't moved in the module hierarchy between versions. A more coarse check would be to compare only the crate names instead of full paths, but that might lead to too many false positives.
Additionally, I believe it would be helpful to show where the differing crates came from, i.e. the information in `rustc::middle::cstore::CrateSource`, but I'm not sure yet how to nicely display all of that, so I'm leaving it to a future PR.
This PR is an implementation of [RFC 1974] which specifies a new method of
defining a global allocator for a program. This obsoletes the old
`#![allocator]` attribute and also removes support for it.
[RFC 1974]: https://github.com/rust-lang/rfcs/pull/197
The new `#[global_allocator]` attribute solves many issues encountered with the
`#![allocator]` attribute such as composition and restrictions on the crate
graph itself. The compiler now has much more control over the ABI of the
allocator and how it's implemented, allowing much more freedom in terms of how
this feature is implemented.
cc #27389
Prior to this PR, when we aborted because a "critical pass" failed, we
displayed the number of errors from that critical pass. While that's the
number of errors that caused compilation to abort in *that place*,
that's not what people really want to know. Instead, always report the
total number of errors, and don't bother to track the number of errors
from the last pass that failed.
This changes the compiler driver API to handle errors more smoothly,
and therefore is a compiler-api-[breaking-change].
Fixes#42793.
A long time coming this commit removes the `flate` crate in favor of the
`flate2` crate on crates.io. The functionality in `flate2` originally flowered
out of `flate` itself and is additionally the namesake for the crate. This will
leave a gap in the naming (there's not `flate` crate), which will likely cause a
particle collapse of some form somewhere.
save-analysis: remove a lot of stuff
This commits us to the JSON format and the more general def/ref style of output, rather than also supporting different data formats for different data structures. This does not affect the RLS at all, but will break any clients of the CSV form - AFAIK there are none (beyond a few of my own toy projects) - DXR stopped working long ago.
r? @eddyb
Build instruction profiler runtime as part of compiler-rt
r? @alexcrichton
This is #38608 with some fixes.
Still missing:
- [x] testing with profiler enabled on some builders (on which ones? Should I add the option to some of the already existing configurations, or create a new configuration?);
- [x] enabling distribution (on which builders?);
- [x] documentation.
Currently rustdoc will fail if passed `-o foo/doc` if the `foo`
directory doesn't exist.
Also remove unneeded `mkdir` as `create_dir_all` can now handle
concurrent invocations.
Consequently, session creation can no longer initialize LLVM.
The few places that use the compiler without going through
rustc_driver/CompilerCalls thus need to be careful to manually
initialize LLVM (via rustc_trans!) immediately after session
creation.
This means librustc is not rebuilt when LLVM changes.
Move the code for loading metadata from rlibs and dylibs from
rustc_metadata into rustc_trans, and introduce a trait to avoid
introducing a direct dependency on rustc_trans.
This means rustc_metadata is no longer rebuilt when LLVM changes.
As discussed in #41570, UNIX systems often cache the contents of
/etc/resolv.conf, which can cause lookup failures to persist even after
a network connection becomes available. This patch modifies lookup_host
to force a reload of the nameserver entries following a lookup failure.
This is in line with what many C programs already do (see #41570 for
details). On systems with nscd, this should not be necessary, but not
all systems run nscd.
Introduces an std linkage dependency on libresolv on macOS/iOS (which
also makes it necessary to update run-make/tools.mk).
Fixes#41570.
Depends on rust-lang/libc#585.
When -Z profile is passed, the GCDAProfiling LLVM pass is added
to the pipeline, which uses debug information to instrument the IR.
After compiling with -Z profile, the $(OUT_DIR)/$(CRATE_NAME).gcno
file is created, containing initial profiling information.
After running the program built, the $(OUT_DIR)/$(CRATE_NAME).gcda
file is created, containing branch counters.
The created *.gcno and *.gcda files can be processed using
the "llvm-cov gcov" and "lcov" tools. The profiling data LLVM
generates does not faithfully follow the GCC's format for *.gcno
and *.gcda files, and so it will probably not work with other tools
(such as gcov itself) that consume these files.
ASan and TSan are supported on macOS, and this commit enables their
support.
The sanitizers are always built as *.dylib on Apple platforms, so they
cannot be statically linked into the corresponding `rustc_?san.rlib`. The
dylibs are directly copied to `lib/rustlib/x86_64-apple-darwin/lib/`
instead.
Note, although Xcode also ships with their own copies of ASan/TSan dylibs,
we cannot use them due to version mismatch.
There is a caveat: the sanitizer libraries are linked as @rpath, so the
user needs to additionally pass `-C rpath`:
rustc -Z sanitizer=address -C rpath file.rs
^~~~~~~~
Otherwise there will be a runtime error:
dyld: Library not loaded: @rpath/libclang_rt.asan_osx_dynamic.dylib
Referenced from: /path/to/executable
Reason: image not found
Abort trap: 6
The next commit includes a temporary change in compiler to force the linker
to emit a usable @rpath.
This patch adds a `-Z linker-flavor` flag to rustc which can be used to invoke
the linker using a different interface.
For example, by default rustc assumes that all the Linux targets will be linked
using GCC. This makes it impossible to use LLD as a linker using just `-C
linker=ld.lld` because that will invoke LLD with invalid command line
arguments. (e.g. rustc will pass -Wl,--gc-sections to LLD but LLD doesn't
understand that; --gc-sections would be the right argument)
With this patch one can pass `-Z linker-flavor=ld` to rustc to invoke the linker
using a LD-like interface. This way, `rustc -C linker=ld.lld -Z
linker-flavor=ld` will invoke LLD with the right arguments.
`-Z linker-flavor` accepts 4 different arguments: `em` (emcc), `ld`,
`gcc`, `msvc` (link.exe). `em`, `gnu` and `msvc` cover all the existing linker
interfaces. `ld` is a new flavor for interfacing GNU's ld and LLD.
This patch also changes target specifications. `linker-flavor` is now a
mandatory field that specifies the *default* linker flavor that the target will
use. This change also makes the linker interface *explicit*; before, it used to
be derived from other fields like linker-is-gnu, is-like-msvc,
is-like-emscripten, etc.
Another change to target specifications is that the fields `pre-link-args`,
`post-link-args` and `late-link-args` now expect a map from flavor to linker
arguments.
``` diff
- "pre-link-args": ["-Wl,--as-needed", "-Wl,-z,-noexecstack"],
+ "pre-link-args": {
+ "gcc": ["-Wl,--as-needed", "-Wl,-z,-noexecstack"],
+ "ld": ["--as-needed", "-z,-noexecstack"],
+ },
```
[breaking-change] for users of custom targets specifications
#[used] attribute
(For an explanation of what this feature does, read the commit message)
I'd like to propose landing this as an experimental feature (experimental as in:
no clear stabilization path -- like `asm!`, `#[linkage]`) as it's low
maintenance (I think) and relevant to the "Usage in resource-constrained
environments" exploration area.
The main use case I see is running code before `main`. This could be used, for
instance, to cheaply initialize an allocator before `main` where the alternative
is to use `lazy_static` to initialize the allocator on its first use which it's
more expensive (atomics) and doesn't work on ARM Cortex-M0 microcontrollers (no
`AtomicUsize` on that platform)
Here's a `std` example of that:
``` rust
unsafe extern "C" fn before_main_1() {
println!("Hello");
}
unsafe extern "C" fn before_main_2() {
println!("World");
}
#[link_section = ".init_arary"]
#[used]
static INIT_ARRAY: [unsafe extern "C" fn(); 2] = [before_main_1, before_main_2];
fn main() {
println!("Goodbye");
}
```
```
$ rustc -C lto -C opt-level=3 before_main.rs
$ ./before_main
Hello
World
Goodbye
```
In general, this pattern could be used to let *dependencies* run code before
`main` (which sounds like it could go very wrong in some cases). There are
probably other use cases; I hope that the people I have cc-ed can comment on
those.
Note that I'm personally unsure if the above pattern is something we want to
promote / allow and that's why I'm proposing this feature as experimental. If
this leads to more footguns than benefits then we can just axe the feature.
cc @nikomatsakis ^ I know you have some thoughts on having a process for
experimental features though I'm fine with writing an RFC before landing this.
- `dead_code` lint will have to be updated to special case `#[used]` symbols.
- Should we extend `#[used]` to work on non-generic functions?
cc rust-lang/rfcs#1002
cc rust-lang/rfcs#1459
cc @dpc @JinShil
Properly adjust filenames when multiple emissions
Fixes#40993
Should backport just fine to beta but not sure if we want to do this since this is quite old stable regression.
rustc: Stabilize the `#![windows_subsystem]` attribute
This commit stabilizes the `#![windows_subsystem]` attribute which is a
conservative exposure of the `/SUBSYSTEM` linker flag on Widnows platforms. This
is useful for creating applications as well as console programs.
Closes#37499
