This commit adds LLVM Kernel Control Flow Integrity (KCFI) support to
the Rust compiler. It initially provides forward-edge control flow
protection for operating systems kernels for Rust-compiled code only by
aggregating function pointers in groups identified by their return and
parameter types. (See llvm/llvm-project@cff5bef.)
Forward-edge control flow protection for C or C++ and Rust -compiled
code "mixed binaries" (i.e., for when C or C++ and Rust -compiled code
share the same virtual address space) will be provided in later work as
part of this project by identifying C char and integer type uses at the
time types are encoded (see Type metadata in the design document in the
tracking issue #89653).
LLVM KCFI can be enabled with -Zsanitizer=kcfi.
Co-authored-by: bjorn3 <17426603+bjorn3@users.noreply.github.com>
Detect long types in E0308 and write them to disk
On type error with long types, print an abridged type and write the full type to disk.
Print the widest possible short type while still fitting in the terminal.
Add -Z maximal-hir-to-mir-coverage flag
This PR adds a new unstable flag `-Z maximal-hir-to-mir-coverage` that changes the behavior of `maybe_lint_level_root_bounded`, pursuant to [a discussion on Zulip](https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/Mapping.20MIR.20to.20HIR). When enabled, this function will not search upwards for a lint root, but rather immediately return the provided HIR node ID. This change increases the granularity of the mapping between MIR locations and HIR nodes inside the `SourceScopeLocalData` data structures. This increase in granularity is useful for rustc consumers like [Flowistry](https://github.com/willcrichton/flowistry) that rely on getting source-mapping information about the MIR CFG that is as precise as possible.
A test `maximal_mir_to_hir_coverage.rs` has been added to verify that this flag does not break anything.
r? `@cjgillot`
cc `@gavinleroy`
Add StableOrd trait as proposed in MCP 533.
The `StableOrd` trait can be used to mark types as having a stable sort order across compilation sessions. Collections that sort their items in a stable way can safely implement HashStable by hashing items in sort order.
See https://github.com/rust-lang/compiler-team/issues/533 for more information.
Remove useless borrows and derefs
They are nothing more than noise.
<sub>These are not all of them, but my clippy started crashing (stack overflow), so rip :(</sub>
The StableOrd trait can be used to mark types as having a stable
sort order across compilation sessions. Collections that sort their
items in a stable way can safely implement HashStable by
hashing items in sort order.
On type error with long types, print an abridged type and write the full
type to disk.
Print the widest possible short type while still fitting in the
terminal.
make `error_reported` check for delayed bugs
Fixes#104768
`error_reported()` was only checking if there were errors emitted, not for `delay_bug`s which can also be a source of `ErrorGuaranteed`. I assume the same is true of `lint_err_count` but i dont know
Use `as_deref` in compiler (but only where it makes sense)
This simplifies some code :3
(there are some changes that are not exacly `as_deref`, but more like "clever `Option`/`Result` method use")
Instead of `ast::Lit`.
Literal lowering now happens at two different times. Expression literals
are lowered when HIR is crated. Attribute literals are lowered during
parsing.
This commit changes the language very slightly. Some programs that used
to not compile now will compile. This is because some invalid literals
that are removed by `cfg` or attribute macros will no longer trigger
errors. See this comment for more details:
https://github.com/rust-lang/rust/pull/102944#issuecomment-1277476773
improve `filesearch::get_or_default_sysroot`
`fn get_or_default_sysroot` is now improved and used in `miri` and `clippy`, and tests are still passing as they should. So we no longer need to implement custom workarounds/hacks to find sysroot in tools like miri/clippy.
Resolves https://github.com/rust-lang/rust/issues/98832
re-opened from #103581
Allow use of `-Clto=thin` with `-Ccodegen-units=1` in general
The current logic to ignore ThinLTO when `-Ccodegen-units=1` makes sense for local ThinLTO but even in this scenario, a user may still want (non-local) ThinLTO for the purpose of optimizing dependencies into the final crate which is being compiled with 1 CGU.
The previous behavior was even more confusing because if you were generating a binary (`--emit=link`), then you would get ThinLTO but if you asked for LLVM IR or bytecode, then it would silently change to using regular LTO.
With this change, we only override the defaults for local ThinLTO if you ask for a single output such as LLVM IR or bytecode and in all other cases honor the requested LTO setting.
r? `@michaelwoerister`
Print valid `--print` requests if request is invalid
When someone makes a typo, it can be useful to see the valid options. This is also useful if someone wants to find out about all the options.
Track where diagnostics were created.
This implements the `-Ztrack-diagnostics` flag, which uses `#[track_caller]` to track where diagnostics are created. It is meant as a debugging tool much like `-Ztreat-err-as-bug`.
For example, the following code...
```rust
struct A;
struct B;
fn main(){
let _: A = B;
}
```
...now emits the following error message:
```
error[E0308]: mismatched types
--> src\main.rs:5:16
|
5 | let _: A = B;
| - ^ expected struct `A`, found struct `B`
| |
| expected due to this
-Ztrack-diagnostics: created at compiler\rustc_infer\src\infer\error_reporting\mod.rs:2275:31
```
The current logic to ignore ThinLTO when `-Ccodegen-units=1` makes sense
for local ThinLTO but even in this scenario, a user may still want
(non-local) ThinLTO for the purpose of optimizing dependencies into the
final crate which is being compiled with 1 CGU.
The previous behavior was even more confusing because if you were
generating a binary (`--emit=link`), then you would get ThinLTO but if
you asked for LLVM IR or bytecode, then it would silently change to
using regular LTO.
With this change, we only override the defaults for local ThinLTO if you
ask for a single output such as LLVM IR or bytecode and in all other
cases honor the requested LTO setting.
Get rid of native_library projection queries
They don't seem particularly useful as I don't expect native libraries to change frequently.
Maybe they do provide significant value of keeping incremental compilation green though, I'm not sure.
Change process spawning to inherit the parent's signal mask by default
Previously, the signal mask was always reset when a child process is
started. This breaks tools like `nohup` which expect `SIGHUP` to be
blocked for all transitive processes.
With this change, the default behavior changes to inherit the signal mask.
This also changes the signal disposition for `SIGPIPE` to only be changed if the `#[unix_sigpipe]` attribute isn't set.
translation: doc comments with derives, subdiagnostic-less enum variants, more derive use
- Adds support for `doc` attributes in the diagnostic derives so that documentation comments don't result in the derive failing.
- Adds support for enum variants in the subdiagnostic derive to not actually correspond to an addition to a diagnostic.
- Made use of the derive in more places in the `rustc_ast_lowering`, `rustc_ast_passes`, `rustc_lint`, `rustc_session`, `rustc_infer` - taking advantage of recent additions like eager subdiagnostics, multispan suggestions, etc.
cc #100717
Previously, the signal mask is always reset when a child process is
started. This breaks tools like `nohup` which expect `SIGHUP` to be
blocked.
With this change, the default behavior changes to inherit the signal mask.
This also changes the signal disposition for `SIGPIPE` to only be
changed if the `#[unix_sigpipe]` attribute isn't set.
More dupe word typos
I only picked those changes (from the regex search) that I am pretty certain doesn't change meaning and is just a typo fix. Do correct me if any fix is undesirable and I can revert those. Thanks.
Add `Noted` marker struct that implements `EmissionGuarantee` so that
`emit_note` and `create_note` can be implemented for struct diagnostics.
Signed-off-by: David Wood <david.wood@huawei.com>
The compiler currently has `-Ztime` and `-Ztime-passes`. I've used
`-Ztime-passes` for years but only recently learned about `-Ztime`.
What's the difference? Let's look at the `-Zhelp` output:
```
-Z time=val -- measure time of rustc processes (default: no)
-Z time-passes=val -- measure time of each rustc pass (default: no)
```
The `-Ztime-passes` description is clear, but the `-Ztime` one is less so.
Sounds like it measures the time for the entire process?
No. The real difference is that `-Ztime-passes` prints out info about passes,
and `-Ztime` does the same, but only for a subset of those passes. More
specifically, there is a distinction in the profiling code between a "verbose
generic activity" and an "extra verbose generic activity". `-Ztime-passes`
prints both kinds, while `-Ztime` only prints the first one. (It took me
a close reading of the source code to determine this difference.)
In practice this distinction has low value. Perhaps in the past the "extra
verbose" output was more voluminous, but now that we only print stats for a
pass if it exceeds 5ms or alters the RSS, `-Ztime-passes` is less spammy. Also,
a lot of the "extra verbose" cases are for individual lint passes, and you need
to also use `-Zno-interleave-lints` to see those anyway.
Therefore, this commit removes `-Ztime` and the associated machinery. One thing
to note is that the existing "extra verbose" activities all have an extra
string argument, so the commit adds the ability to accept an extra argument to
the "verbose" activities.
Migrate more of rustc_parse to SessionDiagnostic
Still far from complete, but I thought I'd add a checkpoint here because rebasing was starting to get annoying.
In #101230, the internal diagnostic migration lints -
`diagnostic_outside_of_impl` and `untranslatable_diagnostic` - were
modified so that they wouldn't trigger on functions annotated with
`#[rustc_lint_diagnostics]`. However, this change has to make it into
the bootstrap compiler before the `#[allow]` annotations that it aims to
remove can be removed, which is possible now that #102051 has landed.
Signed-off-by: David Wood <david.wood@huawei.com>
Remove support for legacy PM
This removes support for optimizing with LLVM's legacy pass manager, as well as the unstable `-Znew-llvm-pass-manager` option. We have been defaulting to the new PM since LLVM 13 (except for s390x that waited for 14), and LLVM 15 removed support altogether. The only place we still use the legacy PM is for writing the output file, just like `llc` does.
cc #74705
r? ``@nikic``
FIX - ambiguous Diagnostic link in docs
UPDATE - rename diagnostic_items to IntoDiagnostic and AddToDiagnostic
[Gardening] FIX - formatting via `x fmt`
FIX - rebase conflicts. NOTE: Confirm wheather or not we want to handle TargetDataLayoutErrorsWrapper this way
DELETE - unneeded allow attributes in Handler method
FIX - broken test
FIX - Rebase conflict
UPDATE - rename residual _SessionDiagnostic and fix LintDiag link
On later stages, the feature is already stable.
Result of running:
rg -l "feature.let_else" compiler/ src/librustdoc/ library/ | xargs sed -s -i "s#\\[feature.let_else#\\[cfg_attr\\(bootstrap, feature\\(let_else\\)#"
Avoid `Iterator::last`
Adapters like `Filter` and `Map` use the default implementation of `Iterator::last` which is not short-circuiting (and so does `core::str::Split`). The predicate function will be run for every single item of the underlying iterator. I hope that removing those calls to `last` results in slight performance improvements.
change rlib format to distinguish native dependencies
Another one method to solve problem mentioned in #99429.
Changed .rlib format, it contains all bundled native libraries as archieves.
At link time rlib is unpacked and native dependencies linked separately.
New behavior hidden under separate_native_rlib_dependencies flag.
Add inline-llvm option for disabling/enabling LLVM inlining
In this PR, a new -Z option `inline-llvm` is added in order to be able to turn on/off LLVM inlining.
The capability of turning on/off inlining in LLVM backend is needed for testing performance implications of using recently enabled inlining in rustc's frontend (with -Z inline-mir=yes option, #91743). It would be interesting to see the performance effect using rustc's frontend inlining only without LLVM inlining enabled. Currently LLVM is still doing inlining no mater what value inline-mir is set to. With the option `inline-llvm` being added in this PR, user can turn off LLVM inlining by using `-Z inline-llvm=no` option (the default of inline-llvm is 'yes', LLVM inlining enabled).
This commit removes the allows rules for the SessionDiagnostic lint
that were being used in the session.rs file.
Thanks to the PR #101230 we do not need to annotate the methods with
the allow rule as they are part of the diagnostic machinery.
Suggested by the team in this Zulip Topic https://rust-lang.zulipchat.com/#narrow/stream/336883-i18n/topic/.23100717.20SessionDiagnostic.20on.20Handler
Handler already has almost all the capabilities of ParseSess when it comes to diagnostic emission, in this migration we only needed to add the ability to access source_map from the emitter in order to get a Snippet and the start_point. Not sure if this is the best way to address this gap
Add warning against unexpected --cfg with --check-cfg
This PR adds a warning when an unexpected `--cfg` is specified but not in the specified list of `--check-cfg`.
This is the follow-up PR I mentioned in https://github.com/rust-lang/rust/pull/99519.
r? `@petrochenkov`
Support `#[unix_sigpipe = "inherit|sig_dfl"]` on `fn main()` to prevent ignoring `SIGPIPE`
When enabled, programs don't have to explicitly handle `ErrorKind::BrokenPipe` any longer. Currently, the program
```rust
fn main() { loop { println!("hello world"); } }
```
will print an error if used with a short-lived pipe, e.g.
% ./main | head -n 1
hello world
thread 'main' panicked at 'failed printing to stdout: Broken pipe (os error 32)', library/std/src/io/stdio.rs:1016:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
by enabling `#[unix_sigpipe = "sig_dfl"]` like this
```rust
#![feature(unix_sigpipe)]
#[unix_sigpipe = "sig_dfl"]
fn main() { loop { println!("hello world"); } }
```
there is no error, because `SIGPIPE` will not be ignored and thus the program will be killed appropriately:
% ./main | head -n 1
hello world
The current libstd behaviour of ignoring `SIGPIPE` before `fn main()` can be explicitly requested by using `#[unix_sigpipe = "sig_ign"]`.
With `#[unix_sigpipe = "inherit"]`, no change at all is made to `SIGPIPE`, which typically means the behaviour will be the same as `#[unix_sigpipe = "sig_dfl"]`.
See https://github.com/rust-lang/rust/issues/62569 and referenced issues for discussions regarding the `SIGPIPE` problem itself
See the [this](https://rust-lang.zulipchat.com/#narrow/stream/219381-t-libs/topic/Proposal.3A.20First.20step.20towards.20solving.20the.20SIGPIPE.20problem) Zulip topic for more discussions, including about this PR.
Tracking issue: https://github.com/rust-lang/rust/issues/97889
rustc_target: Add a compatibility layer to separate internal and user-facing linker flavors
I want to do some refactorings in `rustc_target` - merge `lld_flavor` and `linker_is_gnu` into `linker_flavor`, support combination gcc+lld (https://github.com/rust-lang/rust/pull/96827).
This PR adds some compatibility infra that makes that possible without making any changes to user-facing interfaces - `-Clinker-flavor` values and json target specs. (For json target specs this infra may eventually go away since they are not very stable.)
The second commit does some light refactoring of internal linker flavors (applies changes from 53eca42973 that don't require mass-editing target specs).
translations(rustc_session): migrates `rustc_session` to use `SessionDiagnostic` - Pt. 1
## Description
This is the first PR for the migration of the module `rustc_session`. You can follow my progress [here](https://github.com/rust-lang/rust/issues/100717#issuecomment-1220279883).
The PR migrates the files `cgu_reuse_tracker` and `parse.rs` to use `SessionDiagnostic `.
This makes it possible to instruct libstd to never touch the signal
handler for `SIGPIPE`, which makes programs pipeable by default (e.g.
with `./your-program | head -n 1`) without `ErrorKind::BrokenPipe`
errors.
Implementation of import_name_type
Fixes#96534 by implementing https://github.com/rust-lang/compiler-team/issues/525
Symbols that are exported or imported from a binary on 32bit x86 Windows can be named in four separate ways, corresponding to the [import name types](https://docs.microsoft.com/en-us/windows/win32/debug/pe-format#import-name-type) from the PE-COFF spec. The exporting and importing binaries must use the same name encoding, otherwise mismatches can lead to link failures due to "missing symbols" or to 0xc0000139 (`STATUS_ENTRYPOINT_NOT_FOUND`) errors when the executable/library is loaded. For details, see the comments on the raw-dylib feature's https://github.com/rust-lang/rust/issues/58713. To generate the correct import libraries for these DLLs, therefore, rustc must know the import name type for each `extern` function, and there is currently no way for users to provide this information.
This change adds a new `MetaNameValueStr` key to the `#[link]` attribute called `import_name_type`, and which accepts one of three values: `decorated`, `noprefix`, and `undecorated`.
A single DLL is likely to export all its functions using the same import type name, hence `import_name_type` is a parameter of `#[link]` rather than being its own attribute that is applied per-function. It is possible to have a single DLL that exports different functions using different import name types, but users could express such cases by providing multiple export blocks for the same DLL, each with a different import name type.
Note: there is a fourth import name type defined in the PE-COFF spec, `IMPORT_ORDINAL`. This case is already handled by the `#[link_ordinal]` attribute. While it could be merged into `import_type_name`, that would not make sense as `#[link_ordinal]` provides per-function information (namely the ordinal itself).
Design decisions (these match the MCP linked above):
* For GNU, `decorated` matches the PE Spec and MSVC rather than the default behavior of `dlltool` (i.e., there will be a leading `_` for `stdcall`).
* If `import_name_type` is not present, we will keep our current behavior of matching the environment (MSVC vs GNU) default for decorating.
* Using `import_name_type` on architectures other than 32bit x86 will result in an error.
* Using `import_name_type` with link kinds other than `"raw-dylib"` will result in an error.
Diagnostics migr const eval
This PR should eventually contain all diagnostic migrations for the `rustc_const_eval` crate.
r? `@davidtwco`
`@rustbot` label +A-translation
session: stabilize split debuginfo on linux
Stabilize the `-Csplit-debuginfo` flag...
- ...on Linux for all values of the flag. Split DWARF has been implemented for a few months, hasn't had any bug reports and has had some promising benchmarking for incremental debug build performance.
- ..on other platforms for the default value. It doesn't make any sense that `-Csplit-debuginfo=packed` is unstable on Windows MSVC when that's the default behaviour, but keep the other values unstable.
This commit migrates the errors in the function check_expected_reuse
to use the new SessionDiagnostic. It also does some small refactor
for the IncorrectCguReuseType to include the 'at least' word in the
fluent translation file
Stabilize the `-Csplit-debuginfo` flag...
- ...on Linux for all values of the flag. Split DWARF has been
implemented for a few months, hasn't had any bug reports and has had
some promising benchmarking for incremental debug build performance.
- ..on other platforms for the default value. It doesn't make any sense
that `-Csplit-debuginfo=packed` is unstable on Windows MSVC when
that's the default behaviour, but keep the other values unstable.
Signed-off-by: David Wood <david.wood@huawei.com>
`-Z location-detail`: provide option to disable all location details
As reported [here](https://github.com/rust-lang/rust/pull/89920#issuecomment-1190598924), when I first implemented the `-Z location-detail` flag there was a bug, where passing an empty list was not correctly supported, and instead rejected by the compiler. This PR fixes that such that passing an empty list results in no location details being tracked, as originally specified in https://github.com/rust-lang/rfcs/pull/2091 .
This PR also adds a test case to verify that this option continues to work as intended.
Remove implicit names and values from `--cfg` in `--check-cfg`
This PR remove the implicit names and values from `--cfg` in `--check-cfg` because the behavior is quite surprising but also because it's really easy to inadvertently really on the implicitness and when the `--cfg` is not set anymore to have an unexpected warning from an unexpected condition that pass with the implicitness.
This change in behavior will also enable us to warn when an unexpected `--cfg` is passed, ex: the user wrote `--cfg=unstabl` instead of `--cfg=unstable`. The implementation of the warning will be done in a follow-up PR.
cc `@petrochenkov`
proc_macro: use crossbeam channels for the proc_macro cross-thread bridge
This is done by having the crossbeam dependency inserted into the `proc_macro` server code from the server side, to avoid adding a dependency to `proc_macro`.
In addition, this introduces a -Z command-line option which will switch rustc to run proc-macros using this cross-thread executor. With the changes to the bridge in #98186, #98187, #98188 and #98189, the performance of the executor should be much closer to same-thread execution.
In local testing, the crossbeam executor was substantially more performant than either of the two existing `CrossThread` strategies, so they have been removed to keep things simple.
r? `@eddyb`
This is done by having the crossbeam dependency inserted into the
proc_macro server code from the server side, to avoid adding a
dependency to proc_macro.
In addition, this introduces a -Z command-line option which will switch
rustc to run proc-macros using this cross-thread executor. With the
changes to the bridge in #98186, #98187, #98188 and #98189, the
performance of the executor should be much closer to same-thread
execution.
In local testing, the crossbeam executor was substantially more
performant than either of the two existing CrossThread strategies, so
they have been removed to keep things simple.
Prior to this fix, `-Z location-detail` provided no mechanism for
disabling all location details. This commit also adds a test case
to verify that this option continues to work as intended, and
clarifies the documentation of this option.
Some command-line options accessible through `sess.opts` are best
accessed through wrapper functions on `Session`, `TyCtxt` or otherwise,
rather than through field access on the option struct in the `Session`.
Adds a new lint which triggers on those options that should be accessed
through a wrapper function so that this is prohibited. Options are
annotated with a new attribute `rustc_lint_opt_deny_field_access` which
can specify the error message (i.e. "use this other function instead")
to be emitted.
A simpler alternative would be to simply rename the options in the
option type so that it is clear they should not be used, however this
doesn't prevent uses, just discourages them. Another alternative would
be to make the option fields private, and adding accessor functions on
the option types, however the wrapper functions sometimes rely on
additional state from `Session` or `TyCtxt` which wouldn't be available
in an function on the option type, so the accessor would simply make the
field available and its use would be discouraged too.
Signed-off-by: David Wood <david.wood@huawei.com>
If an internal lint uses `typeck_results` or similar queries then that
can result in rustdoc checking code that it shouldn't (e.g. from other
platforms) and emit compilation errors.
Signed-off-by: David Wood <david.wood@huawei.com>
Add support for LLVM ShadowCallStack.
LLVMs ShadowCallStack provides backward edge control flow integrity protection by using a separate shadow stack to store and retrieve a function's return address.
LLVM currently only supports this for AArch64 targets. The x18 register is used to hold the pointer to the shadow stack, and therefore this only works on ABIs which reserve x18. Further details are available in the [LLVM ShadowCallStack](https://clang.llvm.org/docs/ShadowCallStack.html) docs.
# Usage
`-Zsanitizer=shadow-call-stack`
# Comments/Caveats
* Currently only enabled for the aarch64-linux-android target
* Requires the platform to define a runtime to initialize the shadow stack, see the [LLVM docs](https://clang.llvm.org/docs/ShadowCallStack.html) for more detail.
Allow to disable thinLTO buffer to support lto-embed-bitcode lld feature
Hello
This change is to fix issue (https://github.com/rust-lang/rust/issues/84395) in which passing "-lto-embed-bitcode=optimized" to lld when linking rust code via linker-plugin-lto doesn't produce the expected result.
Instead of emitting a single unified module into a llvmbc section of the linked elf, it emits multiple submodules.
This is caused because rustc emits the BC modules after running llvm `createWriteThinLTOBitcodePass` pass.
Which in turn triggers a thinLTO linkage and causes the said issue.
This patch allows via compiler flag (-Cemit-thin-lto=<bool>) to select between running `createWriteThinLTOBitcodePass` and `createBitcodeWriterPass`.
Note this pattern of selecting between those 2 passes is common inside of LLVM code.
The default is to match the old behavior.
Adding the option to control from rustc CLI
if the resulted ".o" bitcode module files are with
thinLTO info or regular LTO info.
Allows using "-lto-embed-bitcode=optimized" during linkage
correctly.
Signed-off-by: Ziv Dunkelman <ziv.dunkelman@nextsilicon.com>
Keep unstable target features for asm feature checking
Inline assembly uses the target features to determine which registers
are available on the current target. However it needs to be able to
access unstable target features for this.
Fixes#99071
Inline assembly uses the target features to determine which registers
are available on the current target. However it needs to be able to
access unstable target features for this.
Fixes#99071
DWARF version 5 brings a number of improvements over version 4. Quoting from
the announcement [1]:
> Version 5 incorporates improvements in many areas: better data compression,
> separation of debugging data from executable files, improved description of
> macros and source files, faster searching for symbols, improved debugging
> optimized code, as well as numerous improvements in functionality and
> performance.
On platforms where DWARF version 5 is supported (Linux, primarily), this commit
adds support for it behind a new `-Z dwarf-version=5` flag.
[1]: https://dwarfstd.org/Public_Review.php
Migrate some diagnostics from `rustc_const_eval` to `SessionDiagnostic`
I'm still trying to get the hang of this, so it doesn't migrate _all_ of `rustc_const_eval`. Working on that later.
r? `@davidtwco`
Add a `-Zdump-drop-tracking-cfg` debugging flag
This is useful for debugging drop-tracking; previously, you had to recompile
rustc from source and manually add a call to `write_graph_to_file`. This
makes the option more discoverable and configurable at runtime.
I also took the liberty of making the labels for the CFG nodes much easier to read:
previously, they looked like `id(2), local_id: 48`, now they look like
```
expr from_config (hir_id=HirId { owner: DefId(0:10 ~ default_struct_update[79f9]::foo), local_id: 2})
```
r? ``@eholk``
Rename the `--output-width` flag to `--diagnostic-width` as this appears
to be the preferred name within the compiler team.
Signed-off-by: David Wood <david.wood@huawei.com>
Rename the `--terminal-width` flag to `--output-width` as the behaviour
doesn't just apply to terminals (and so is slightly less accurate).
Signed-off-by: David Wood <david.wood@huawei.com>
Formerly `-Zterminal-width`, `--terminal-width` allows the user or build
tool to inform rustc of the width of the terminal so that diagnostics
can be truncated.
Signed-off-by: David Wood <david.wood@huawei.com>
The `rustc_lint_diagnostics` attribute is used by the diagnostic
translation/struct migration lints to identify calls where
non-translatable diagnostics or diagnostics outwith impls are being
created. Any function used in creating a diagnostic should be annotated
with this attribute so this commit adds the attribute to many more
functions.
Signed-off-by: David Wood <david.wood@huawei.com>
This is useful for debugging drop-tracking; previously, you had to recompile
rustc from source and manually add a call to `write_graph_to_file`. This
makes the option more discoverable and configurable at runtime.
I also took the liberty of making the labels for the CFG nodes much easier to read:
previously, they looked like `id(2), local_id: 48`, now they look like
```
expr from_config (hir_id=HirId { owner: DefId(0:10 ~ default_struct_update[79f9]::foo), local_id: 2})
```
Support lint expectations for `--force-warn` lints (RFC 2383)
Rustc has a `--force-warn` flag, which overrides lint level attributes and forces the diagnostics to always be warn. This means, that for lint expectations, the diagnostic can't be suppressed as usual. This also means that the expectation would not be fulfilled, even if a lint had been triggered in the expected scope.
This PR now also tracks the expectation ID in the `ForceWarn` level. I've also made some minor adjustments, to possibly catch more bugs and make the whole implementation more robust.
This will probably conflict with https://github.com/rust-lang/rust/pull/97718. That PR should ideally be reviewed and merged first. The conflict itself will be trivial to fix.
---
r? `@wesleywiser`
cc: `@flip1995` since you've helped with the initial review and also discussed this topic with me. 🙃
Follow-up of: https://github.com/rust-lang/rust/pull/87835
Issue: https://github.com/rust-lang/rust/issues/85549
Yeah, and that's it.
Integrate measureme's hardware performance counter support.
*Note: this is a companion to https://github.com/rust-lang/measureme/pull/143, and duplicates some information with it for convenience*
**(much later) EDIT**: take any numbers with a grain of salt, they may have changed since initial PR open.
## Credits
I'd like to start by thanking `@alyssais,` `@cuviper,` `@edef1c,` `@glandium,` `@jix,` `@Mark-Simulacrum,` `@m-ou-se,` `@mystor,` `@nagisa,` `@puckipedia,` and `@yorickvP,` for all of their help with testing, and valuable insight and suggestions.
Getting here wouldn't have been possible without you!
(If I've forgotten anyone please let me know, I'm going off memory here, plus some discussion logs)
## Summary
This PR adds support to `-Z self-profile` for counting hardware events such as "instructions retired" (as opposed to being limited to time measurements), using the `rdpmc` instruction on `x86_64` Linux.
While other OSes may eventually be supported, preliminary research suggests some kind of kernel extension/driver is required to enable this, whereas on Linux any user can profile (at least) their own threads.
Supporting Linux on architectures other than x86_64 should be much easier (provided the hardware supports such performance counters), and was mostly not done due to a lack of readily available test hardware.
That said, 32-bit `x86` (aka `i686`) would be almost trivial to add and test once we land the initial `x86_64` version (as all the CPU detection code can be reused).
A new flag `-Z self-profile-counter` was added, to control which of the named `measureme` counters is used, and which defaults to `wall-time`, in order to keep `-Z self-profile`'s current functionality unchanged (at least for now).
The named counters so far are:
* `wall-time`: the existing time measurement
* name chosen for consistency with `perf.rust-lang.org`
* continues to use `std::time::Instant` for a nanosecond-precision "monotonic clock"
* `instructions:u`: the hardware performance counter usually referred to as "Instructions retired"
* here "retired" (roughly) means "fully executed"
* the `:u` suffix is from the Linux `perf` tool and indicates the counter only runs while userspace code is executing, and therefore counts no kernel instructions
* *see [Caveats/Subtracting IRQs](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Subtracting-IRQs) for why this isn't entirely true and why `instructions-minus-irqs:u` should be preferred instead*
* `instructions-minus-irqs:u`: same as `instructions:u`, except the count of hardware interrupts ("IRQs" here for brevity) is subtracted
* *see [Caveats/Subtracting IRQs](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Subtracting-IRQs) for why this should be preferred over `instructions:u`*
* `instructions-minus-r0420:u`: experimental counter, same as `instructions-minus-irqs:u` but subtracting an undocumented counter (`r0420:u`) instead of IRQs
* the `rXXXX` notation is again from Linux `perf`, and indicates a "raw" counter, with a hex representation of the low-level counter configuration - this was picked because we still don't *really* know what it is
* this only exists for (future) testing and isn't included/used in any comparisons/data we've put together so far
* *see [Challenges/Zen's undocumented 420 counter](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Epilogue-Zen’s-undocumented-420-counter) for details on how this counter was found and what it does*
---
There are also some additional commits:
* ~~see [Challenges/Rebasing *shouldn't* affect the results, right?](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Rebasing-*shouldn’t*-affect-the-results,-right) for details on the changes to `rustc_parse` and `rustc_trait_section` (the latter far more dubious, and probably shouldn't be merged, or not as-is)~~
* **EDIT**: the effects of these are no long quantifiable, the PR includes reverts for them
* ~~see [Challenges/`jemalloc`: purging will commence in ten seconds](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#jemalloc-purging-will-commence-in-ten-seconds) for details on the `jemalloc` change~~
* this is also separately found in #77162, and we probably want to avoid doing it by default, ideally we'd use the runtime control API `jemalloc` offers (assuming that can stop the timer that's already running, which I'm not sure about)
* **EDIT**: until we can do this based on `-Z` flags, this commit has also been reverted
* the `proc_macro` change was to avoid randomized hashing and therefore ASLR-like effects
---
**(much later) EDIT**: take any numbers with a grain of salt, they may have changed since initial PR open.
#### Write-up / report
Because of how extensive the full report ended up being, I've kept most of it [on `hackmd.io`](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view), but for convenient access, here are all the sections (with individual links):
<sup>(someone suggested I'd make a backup, so [here it is on the wayback machine](http://web.archive.org/web/20201127164748/https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view) - I'll need to remember to update that if I have to edit the write-up)</sup>
* [**Motivation**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Motivation)
* [**Results**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Results)
* [**Overhead**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Overhead)
*Preview (see the report itself for more details):*
|Counter|Total<br>`instructions-minus-irqs:u`|Overhead from "Baseline"<br>(for all 1903881<br>counter reads)|Overhead from "Baseline"<br>(per each counter read)|
|-|-|-|-|
|Baseline|63637621286 ±6||
|`instructions:u`|63658815885 ±2| +21194599 ±8| +11|
|`instructions-minus-irqs:u`|63680307361 ±13| +42686075 ±19| +22|
|`wall-time`|63951958376 ±10275|+314337090 ±10281|+165|
* [**"Macro" noise (self time)**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#“Macro”-noise-(self-time))
*Preview (see the report itself for more details):*
|| `wall-time` (ns) | `instructions:u` | `instructions-minus-irqs:u`
-: | -: | -: | -:
`typeck` | 5478261360 ±283933373 (±~5.2%) | 17350144522 ±6392 (±~0.00004%) | 17351035832.5 ±4.5 (±~0.00000003%)
`expand_crate` | 2342096719 ±110465856 (±~4.7%) | 8263777916 ±2937 (±~0.00004%) | 8263708389 ±0 (±~0%)
`mir_borrowck` | 2216149671 ±119458444 (±~5.4%) | 8340920100 ±2794 (±~0.00003%) | 8341613983.5 ±2.5 (±~0.00000003%)
`mir_built` | 1269059734 ±91514604 (±~7.2%) | 4454959122 ±1618 (±~0.00004%) | 4455303811 ±1 (±~0.00000002%)
`resolve_crate` | 942154987.5 ±53068423.5 (±~5.6%) | 3951197709 ±39 (±~0.000001%) | 3951196865 ±0 (±~0%)
* [**"Micro" noise (individual sampling intervals)**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#“Micro”-noise-(individual-sampling-intervals))
* [**Caveats**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Caveats)
* [**Disabling ASLR**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Disabling-ASLR)
* [**Non-deterministic proc macros**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Non-deterministic-proc-macros)
* [**Subtracting IRQs**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Subtracting-IRQs)
* [**Lack of support for multiple threads**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Lack-of-support-for-multiple-threads)
* [**Challenges**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Challenges)
* [**How do we even read hardware performance counters?**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#How-do-we-even-read-hardware-performance-counters)
* [**ASLR: it's free entropy**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#ASLR-it’s-free-entropy)
* [**The serializing instruction**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#The-serializing-instruction)
* [**Getting constantly interrupted**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Getting-constantly-interrupted)
* [**AMD patented time-travel and dubbed it `SpecLockMap`<br><sup> or: "how we accidentally unlocked `rr` on AMD Zen"</sup>**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#AMD-patented-time-travel-and-dubbed-it-SpecLockMapnbspnbspnbspnbspnbspnbspnbspnbspor-“how-we-accidentally-unlocked-rr-on-AMD-Zen”)
* [**`jemalloc`: purging will commence in ten seconds**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#jemalloc-purging-will-commence-in-ten-seconds)
* [**Rebasing *shouldn't* affect the results, right?**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Rebasing-*shouldn’t*-affect-the-results,-right)
* [**Epilogue: Zen's undocumented 420 counter**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Epilogue-Zen’s-undocumented-420-counter)
Adds the virtual-function-elimination unstable compiler flag and a check
that this flag is only used in combination with -Clto. LLVM can only
apply this optimization with fat LTO.
lint: add diagnostic translation migration lints
Introduce allow-by-default lints for checking whether diagnostics are written in
`SessionDiagnostic` or `AddSubdiagnostic` impls and whether diagnostics are translatable. These lints can be denied for modules once they are fully migrated to impls and translation.
These lints are intended to be temporary - once all diagnostics have been changed then we can just change the APIs we have and that will enforce these constraints thereafter.
r? `````@oli-obk`````
Fix#71363's test by adding `-Z translate-remapped-path-to-local-path=no`
The test relies on `library/std/src/error.rs` not corresponding to a local path, but remapping might still find the related local file of a remapped path. To fix the test, this PR adds a new `-Z` flag to disable finding the corresponding local path of a remapped path.
Introduce allow-by-default lints for checking whether diagnostics are
written in `SessionDiagnostic`/`AddSubdiagnostic` impls and whether
diagnostics are translatable. These lints can be denied for modules once
they are fully migrated to impls and translation.
Signed-off-by: David Wood <david.wood@huawei.com>
The test relies on library/std/src/error.rs not corresponding to a local
path, but remapping might still find the related local file of a
remapped path. To fix the test, this adds a new -Z flag to disable
finding the corresponding local path of a remapped path.
It looks like the last time had left some remaining cfg's -- which made me think
that the stage0 bump was actually successful. This brings us to a released 1.62
beta though.
Currently, the only API for creating errors from a diagnostic derive
will emit it immediately. This makes it difficult to add subdiagnostics
to diagnostics from the derive, so add `create_{err,warning}` functions
that return the diagnostic without emitting it.
Signed-off-by: David Wood <david.wood@huawei.com>
rustc_ast: Harmonize delimiter naming with `proc_macro::Delimiter`
Compiler cannot reuse `proc_macro::Delimiter` directly due to extra impls, but can at least use the same naming.
After this PR the only difference between these two enums is that `proc_macro::Delimiter::None` is turned into `token::Delimiter::Invisible`.
It's my mistake that the invisible delimiter is called `None` on stable, during the stabilization I audited the naming and wrote the docs, but missed the fact that the `None` naming gives a wrong and confusing impression about what this thing is.
cc https://github.com/rust-lang/rust/pull/96421
r? ``@nnethercote``
Make sure `-Dunused-crate-dependencies --json unused-externs` makes rustc exit with error status
This PR:
- fixes compiletest to understand unused extern notifications
- adds tests for `--json unused-externs`
- makes sure that deny-level unused externs notifications are treated as compile errors
- refactors the `emit_unused_externs` callstack to plumb through the level as an enum as a string, and adds `Level::is_error`
Update: adds `--json unused-externs-silent` with the original behaviour since Cargo needs it. Should address `@est31's` concerns.
Fixes: https://github.com/rust-lang/rust/issues/96068
Add missing `target_feature` to the list of well known cfg names
This PR adds the missing `target_feature` cfg name to the list of well known cfg names.
It was notice missing in https://github.com/rust-lang/rust/issues/96472 thanks to `@bjorn3,` the reason being that `--check-cfg=names()` automatically inherit the names passed by `--cfg` (or internal to `rustc`) and is seems that the vast majority of targets have at least one target feature leading to `target_feature` being a well known name in most target but it should always be a well known name so this PR add it unconditionally to list.
r? `@petrochenkov`
Since Cargo wants to do its own fatal error handling for unused
dependencies, add the option `--json unused-externs-silent` which
has the original behaviour of not indicating non-zero exit status for
`deny`/`forbid`-level unused dependencies.
Implement MIR opt unit tests
This implements rust-lang/compiler-team#502 .
There's not much to say here, this implementation does everything as proposed. I also added the flag to a bunch of existing tests (mostly those to which I could add it without causing huge diffs due to changes in line numbers). Summarizing the changes to test outputs:
- Every time an `MirPatch` is created, it adds a cleanup block to the body if it did not exist already. If this block is unused (as is usually the case), it usually gets removed soon after by some pass calling `SimplifyCFG` for unrelated reasons (in many cases this cycle happens quite a few times for a single body). We now run `SimplifyCFG` less often, so those blocks end up in some of our outputs. I looked at changing `MirPatch` to not do this, but that seemed too complicated for this PR. I may still do that in a follow-up.
- The `InstCombine` test had set `-C opt-level=0` in its flags and so there were no storage markers. I don't really see a good motivation for doing this, so bringing it back in line with what everything else does seems correct.
- One of the `EarlyOtherwiseBranch` tests had `UnreachableProp` running on it. Preventing that kind of thing is the goal of this feature, so this seems fine.
For the remaining tests for which this feature might be useful, we can gradually migrate them as opportunities present themselves.
In terms of documentation, I plan on submitting a PR to the rustc dev guide in the near future documenting this and other recent changes to MIR. If there's any other places to update, do let me know
r? `@nagisa`
This adds `nounused` to the set of extern flags:
`--extern nounused:core=/path/to/core/libcore.rlib`.
The effect of this flag is to suppress `unused-crate-dependencies`
warnings relating to the crate.
Remove `--extern-location` and all associated code
`--extern-location` was an experiment to investigate the best way to
generate useful diagnostics for unused dependency warnings by enabling a
build system to identify the corresponding build config.
While I did successfully use this, I've since been convinced the
alternative `--json unused-externs` mechanism is the way to go, and
there's no point in having two mechanisms with basically the same
functionality.
This effectively reverts https://github.com/rust-lang/rust/pull/72603
Create (unstable) 2024 edition
[On Zulip](https://rust-lang.zulipchat.com/#narrow/stream/213817-t-lang/topic/Deprecating.20macro.20scoping.20shenanigans/near/272860652), there was a small aside regarding creating the 2024 edition now as opposed to later. There was a reasonable amount of support and no stated opposition.
This change creates the 2024 edition in the compiler and creates a prelude for the 2024 edition. There is no current difference between the 2021 and 2024 editions. Cargo and other tools will need to be updated separately, as it's not in the same repository. This change permits the vast majority of work towards the next edition to proceed _now_ instead of waiting until 2024.
For sanity purposes, I've merged the "hello" UI tests into a single file with multiple revisions. Otherwise we'd end up with a file per edition, despite them being essentially identical.
````@rustbot```` label +T-lang +S-waiting-on-review
Not sure on the relevant team, to be honest.
Stabilize `derive_default_enum`
This stabilizes `#![feature(derive_default_enum)]`, as proposed in [RFC 3107](https://github.com/rust-lang/rfcs/pull/3107) and tracked in #87517. In short, it permits you to `#[derive(Default)]` on `enum`s, indicating what the default should be by placing a `#[default]` attribute on the desired variant (which must be a unit variant in the interest of forward compatibility).
```````@rustbot``````` label +S-waiting-on-review +T-lang
`--extern-location` was an experiment to investigate the best way to
generate useful diagnostics for unused dependency warnings by enabling a
build system to identify the corresponding build config.
While I did successfully use this, I've since been convinced the
alternative `--json unused-externs` mechanism is the way to go, and
there's no point in having two mechanisms with basically the same
functionality.
This effectively reverts https://github.com/rust-lang/rust/pull/72603
Loading the fallback bundle in compilation sessions that won't go on to
emit any errors unnecessarily degrades compile time performance, so
lazily create the Fluent bundle when it is first required.
Signed-off-by: David Wood <david.wood@huawei.com>
Instead of checking only the user provided sysroot or the default (when
no sysroot is provided), search user provided sysroot and then check
default sysroots for locale requested by the user.
Signed-off-by: David Wood <david.wood@huawei.com>
It was used for deduplicating some errors for legacy code which are mostly deduplicated even without that, but at cost of global mutable state, which is not a good tradeoff.
Add an option for enabling and disabling Fluent's directionality
isolation markers in output. Disabled by default as these can render in
some terminals and applications.
Signed-off-by: David Wood <david.wood@huawei.com>
Extend loading of Fluent bundles so that bundles can be loaded from the
sysroot based on the language requested by the user, or using a nightly
flag.
Sysroot bundles are loaded from `$sysroot/share/locale/$locale/*.ftl`.
Signed-off-by: David Wood <david.wood@huawei.com>
This commit updates the signatures of all diagnostic functions to accept
types that can be converted into a `DiagnosticMessage`. This enables
existing diagnostic calls to continue to work as before and Fluent
identifiers to be provided. The `SessionDiagnostic` derive just
generates normal diagnostic calls, so these APIs had to be modified to
accept Fluent identifiers.
In addition, loading of the "fallback" Fluent bundle, which contains the
built-in English messages, has been implemented.
Each diagnostic now has "arguments" which correspond to variables in the
Fluent messages (necessary to render a Fluent message) but no API for
adding arguments has been added yet. Therefore, diagnostics (that do not
require interpolation) can be converted to use Fluent identifiers and
will be output as before.
`MultiSpan` contains labels, which are more complicated with the
introduction of diagnostic translation and will use types from
`rustc_errors` - however, `rustc_errors` depends on `rustc_span` so
`rustc_span` cannot use types like `DiagnosticMessage` without
dependency cycles. Introduce a new `rustc_error_messages` crate that can
contain `DiagnosticMessage` and `MultiSpan`.
Signed-off-by: David Wood <david.wood@huawei.com>
Implement -Z oom=panic
This PR removes the `#[rustc_allocator_nounwind]` attribute on `alloc_error_handler` which allows it to unwind with a panic instead of always aborting. This is then used to implement `-Z oom=panic` as per RFC 2116 (tracking issue #43596).
Perf and binary size tests show negligible impact.
There are a few places were we have to construct it, though, and a few
places that are more invasive to change. To do this, we create a
constructor with a long obvious name.
Treat unstable lints as unknown
This change causes unstable lints to be ignored if the `unknown_lints`
lint is allowed. To achieve this, it also changes lints to apply as soon
as they are processed. Previously, lints in the same set were processed
as a batch and then all simultaneously applied.
Implementation of https://github.com/rust-lang/compiler-team/issues/469
This also affects the `non_exhaustive_omitted_patterns` and
`must_not_suspend` lints as they are not stable. This also changes the
diagnostic level to pull from `unknown_lints` instead of always being
allow or deny.
Add well known values to `--check-cfg` implementation
This pull-request adds well known values for the well known names via `--check-cfg=values()`.
[RFC 3013: Checking conditional compilation at compile time](https://rust-lang.github.io/rfcs/3013-conditional-compilation-checking.html#checking-conditional-compilation-at-compile-time) doesn't define this at all, but this seems a nice improvement.
The activation is done by a empty `values()` (new syntax) similar to `names()` except that `names(foo)` also activate well known names while `values(aa, "aa", "kk")` would not.
As stated this use a different activation logic because well known values for the well known names are not always sufficient.
In fact this is problematic for every `target_*` cfg because of non builtin targets, as the current implementation use those built-ins targets to create the list the well known values.
The implementation is straight forward, first we gather (if necessary) all the values (lazily or not) and then we apply them.
r? ```@petrochenkov```
Remove num_cpus dependency from bootstrap, build-manifest and rustc_s…
…ession
`std::threads::available_parallelism` was stabilized in rust 1.59.
r? ```````````````````````````@Mark-Simulacrum```````````````````````````
Implementation of the `expect` attribute (RFC 2383)
This is an implementation of the `expect` attribute as described in [RFC-2383](https://rust-lang.github.io/rfcs/2383-lint-reasons.html). The attribute allows the suppression of lint message by expecting them. Unfulfilled lint expectations (meaning no expected lint was caught) will emit the `unfulfilled_lint_expectations` lint at the `expect` attribute.
### Example
#### input
```rs
// required feature flag
#![feature(lint_reasons)]
#[expect(unused_mut)] // Will warn about an unfulfilled expectation
#[expect(unused_variables)] // Will be fulfilled by x
fn main() {
let x = 0;
}
```
#### output
```txt
warning: this lint expectation is unfulfilled
--> $DIR/trigger_lint.rs:3:1
|
LL | #[expect(unused_mut)] // Will warn about an unfulfilled expectation
| ^^^^^^^^^^
|
= note: `#[warn(unfulfilled_lint_expectations)]` on by default
```
### Implementation
This implementation introduces `Expect` as a new lint level for diagnostics, which have been expected. All lint expectations marked via the `expect` attribute are collected in the [`LintLevelsBuilder`] and assigned an ID that is stored in the new lint level. The `LintLevelsBuilder` stores all found expectations and the data needed to emit the `unfulfilled_lint_expectations` in the [`LintLevelsMap`] which is the result of the [`lint_levels()`] query.
The [`rustc_errors::HandlerInner`] is the central error handler in rustc and handles the emission of all diagnostics. Lint message with the level `Expect` are suppressed during this emission, while the expectation ID is stored in a set which marks them as fulfilled. The last step is then so simply check if all expectations collected by the [`LintLevelsBuilder`] in the [`LintLevelsMap`] have been marked as fulfilled in the [`rustc_errors::HandlerInner`]. Otherwise, a new lint message will be emitted.
The implementation of the `LintExpectationId` required some special handling to make it stable between sessions. Lints can be emitted during [`EarlyLintPass`]es. At this stage, it's not possible to create a stable identifier. The level instead stores an unstable identifier, which is later converted to a stable `LintExpectationId`.
### Followup TO-DOs
All open TO-DOs have been marked with `FIXME` comments in the code. This is the combined list of them:
* [ ] The current implementation doesn't cover cases where the `unfulfilled_lint_expectations` lint is actually expected by another `expect` attribute.
* This should be easily possible, but I wanted to get some feedback before putting more work into this.
* This could also be done in a new PR to not add to much more code to this one
* [ ] Update unstable documentation to reflect this change.
* [ ] Update unstable expectation ids in [`HandlerInner::stashed_diagnostics`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_errors/struct.HandlerInner.html#structfield.stashed_diagnostics)
### Open questions
I also have a few open questions where I would like to get feedback on:
1. The RFC discussion included a suggestion to change the `expect` attribute to something else. (Initiated by `@Ixrec` [here](https://github.com/rust-lang/rfcs/pull/2383#issuecomment-378424091), suggestion from `@scottmcm` to use `#[should_lint(...)]` [here](https://github.com/rust-lang/rfcs/pull/2383#issuecomment-378648877)). No real conclusion was drawn on that point from my understanding. Is this still open for discussion, or was this discarded with the merge of the RFC?
2. How should the expect attribute deal with the new `force-warn` lint level?
---
This approach was inspired by a discussion with `@LeSeulArtichaut.`
RFC tracking issue: #54503
Mentoring/Implementation issue: #85549
[`LintLevelsBuilder`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_lint/levels/struct.LintLevelsBuilder.html
[`LintLevelsMap`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/lint/struct.LintLevelMap.html
[`lint_levels()`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/context/struct.TyCtxt.html#method.lint_levels
[`rustc_errors::HandlerInner`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_errors/struct.HandlerInner.html
[`EarlyLintPass`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_lint/trait.EarlyLintPass.html
No branch protection metadata unless enabled
Even if we emit metadata disabling branch protection, this metadata may
conflict with other modules (e.g. during LTO) that have different branch
protection metadata set.
This is an unstable flag and feature, so ideally the flag not being
specified should act as if the feature wasn't implemented in the first
place.
Additionally this PR also ensures we emit an error if
`-Zbranch-protection` is set on targets other than the supported
aarch64. For now the error is being output from codegen, but ideally it
should be moved to earlier in the pipeline before stabilization.
Use undef for (some) partially-uninit constants
There needs to be some limit to avoid perf regressions on large arrays
with undef in each element (see comment in the code).
Fixes: #84565
Original PR: #83698
Depends on LLVM 14: #93577
rustc_errors: let `DiagnosticBuilder::emit` return a "guarantee of emission".
That is, `DiagnosticBuilder` is now generic over the return type of `.emit()`, so we'll now have:
* `DiagnosticBuilder<ErrorReported>` for error (incl. fatal/bug) diagnostics
* can only be created via a `const L: Level`-generic constructor, that limits allowed variants via a `where` clause, so not even `rustc_errors` can accidentally bypass this limitation
* asserts `diagnostic.is_error()` on emission, just in case the construction restriction was bypassed (e.g. by replacing the whole `Diagnostic` inside `DiagnosticBuilder`)
* `.emit()` returns `ErrorReported`, as a "proof" token that `.emit()` was called
(though note that this isn't a real guarantee until after completing the work on
#69426)
* `DiagnosticBuilder<()>` for everything else (warnings, notes, etc.)
* can also be obtained from other `DiagnosticBuilder`s by calling `.forget_guarantee()`
This PR is a companion to other ongoing work, namely:
* #69426
and it's ongoing implementation:
#93222
the API changes in this PR are needed to get statically-checked "only errors produce `ErrorReported` from `.emit()`", but doesn't itself provide any really strong guarantees without those other `ErrorReported` changes
* #93244
would make the choices of API changes (esp. naming) in this PR fit better overall
In order to be able to let `.emit()` return anything trustable, several changes had to be made:
* `Diagnostic`'s `level` field is now private to `rustc_errors`, to disallow arbitrary "downgrade"s from "some kind of error" to "warning" (or anything else that doesn't cause compilation to fail)
* it's still possible to replace the whole `Diagnostic` inside the `DiagnosticBuilder`, sadly, that's harder to fix, but it's unlikely enough that we can paper over it with asserts on `.emit()`
* `.cancel()` now consumes `DiagnosticBuilder`, preventing `.emit()` calls on a cancelled diagnostic
* it's also now done internally, through `DiagnosticBuilder`-private state, instead of having a `Level::Cancelled` variant that can be read (or worse, written) by the user
* this removes a hazard of calling `.cancel()` on an error then continuing to attach details to it, and even expect to be able to `.emit()` it
* warnings were switched to *only* `can_emit_warnings` on emission (instead of pre-cancelling early)
* `struct_dummy` was removed (as it relied on a pre-`Cancelled` `Diagnostic`)
* since `.emit()` doesn't consume the `DiagnosticBuilder` <sub>(I tried and gave up, it's much more work than this PR)</sub>,
we have to make `.emit()` idempotent wrt the guarantees it returns
* thankfully, `err.emit(); err.emit();` can return `ErrorReported` both times, as the second `.emit()` call has no side-effects *only* because the first one did do the appropriate emission
* `&mut Diagnostic` is now used in a lot of function signatures, which used to take `&mut DiagnosticBuilder` (in the interest of not having to make those functions generic)
* the APIs were already mostly identical, allowing for low-effort porting to this new setup
* only some of the suggestion methods needed some rework, to have the extra `DiagnosticBuilder` functionality on the `Diagnostic` methods themselves (that change is also present in #93259)
* `.emit()`/`.cancel()` aren't available, but IMO calling them from an "error decorator/annotator" function isn't a good practice, and can lead to strange behavior (from the caller's perspective)
* `.downgrade_to_delayed_bug()` was added, letting you convert any `.is_error()` diagnostic into a `delay_span_bug` one (which works because in both cases the guarantees available are the same)
This PR should ideally be reviewed commit-by-commit, since there is a lot of fallout in each.
r? `@estebank` cc `@Manishearth` `@nikomatsakis` `@mark-i-m`
Improve `--check-cfg` implementation
This pull-request is a mix of improvements regarding the `--check-cfg` implementation:
- Simpler internal representation (usage of `Option` instead of separate bool)
- Add --check-cfg to the unstable book (based on the RFC)
- Improved diagnostics:
* List possible values when the value is unexpected
* Suggest if possible a name or value that is similar
- Add more tests (well known names, mix of combinations, ...)
r? ```@petrochenkov```
Adopt let else in more places
Continuation of #89933, #91018, #91481, #93046, #93590, #94011.
I have extended my clippy lint to also recognize tuple passing and match statements. The diff caused by fixing it is way above 1 thousand lines. Thus, I split it up into multiple pull requests to make reviewing easier. This is the biggest of these PRs and handles the changes outside of rustdoc, rustc_typeck, rustc_const_eval, rustc_trait_selection, which were handled in PRs #94139, #94142, #94143, #94144.
Even if we emit metadata disabling branch protection, this metadata may
conflict with other modules (e.g. during LTO) that have different branch
protection metadata set.
This is an unstable flag and feature, so ideally the flag not being
specified should act as if the feature wasn't implemented in the first
place.
Additionally this PR also ensures we emit an error if
`-Zbranch-protection` is set on targets other than the supported
aarch64. For now the error is being output from codegen, but ideally it
should be moved to earlier in the pipeline before stabilization.
Add MemTagSanitizer Support
Add support for the LLVM [MemTagSanitizer](https://llvm.org/docs/MemTagSanitizer.html).
On hardware which supports it (see caveats below), the MemTagSanitizer can catch bugs similar to AddressSanitizer and HardwareAddressSanitizer, but with lower overhead.
On a tag mismatch, a SIGSEGV is signaled with code SEGV_MTESERR / SEGV_MTEAERR.
# Usage
`-Zsanitizer=memtag -C target-feature="+mte"`
# Comments/Caveats
* MemTagSanitizer is only supported on AArch64 targets with hardware support
* Requires `-C target-feature="+mte"`
* LLVM MemTagSanitizer currently only performs stack tagging.
# TODO
* Tests
* Example
This change adds a flag for configuring control-flow protection in the
LLVM backend. In Clang, this flag is exposed as `-fcf-protection` with
options `none|branch|return|full`. This convention is followed for
`rustc`, though as a codegen option: `rustc -Z
cf-protection=<none|branch|return|full>`.
Co-authored-by: BlackHoleFox <blackholefoxdev@gmail.com>
This option introduced in #15820 allows a custom crate to be imported in
the place of std, but with the name std. I don't think there is any
value to this. At most it is confusing users of a driver that uses this option. There are no users of
this option on github. If anyone still needs it, they can emulate it
injecting #![no_core] in addition to their own prelude.
Delete -Zquery-stats infrastructure
These statistics are computable from the self-profile data and/or ad-hoc collectable as needed, and in the meantime contribute to rustc bootstrap times -- locally, this PR shaves ~2.5% from rustc_query_impl builds in instruction counts.
If this does lose some functionality we want to keep, I think we should migrate it to self-profile (or a similar interface) rather than this ad-hoc reporting.
These statistics are computable from the self-profile data and/or ad-hoc
collectable as needed, and in the meantime contribute to rustc bootstrap times.
Stabilize `-Z instrument-coverage` as `-C instrument-coverage`
(Tracking issue for `instrument-coverage`: https://github.com/rust-lang/rust/issues/79121)
This PR stabilizes support for instrumentation-based code coverage, previously provided via the `-Z instrument-coverage` option. (Continue supporting `-Z instrument-coverage` for compatibility for now, but show a deprecation warning for it.)
Many, many people have tested this support, and there are numerous reports of it working as expected.
Move the documentation from the unstable book to stable rustc documentation. Update uses and documentation to use the `-C` option.
Addressing questions raised in the tracking issue:
> If/when stabilized, will the compiler flag be updated to -C instrument-coverage? (If so, the -Z variant could also be supported for some time, to ease migrations for existing users and scripts.)
This stabilization PR updates the option to `-C` and keeps the `-Z` variant to ease migration.
> The Rust coverage implementation depends on (and automatically turns on) -Z symbol-mangling-version=v0. Will stabilizing this feature depend on stabilizing v0 symbol-mangling first? If so, what is the current status and timeline?
This stabilization PR depends on https://github.com/rust-lang/rust/pull/90128 , which stabilizes `-C symbol-mangling-version=v0` (but does not change the default symbol-mangling-version).
> The Rust coverage implementation implements the latest version of LLVM's Coverage Mapping Format (version 4), which forces a dependency on LLVM 11 or later. A compiler error is generated if attempting to compile with coverage, and using an older version of LLVM.
Given that LLVM 13 has now been released, requiring LLVM 11 for coverage support seems like a reasonable requirement. If people don't have at least LLVM 11, nothing else breaks; they just can't use coverage support. Given that coverage support currently requires a nightly compiler and LLVM 11 or newer, allowing it on a stable compiler built with LLVM 11 or newer seems like an improvement.
The [tracking issue](https://github.com/rust-lang/rust/issues/79121) and the [issue label A-code-coverage](https://github.com/rust-lang/rust/labels/A-code-coverage) link to a few open issues related to `instrument-coverage`, but none of them seem like showstoppers. All of them seem like improvements and refinements we can make after stabilization.
The original `-Z instrument-coverage` support went through a compiler-team MCP at https://github.com/rust-lang/compiler-team/issues/278 . Based on that, `@pnkfelix` suggested that this needed a stabilization PR and a compiler-team FCP.
Currently, it can be `None` if the conversion from `OsString` fails, in
which case all searches will skip over the `SearchPathFile`.
The commit changes things so that the `SearchPathFile` just doesn't get
created in the first place. Same behaviour, but slightly simpler code.
This code and comment appear to be out of date.
`CrateLocator::find_library_crate` is the only caller of this function
and it handles rlib vs dylib overlap itself (see
`CrateLocator::extract_lib`) after inspecting all the files present, so
it doesn't need to see them in any particular order.
Stabilize `-Z print-link-args` as `--print link-args`
We have stable options for adding linker arguments; we should have a
stable option to help debug linker arguments.
Add documentation for the new option. In the documentation, make it clear that
the *exact* format of the output is not a stable guarantee.
Implement raw-dylib support for windows-gnu
Add support for `#[link(kind = "raw-dylib")]` on windows-gnu targets. Work around binutils's linker's inability to read import libraries produced by LLVM by calling out to the binutils `dlltool` utility to create an import library from a temporary .DEF file; this approach is effectively a slightly refined version of `@mati865's` earlier attempt at this strategy in PR #88801. (In particular, this attempt at this strategy adds support for `#[link_ordinal(...)]` as well.)
In support of #58713.
In #79570, `-Z split-dwarf-kind={none,single,split}` was replaced by `-C
split-debuginfo={off,packed,unpacked}`. `-C split-debuginfo`'s packed
and unpacked aren't exact parallels to single and split, respectively.
On Unix, `-C split-debuginfo=packed` will put debuginfo into object
files and package debuginfo into a DWARF package file (`.dwp`) and
`-C split-debuginfo=unpacked` will put debuginfo into dwarf object files
and won't package it.
In the initial implementation of Split DWARF, split mode wrote sections
which did not require relocation into a DWARF object (`.dwo`) file which
was ignored by the linker and then packaged those DWARF objects into
DWARF packages (`.dwp`). In single mode, sections which did not require
relocation were written into object files but ignored by the linker and
were not packaged. However, both split and single modes could be
packaged or not, the primary difference in behaviour was where the
debuginfo sections that did not require link-time relocation were
written (in a DWARF object or the object file).
This commit re-introduces a `-Z split-dwarf-kind` flag, which can be
used to pick between split and single modes when `-C split-debuginfo` is
used to enable Split DWARF (either packed or unpacked).
Signed-off-by: David Wood <david.wood@huawei.com>
These options primarily exist to work around bugs, and those bugs have
largely been fixed. Avoid stabilizing them, so that we don't have to
support them indefinitely.
Continue supporting -Z instrument-coverage for compatibility for now,
but show a deprecation warning for it.
Update uses and documentation to use the -C option.
Move the documentation from the unstable book to stable rustc
documentation.
This allows selecting `v0` symbol-mangling without an unstable option.
Selecting `legacy` still requires -Z unstable-options.
Continue supporting -Z symbol-mangling-version for compatibility for
now, but show a deprecation warning for it.
Import `SourceFile`s from crate before decoding foreign `Span`
Fixes#92163Fixes#92014
When writing to the incremental cache, we encode all `Span`s
we encounter, regardless of whether or not their `SourceFile`
comes from the local crate, or from a foreign crate.
When we decode a `Span`, we use the `StableSourceFileId` we encoded
to locate the matching `SourceFile` in the current session. If this
id corresponds to a `SourceFile` from another crate, then we need to
have already imported that `SourceFile` into our current session.
This usually happens automatically during resolution / macro expansion,
when we try to resolve definitions from other crates. In certain cases,
however, we may try to load a `Span` from a transitive dependency
without having ever imported the `SourceFile`s from that crate, leading
to an ICE.
This PR fixes the issue by enconding the `SourceFile`'s `CrateNum`
when we encode a `Span`. During decoding, we call `imported_source_files()`
when we encounter a foreign `CrateNum`, which ensure that all
`SourceFile`s from that crate are imported into the current session.
Add codegen option for branch protection and pointer authentication on AArch64
The branch-protection codegen option enables the use of hint-space pointer
authentication code for AArch64 targets.
Fixes#92163Fixes#92014
When writing to the incremental cache, we encode all `Span`s
we encounter, regardless of whether or not their `SourceFile`
comes from the local crate, or from a foreign crate.
When we decode a `Span`, we use the `StableSourceFileId` we encoded
to locate the matching `SourceFile` in the current session. If this
id corresponds to a `SourceFile` from another crate, then we need to
have already imported that `SourceFile` into our current session.
This usually happens automatically during resolution / macro expansion,
when we try to resolve definitions from other crates. In certain cases,
however, we may try to load a `Span` from a transitive dependency
without having ever imported the `SourceFile`s from that crate, leading
to an ICE.
This PR fixes the issue by calling `imported_source_files()`
when we encounter a `SourceFile` with a foreign `CrateNum`.
This ensures that all `SourceFile`s from that crate are imported
into the current session.
Enable `#[thread_local]` for all windows-msvc targets
As it stands, `#[thread_local]` is enabled haphazardly for msvc. It seems all 64-bit targets have it enabled, but not 32-bit targets unless they're also UWP targets (perhaps because UWP was added more recently?). So this PR simply enables it for 32-bit targets as well. I can't think of a reason not to and I've confirmed by running tests locally which pass.
See also #91659
Remove `SymbolStr`
This was originally proposed in https://github.com/rust-lang/rust/pull/74554#discussion_r466203544. As well as removing the icky `SymbolStr` type, it allows the removal of a lot of `&` and `*` occurrences.
Best reviewed one commit at a time.
r? `@oli-obk`
Add user seed to `-Z randomize-layout`
Allows users of -`Z randomize-layout` to provide `-Z layout-seed=<seed>` in order to further randomizing type layout randomization. Extension of [compiler-team/#457](https://github.com/rust-lang/compiler-team/issues/457), allows users to change struct layouts without changing code and hoping that item path hashes change, aiding in detecting layout errors
- Changed the separator from '+' to ','.
- Moved the branch protection options from -C to -Z.
- Additional test for incorrect branch-protection option.
- Remove LLVM < 12 code.
- Style fixes.
Co-authored-by: James McGregor <james.mcgregor2@arm.com>
Tokenize emoji as if they were valid identifiers
In the lexer, consider emojis to be valid identifiers and reject
them later to avoid knock down parse errors.
Partially address #86102.
LLVM has built-in heuristics for adding stack canaries to functions. These
heuristics can be selected with LLVM function attributes. This patch adds a
rustc option `-Z stack-protector={none,basic,strong,all}` which controls the use
of these attributes. This gives rustc the same stack smash protection support as
clang offers through options `-fno-stack-protector`, `-fstack-protector`,
`-fstack-protector-strong`, and `-fstack-protector-all`. The protection this can
offer is demonstrated in test/ui/abi/stack-protector.rs. This fills a gap in the
current list of rustc exploit
mitigations (https://doc.rust-lang.org/rustc/exploit-mitigations.html),
originally discussed in #15179.
Stack smash protection adds runtime overhead and is therefore still off by
default, but now users have the option to trade performance for security as they
see fit. An example use case is adding Rust code in an existing C/C++ code base
compiled with stack smash protection. Without the ability to add stack smash
protection to the Rust code, the code base artifacts could be exploitable in
ways not possible if the code base remained pure C/C++.
Stack smash protection support is present in LLVM for almost all the current
tier 1/tier 2 targets: see
test/assembly/stack-protector/stack-protector-target-support.rs. The one
exception is nvptx64-nvidia-cuda. This patch follows clang's example, and adds a
warning message printed if stack smash protection is used with this target (see
test/ui/stack-protector/warn-stack-protector-unsupported.rs). Support for tier 3
targets has not been checked.
Since the heuristics are applied at the LLVM level, the heuristics are expected
to add stack smash protection to a fraction of functions comparable to C/C++.
Some experiments demonstrating how Rust code is affected by the different
heuristics can be found in
test/assembly/stack-protector/stack-protector-heuristics-effect.rs. There is
potential for better heuristics using Rust-specific safety information. For
example it might be reasonable to skip stack smash protection in functions which
transitively only use safe Rust code, or which uses only a subset of functions
the user declares safe (such as anything under `std.*`). Such alternative
heuristics could be added at a later point.
LLVM also offers a "safestack" sanitizer as an alternative way to guard against
stack smashing (see #26612). This could possibly also be included as a
stack-protection heuristic. An alternative is to add it as a sanitizer (#39699).
This is what clang does: safestack is exposed with option
`-fsanitize=safe-stack`.
The options are only supported by the LLVM backend, but as with other codegen
options it is visible in the main codegen option help menu. The heuristic names
"basic", "strong", and "all" are hopefully sufficiently generic to be usable in
other backends as well.
Reviewed-by: Nikita Popov <nikic@php.net>
Extra commits during review:
- [address-review] make the stack-protector option unstable
- [address-review] reduce detail level of stack-protector option help text
- [address-review] correct grammar in comment
- [address-review] use compiler flag to avoid merging functions in test
- [address-review] specify min LLVM version in fortanix stack-protector test
Only for Fortanix test, since this target specifically requests the
`--x86-experimental-lvi-inline-asm-hardening` flag.
- [address-review] specify required LLVM components in stack-protector tests
- move stack protector option enum closer to other similar option enums
- rustc_interface/tests: sort debug option list in tracking hash test
- add an explicit `none` stack-protector option
Revert "set LLVM requirements for all stack protector support test revisions"
This reverts commit a49b74f92a4e7d701d6f6cf63d207a8aff2e0f68.
Try all stable method candidates first before trying unstable ones
Currently we try methods in this order in each step:
* Stable by value
* Unstable by value
* Stable autoref
* Unstable autoref
* ...
This PR changes it to first try pick methods without any unstable candidates, and if none is found, try again to pick unstable ones.
Fix#90320
CC #88971, hopefully would allow us to rename the "unstable_*" methods for integer impls back.
`@rustbot` label T-compiler T-libs-api
Leave -Z strip available temporarily as an alias, to avoid breaking
cargo until cargo transitions to using -C strip. (If the user passes
both, the -C version wins.)
Tweak the `options!` macro to allow for -Z and -C options with the same
name without generating conflicting internal parsing functions.
Split out of the commit stabilizing -Z strip as -C strip.
Added the --temps-dir option
Fixes#10971.
The new `--temps-dir` option puts intermediate files in a user-specified directory. This provides a fix for the issue where parallel invocations of rustc would overwrite each other's intermediate files.
No files are kept in the intermediate directory unless `-C save-temps=yes`.
If additional files are specifically requested using `--emit asm,llvm-bc,llvm-ir,obj,metadata,link,dep-info,mir`, these will be put in the output directory rather than the intermediate directory.
This is a backward-compatible change, i.e. if `--temps-dir` is not specified, the behavior is the same as before.
Don't abort compilation after giving a lint error
The only reason to use `abort_if_errors` is when the program is so broken that either:
1. later passes get confused and ICE
2. any diagnostics from later passes would be noise
This is never the case for lints, because the compiler has to be able to deal with `allow`-ed lints.
So it can continue to lint and compile even if there are lint errors.
Closes https://github.com/rust-lang/rust/issues/82761. This is a WIP because I have a feeling it will exit with 0 even if there were lint errors; I don't have a computer that can build rustc locally at the moment.
Add LLVM CFI support to the Rust compiler
This PR adds LLVM Control Flow Integrity (CFI) support to the Rust compiler. It initially provides forward-edge control flow protection for Rust-compiled code only by aggregating function pointers in groups identified by their number of arguments.
Forward-edge control flow protection for C or C++ and Rust -compiled code "mixed binaries" (i.e., for when C or C++ and Rust -compiled code share the same virtual address space) will be provided in later work as part of this project by defining and using compatible type identifiers (see Type metadata in the design document in the tracking issue #89653).
LLVM CFI can be enabled with -Zsanitizer=cfi and requires LTO (i.e., -Clto).
Thank you, `@eddyb` and `@pcc,` for all the help!
This commit adds LLVM Control Flow Integrity (CFI) support to the Rust
compiler. It initially provides forward-edge control flow protection for
Rust-compiled code only by aggregating function pointers in groups
identified by their number of arguments.
Forward-edge control flow protection for C or C++ and Rust -compiled
code "mixed binaries" (i.e., for when C or C++ and Rust -compiled code
share the same virtual address space) will be provided in later work as
part of this project by defining and using compatible type identifiers
(see Type metadata in the design document in the tracking issue #89653).
LLVM CFI can be enabled with -Zsanitizer=cfi and requires LTO (i.e.,
-Clto).
Add -Z no-unique-section-names to reduce ELF header bloat.
This change adds a new compiler flag that can help reduce the size of ELF binaries that contain many functions.
By default, when enabling function sections (which is the default for most targets), the LLVM backend will generate different section names for each function. For example, a function `func` would generate a section called `.text.func`. Normally this is fine because the linker will merge all those sections into a single one in the binary. However, starting with [LLVM 12](https://github.com/llvm/llvm-project/commit/ee5d1a04), the backend will also generate unique section names for exception handling, resulting in thousands of `.gcc_except_table.*` sections ending up in the final binary because some linkers like LLD don't currently merge or strip these EH sections (see discussion [here](https://reviews.llvm.org/D83655)). This can bloat the ELF headers and string table significantly in binaries that contain many functions.
The new option is analogous to Clang's `-fno-unique-section-names`, and instructs LLVM to generate the same `.text` and `.gcc_except_table` section for each function, resulting in a smaller final binary.
The motivation to add this new option was because we have a binary that ended up with so many ELF sections (over 65,000) that it broke some existing ELF tools, which couldn't handle so many sections.
Here's our old binary:
```
$ readelf --sections old.elf | head -1
There are 71746 section headers, starting at offset 0x2a246508:
$ readelf --sections old.elf | grep shstrtab
[71742] .shstrtab STRTAB 0000000000000000 2977204c ad44bb 00 0 0 1
```
That's an 11MB+ string table. Here's the new binary using this option:
```
$ readelf --sections new.elf | head -1
There are 43 section headers, starting at offset 0x29143ca8:
$ readelf --sections new.elf | grep shstrtab
[40] .shstrtab STRTAB 0000000000000000 29143acc 0001db 00 0 0 1
```
The whole binary size went down by over 20MB, which is quite significant.
Implement -Z location-detail flag
This PR implements the `-Z location-detail` flag as described in https://github.com/rust-lang/rfcs/pull/2091 .
`-Z location-detail=val` controls what location details are tracked when using `caller_location`. This allows users to control what location details are printed as part of panic messages, by allowing them to exclude any combination of filenames, line numbers, and column numbers. This option is intended to provide users with a way to mitigate the size impact of `#[track_caller]`.
Some measurements of the savings of this approach on an embedded binary can be found here: https://github.com/rust-lang/rust/issues/70579#issuecomment-942556822 .
Closes#70580 (unless people want to leave that open as a place for discussion of further improvements).
This is my first real PR to rust, so any help correcting mistakes / understanding side effects / improving my tests is appreciated :)
I have one question: RFC 2091 specified this as a debugging option (I think that is what -Z implies?). Does that mean this can never be stabilized without a separate MCP? If so, do I need to submit an MCP now, or is the initial RFC specifying this option sufficient for this to be merged as is, and then an MCP would be needed for eventual stabilization?
Report fatal lexer errors in `--cfg` command line arguments
Fixes#89358. The erroneous behavior was apparently introduced by `@Mark-Simulacrum` in a678e31911; the idea is to silence individual parser errors and instead emit one catch-all error message after parsing. However, for the example in #89358, a fatal lexer error is created here:
edebf77e00/compiler/rustc_parse/src/lexer/mod.rs (L340-L349)
This fatal error aborts the compilation, and so the call to `new_parser_from_source_str()` never returns and the catch-all error message is never emitted. I have therefore changed the `SilentEmitter` to silence only non-fatal errors; with my changes, for the rustc invocation described in #89358:
```sh
rustc --cfg "abc\""
```
I get the following output:
```
error[E0765]: unterminated double quote string
|
= note: this error occurred on the command line: `--cfg=abc"`
```
Add support for artifact size profiling
This adds support for profiling artifact file sizes (incremental compilation artifacts and query cache to begin with).
Eventually we want to track this in perf.rlo so we can ensure that file sizes do not change dramatically on each pull request.
This relies on support in measureme: https://github.com/rust-lang/measureme/pull/169. Once that lands we can update this PR to not point to a git dependency.
This was worked on together with `@michaelwoerister.`
r? `@wesleywiser`
This change adds a new compiler flag that can help reduce the size of
ELF binaries that contain many functions.
By default, when enabling function sections (which is the default for most
targets), the LLVM backend will generate different section names for each
function. For example, a function "func" would generate a section called
".text.func". Normally this is fine because the linker will merge all those
sections into a single one in the binary. However, starting with LLVM 12
(llvm/llvm-project@ee5d1a0), the backend will
also generate unique section names for exception handling, resulting in
thousands of ".gcc_except_table.*" sections ending up in the final binary
because some linkers don't currently merge or strip these EH sections.
This can bloat the ELF headers and string table significantly in
binaries that contain many functions.
The new option is analogous to Clang's -fno-unique-section-names, and
instructs LLVM to generate the same ".text" and ".gcc_except_table"
section for each function, resulting in smaller object files and
potentially a smaller final binary.
Correct decoding of foreign expansions during incr. comp.
Fixes https://github.com/rust-lang/rust/issues/74946
The original issue was due to a wrong assertion in `expn_hash_to_expn_id`.
The secondary issue was due to a mismatch between the encoding and decoding paths for expansions that are created after the TyCtxt is created.
This largely involves implementing the options debug-info-for-profiling
and profile-sample-use and forwarding them on to LLVM.
AutoFDO can be used on x86-64 Linux like this:
rustc -O -Cdebug-info-for-profiling main.rs -o main
perf record -b ./main
create_llvm_prof --binary=main --out=code.prof
rustc -O -Cprofile-sample-use=code.prof main.rs -o main2
Now `main2` will have feedback directed optimization applied to it.
The create_llvm_prof tool can be obtained from this github repository:
https://github.com/google/autofdoFixes#64892.
