This commit adds LLVM Kernel Control Flow Integrity (KCFI) support to
the Rust compiler. It initially provides forward-edge control flow
protection for operating systems kernels for Rust-compiled code only by
aggregating function pointers in groups identified by their return and
parameter types. (See llvm/llvm-project@cff5bef.)
Forward-edge control flow protection for C or C++ and Rust -compiled
code "mixed binaries" (i.e., for when C or C++ and Rust -compiled code
share the same virtual address space) will be provided in later work as
part of this project by identifying C char and integer type uses at the
time types are encoded (see Type metadata in the design document in the
tracking issue #89653).
LLVM KCFI can be enabled with -Zsanitizer=kcfi.
Co-authored-by: bjorn3 <17426603+bjorn3@users.noreply.github.com>
Detect long types in E0308 and write them to disk
On type error with long types, print an abridged type and write the full type to disk.
Print the widest possible short type while still fitting in the terminal.
Add -Z maximal-hir-to-mir-coverage flag
This PR adds a new unstable flag `-Z maximal-hir-to-mir-coverage` that changes the behavior of `maybe_lint_level_root_bounded`, pursuant to [a discussion on Zulip](https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/Mapping.20MIR.20to.20HIR). When enabled, this function will not search upwards for a lint root, but rather immediately return the provided HIR node ID. This change increases the granularity of the mapping between MIR locations and HIR nodes inside the `SourceScopeLocalData` data structures. This increase in granularity is useful for rustc consumers like [Flowistry](https://github.com/willcrichton/flowistry) that rely on getting source-mapping information about the MIR CFG that is as precise as possible.
A test `maximal_mir_to_hir_coverage.rs` has been added to verify that this flag does not break anything.
r? `@cjgillot`
cc `@gavinleroy`
Add StableOrd trait as proposed in MCP 533.
The `StableOrd` trait can be used to mark types as having a stable sort order across compilation sessions. Collections that sort their items in a stable way can safely implement HashStable by hashing items in sort order.
See https://github.com/rust-lang/compiler-team/issues/533 for more information.
Remove useless borrows and derefs
They are nothing more than noise.
<sub>These are not all of them, but my clippy started crashing (stack overflow), so rip :(</sub>
The StableOrd trait can be used to mark types as having a stable
sort order across compilation sessions. Collections that sort their
items in a stable way can safely implement HashStable by
hashing items in sort order.
On type error with long types, print an abridged type and write the full
type to disk.
Print the widest possible short type while still fitting in the
terminal.
make `error_reported` check for delayed bugs
Fixes#104768
`error_reported()` was only checking if there were errors emitted, not for `delay_bug`s which can also be a source of `ErrorGuaranteed`. I assume the same is true of `lint_err_count` but i dont know
Use `as_deref` in compiler (but only where it makes sense)
This simplifies some code :3
(there are some changes that are not exacly `as_deref`, but more like "clever `Option`/`Result` method use")
Instead of `ast::Lit`.
Literal lowering now happens at two different times. Expression literals
are lowered when HIR is crated. Attribute literals are lowered during
parsing.
This commit changes the language very slightly. Some programs that used
to not compile now will compile. This is because some invalid literals
that are removed by `cfg` or attribute macros will no longer trigger
errors. See this comment for more details:
https://github.com/rust-lang/rust/pull/102944#issuecomment-1277476773
improve `filesearch::get_or_default_sysroot`
`fn get_or_default_sysroot` is now improved and used in `miri` and `clippy`, and tests are still passing as they should. So we no longer need to implement custom workarounds/hacks to find sysroot in tools like miri/clippy.
Resolves https://github.com/rust-lang/rust/issues/98832
re-opened from #103581
Allow use of `-Clto=thin` with `-Ccodegen-units=1` in general
The current logic to ignore ThinLTO when `-Ccodegen-units=1` makes sense for local ThinLTO but even in this scenario, a user may still want (non-local) ThinLTO for the purpose of optimizing dependencies into the final crate which is being compiled with 1 CGU.
The previous behavior was even more confusing because if you were generating a binary (`--emit=link`), then you would get ThinLTO but if you asked for LLVM IR or bytecode, then it would silently change to using regular LTO.
With this change, we only override the defaults for local ThinLTO if you ask for a single output such as LLVM IR or bytecode and in all other cases honor the requested LTO setting.
r? `@michaelwoerister`
Print valid `--print` requests if request is invalid
When someone makes a typo, it can be useful to see the valid options. This is also useful if someone wants to find out about all the options.
Track where diagnostics were created.
This implements the `-Ztrack-diagnostics` flag, which uses `#[track_caller]` to track where diagnostics are created. It is meant as a debugging tool much like `-Ztreat-err-as-bug`.
For example, the following code...
```rust
struct A;
struct B;
fn main(){
let _: A = B;
}
```
...now emits the following error message:
```
error[E0308]: mismatched types
--> src\main.rs:5:16
|
5 | let _: A = B;
| - ^ expected struct `A`, found struct `B`
| |
| expected due to this
-Ztrack-diagnostics: created at compiler\rustc_infer\src\infer\error_reporting\mod.rs:2275:31
```
The current logic to ignore ThinLTO when `-Ccodegen-units=1` makes sense
for local ThinLTO but even in this scenario, a user may still want
(non-local) ThinLTO for the purpose of optimizing dependencies into the
final crate which is being compiled with 1 CGU.
The previous behavior was even more confusing because if you were
generating a binary (`--emit=link`), then you would get ThinLTO but if
you asked for LLVM IR or bytecode, then it would silently change to
using regular LTO.
With this change, we only override the defaults for local ThinLTO if you
ask for a single output such as LLVM IR or bytecode and in all other
cases honor the requested LTO setting.
Get rid of native_library projection queries
They don't seem particularly useful as I don't expect native libraries to change frequently.
Maybe they do provide significant value of keeping incremental compilation green though, I'm not sure.
Change process spawning to inherit the parent's signal mask by default
Previously, the signal mask was always reset when a child process is
started. This breaks tools like `nohup` which expect `SIGHUP` to be
blocked for all transitive processes.
With this change, the default behavior changes to inherit the signal mask.
This also changes the signal disposition for `SIGPIPE` to only be changed if the `#[unix_sigpipe]` attribute isn't set.
translation: doc comments with derives, subdiagnostic-less enum variants, more derive use
- Adds support for `doc` attributes in the diagnostic derives so that documentation comments don't result in the derive failing.
- Adds support for enum variants in the subdiagnostic derive to not actually correspond to an addition to a diagnostic.
- Made use of the derive in more places in the `rustc_ast_lowering`, `rustc_ast_passes`, `rustc_lint`, `rustc_session`, `rustc_infer` - taking advantage of recent additions like eager subdiagnostics, multispan suggestions, etc.
cc #100717
Previously, the signal mask is always reset when a child process is
started. This breaks tools like `nohup` which expect `SIGHUP` to be
blocked.
With this change, the default behavior changes to inherit the signal mask.
This also changes the signal disposition for `SIGPIPE` to only be
changed if the `#[unix_sigpipe]` attribute isn't set.
More dupe word typos
I only picked those changes (from the regex search) that I am pretty certain doesn't change meaning and is just a typo fix. Do correct me if any fix is undesirable and I can revert those. Thanks.
Add `Noted` marker struct that implements `EmissionGuarantee` so that
`emit_note` and `create_note` can be implemented for struct diagnostics.
Signed-off-by: David Wood <david.wood@huawei.com>
The compiler currently has `-Ztime` and `-Ztime-passes`. I've used
`-Ztime-passes` for years but only recently learned about `-Ztime`.
What's the difference? Let's look at the `-Zhelp` output:
```
-Z time=val -- measure time of rustc processes (default: no)
-Z time-passes=val -- measure time of each rustc pass (default: no)
```
The `-Ztime-passes` description is clear, but the `-Ztime` one is less so.
Sounds like it measures the time for the entire process?
No. The real difference is that `-Ztime-passes` prints out info about passes,
and `-Ztime` does the same, but only for a subset of those passes. More
specifically, there is a distinction in the profiling code between a "verbose
generic activity" and an "extra verbose generic activity". `-Ztime-passes`
prints both kinds, while `-Ztime` only prints the first one. (It took me
a close reading of the source code to determine this difference.)
In practice this distinction has low value. Perhaps in the past the "extra
verbose" output was more voluminous, but now that we only print stats for a
pass if it exceeds 5ms or alters the RSS, `-Ztime-passes` is less spammy. Also,
a lot of the "extra verbose" cases are for individual lint passes, and you need
to also use `-Zno-interleave-lints` to see those anyway.
Therefore, this commit removes `-Ztime` and the associated machinery. One thing
to note is that the existing "extra verbose" activities all have an extra
string argument, so the commit adds the ability to accept an extra argument to
the "verbose" activities.
Migrate more of rustc_parse to SessionDiagnostic
Still far from complete, but I thought I'd add a checkpoint here because rebasing was starting to get annoying.
In #101230, the internal diagnostic migration lints -
`diagnostic_outside_of_impl` and `untranslatable_diagnostic` - were
modified so that they wouldn't trigger on functions annotated with
`#[rustc_lint_diagnostics]`. However, this change has to make it into
the bootstrap compiler before the `#[allow]` annotations that it aims to
remove can be removed, which is possible now that #102051 has landed.
Signed-off-by: David Wood <david.wood@huawei.com>
Remove support for legacy PM
This removes support for optimizing with LLVM's legacy pass manager, as well as the unstable `-Znew-llvm-pass-manager` option. We have been defaulting to the new PM since LLVM 13 (except for s390x that waited for 14), and LLVM 15 removed support altogether. The only place we still use the legacy PM is for writing the output file, just like `llc` does.
cc #74705
r? ``@nikic``
FIX - ambiguous Diagnostic link in docs
UPDATE - rename diagnostic_items to IntoDiagnostic and AddToDiagnostic
[Gardening] FIX - formatting via `x fmt`
FIX - rebase conflicts. NOTE: Confirm wheather or not we want to handle TargetDataLayoutErrorsWrapper this way
DELETE - unneeded allow attributes in Handler method
FIX - broken test
FIX - Rebase conflict
UPDATE - rename residual _SessionDiagnostic and fix LintDiag link
On later stages, the feature is already stable.
Result of running:
rg -l "feature.let_else" compiler/ src/librustdoc/ library/ | xargs sed -s -i "s#\\[feature.let_else#\\[cfg_attr\\(bootstrap, feature\\(let_else\\)#"
Avoid `Iterator::last`
Adapters like `Filter` and `Map` use the default implementation of `Iterator::last` which is not short-circuiting (and so does `core::str::Split`). The predicate function will be run for every single item of the underlying iterator. I hope that removing those calls to `last` results in slight performance improvements.
change rlib format to distinguish native dependencies
Another one method to solve problem mentioned in #99429.
Changed .rlib format, it contains all bundled native libraries as archieves.
At link time rlib is unpacked and native dependencies linked separately.
New behavior hidden under separate_native_rlib_dependencies flag.
Add inline-llvm option for disabling/enabling LLVM inlining
In this PR, a new -Z option `inline-llvm` is added in order to be able to turn on/off LLVM inlining.
The capability of turning on/off inlining in LLVM backend is needed for testing performance implications of using recently enabled inlining in rustc's frontend (with -Z inline-mir=yes option, #91743). It would be interesting to see the performance effect using rustc's frontend inlining only without LLVM inlining enabled. Currently LLVM is still doing inlining no mater what value inline-mir is set to. With the option `inline-llvm` being added in this PR, user can turn off LLVM inlining by using `-Z inline-llvm=no` option (the default of inline-llvm is 'yes', LLVM inlining enabled).
This commit removes the allows rules for the SessionDiagnostic lint
that were being used in the session.rs file.
Thanks to the PR #101230 we do not need to annotate the methods with
the allow rule as they are part of the diagnostic machinery.
Suggested by the team in this Zulip Topic https://rust-lang.zulipchat.com/#narrow/stream/336883-i18n/topic/.23100717.20SessionDiagnostic.20on.20Handler
Handler already has almost all the capabilities of ParseSess when it comes to diagnostic emission, in this migration we only needed to add the ability to access source_map from the emitter in order to get a Snippet and the start_point. Not sure if this is the best way to address this gap
Add warning against unexpected --cfg with --check-cfg
This PR adds a warning when an unexpected `--cfg` is specified but not in the specified list of `--check-cfg`.
This is the follow-up PR I mentioned in https://github.com/rust-lang/rust/pull/99519.
r? `@petrochenkov`
Support `#[unix_sigpipe = "inherit|sig_dfl"]` on `fn main()` to prevent ignoring `SIGPIPE`
When enabled, programs don't have to explicitly handle `ErrorKind::BrokenPipe` any longer. Currently, the program
```rust
fn main() { loop { println!("hello world"); } }
```
will print an error if used with a short-lived pipe, e.g.
% ./main | head -n 1
hello world
thread 'main' panicked at 'failed printing to stdout: Broken pipe (os error 32)', library/std/src/io/stdio.rs:1016:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
by enabling `#[unix_sigpipe = "sig_dfl"]` like this
```rust
#![feature(unix_sigpipe)]
#[unix_sigpipe = "sig_dfl"]
fn main() { loop { println!("hello world"); } }
```
there is no error, because `SIGPIPE` will not be ignored and thus the program will be killed appropriately:
% ./main | head -n 1
hello world
The current libstd behaviour of ignoring `SIGPIPE` before `fn main()` can be explicitly requested by using `#[unix_sigpipe = "sig_ign"]`.
With `#[unix_sigpipe = "inherit"]`, no change at all is made to `SIGPIPE`, which typically means the behaviour will be the same as `#[unix_sigpipe = "sig_dfl"]`.
See https://github.com/rust-lang/rust/issues/62569 and referenced issues for discussions regarding the `SIGPIPE` problem itself
See the [this](https://rust-lang.zulipchat.com/#narrow/stream/219381-t-libs/topic/Proposal.3A.20First.20step.20towards.20solving.20the.20SIGPIPE.20problem) Zulip topic for more discussions, including about this PR.
Tracking issue: https://github.com/rust-lang/rust/issues/97889
rustc_target: Add a compatibility layer to separate internal and user-facing linker flavors
I want to do some refactorings in `rustc_target` - merge `lld_flavor` and `linker_is_gnu` into `linker_flavor`, support combination gcc+lld (https://github.com/rust-lang/rust/pull/96827).
This PR adds some compatibility infra that makes that possible without making any changes to user-facing interfaces - `-Clinker-flavor` values and json target specs. (For json target specs this infra may eventually go away since they are not very stable.)
The second commit does some light refactoring of internal linker flavors (applies changes from 53eca42973 that don't require mass-editing target specs).
translations(rustc_session): migrates `rustc_session` to use `SessionDiagnostic` - Pt. 1
## Description
This is the first PR for the migration of the module `rustc_session`. You can follow my progress [here](https://github.com/rust-lang/rust/issues/100717#issuecomment-1220279883).
The PR migrates the files `cgu_reuse_tracker` and `parse.rs` to use `SessionDiagnostic `.
This makes it possible to instruct libstd to never touch the signal
handler for `SIGPIPE`, which makes programs pipeable by default (e.g.
with `./your-program | head -n 1`) without `ErrorKind::BrokenPipe`
errors.
Implementation of import_name_type
Fixes#96534 by implementing https://github.com/rust-lang/compiler-team/issues/525
Symbols that are exported or imported from a binary on 32bit x86 Windows can be named in four separate ways, corresponding to the [import name types](https://docs.microsoft.com/en-us/windows/win32/debug/pe-format#import-name-type) from the PE-COFF spec. The exporting and importing binaries must use the same name encoding, otherwise mismatches can lead to link failures due to "missing symbols" or to 0xc0000139 (`STATUS_ENTRYPOINT_NOT_FOUND`) errors when the executable/library is loaded. For details, see the comments on the raw-dylib feature's https://github.com/rust-lang/rust/issues/58713. To generate the correct import libraries for these DLLs, therefore, rustc must know the import name type for each `extern` function, and there is currently no way for users to provide this information.
This change adds a new `MetaNameValueStr` key to the `#[link]` attribute called `import_name_type`, and which accepts one of three values: `decorated`, `noprefix`, and `undecorated`.
A single DLL is likely to export all its functions using the same import type name, hence `import_name_type` is a parameter of `#[link]` rather than being its own attribute that is applied per-function. It is possible to have a single DLL that exports different functions using different import name types, but users could express such cases by providing multiple export blocks for the same DLL, each with a different import name type.
Note: there is a fourth import name type defined in the PE-COFF spec, `IMPORT_ORDINAL`. This case is already handled by the `#[link_ordinal]` attribute. While it could be merged into `import_type_name`, that would not make sense as `#[link_ordinal]` provides per-function information (namely the ordinal itself).
Design decisions (these match the MCP linked above):
* For GNU, `decorated` matches the PE Spec and MSVC rather than the default behavior of `dlltool` (i.e., there will be a leading `_` for `stdcall`).
* If `import_name_type` is not present, we will keep our current behavior of matching the environment (MSVC vs GNU) default for decorating.
* Using `import_name_type` on architectures other than 32bit x86 will result in an error.
* Using `import_name_type` with link kinds other than `"raw-dylib"` will result in an error.
Diagnostics migr const eval
This PR should eventually contain all diagnostic migrations for the `rustc_const_eval` crate.
r? `@davidtwco`
`@rustbot` label +A-translation
session: stabilize split debuginfo on linux
Stabilize the `-Csplit-debuginfo` flag...
- ...on Linux for all values of the flag. Split DWARF has been implemented for a few months, hasn't had any bug reports and has had some promising benchmarking for incremental debug build performance.
- ..on other platforms for the default value. It doesn't make any sense that `-Csplit-debuginfo=packed` is unstable on Windows MSVC when that's the default behaviour, but keep the other values unstable.
This commit migrates the errors in the function check_expected_reuse
to use the new SessionDiagnostic. It also does some small refactor
for the IncorrectCguReuseType to include the 'at least' word in the
fluent translation file
Stabilize the `-Csplit-debuginfo` flag...
- ...on Linux for all values of the flag. Split DWARF has been
implemented for a few months, hasn't had any bug reports and has had
some promising benchmarking for incremental debug build performance.
- ..on other platforms for the default value. It doesn't make any sense
that `-Csplit-debuginfo=packed` is unstable on Windows MSVC when
that's the default behaviour, but keep the other values unstable.
Signed-off-by: David Wood <david.wood@huawei.com>
`-Z location-detail`: provide option to disable all location details
As reported [here](https://github.com/rust-lang/rust/pull/89920#issuecomment-1190598924), when I first implemented the `-Z location-detail` flag there was a bug, where passing an empty list was not correctly supported, and instead rejected by the compiler. This PR fixes that such that passing an empty list results in no location details being tracked, as originally specified in https://github.com/rust-lang/rfcs/pull/2091 .
This PR also adds a test case to verify that this option continues to work as intended.
Remove implicit names and values from `--cfg` in `--check-cfg`
This PR remove the implicit names and values from `--cfg` in `--check-cfg` because the behavior is quite surprising but also because it's really easy to inadvertently really on the implicitness and when the `--cfg` is not set anymore to have an unexpected warning from an unexpected condition that pass with the implicitness.
This change in behavior will also enable us to warn when an unexpected `--cfg` is passed, ex: the user wrote `--cfg=unstabl` instead of `--cfg=unstable`. The implementation of the warning will be done in a follow-up PR.
cc `@petrochenkov`
proc_macro: use crossbeam channels for the proc_macro cross-thread bridge
This is done by having the crossbeam dependency inserted into the `proc_macro` server code from the server side, to avoid adding a dependency to `proc_macro`.
In addition, this introduces a -Z command-line option which will switch rustc to run proc-macros using this cross-thread executor. With the changes to the bridge in #98186, #98187, #98188 and #98189, the performance of the executor should be much closer to same-thread execution.
In local testing, the crossbeam executor was substantially more performant than either of the two existing `CrossThread` strategies, so they have been removed to keep things simple.
r? `@eddyb`
This is done by having the crossbeam dependency inserted into the
proc_macro server code from the server side, to avoid adding a
dependency to proc_macro.
In addition, this introduces a -Z command-line option which will switch
rustc to run proc-macros using this cross-thread executor. With the
changes to the bridge in #98186, #98187, #98188 and #98189, the
performance of the executor should be much closer to same-thread
execution.
In local testing, the crossbeam executor was substantially more
performant than either of the two existing CrossThread strategies, so
they have been removed to keep things simple.
Prior to this fix, `-Z location-detail` provided no mechanism for
disabling all location details. This commit also adds a test case
to verify that this option continues to work as intended, and
clarifies the documentation of this option.
Some command-line options accessible through `sess.opts` are best
accessed through wrapper functions on `Session`, `TyCtxt` or otherwise,
rather than through field access on the option struct in the `Session`.
Adds a new lint which triggers on those options that should be accessed
through a wrapper function so that this is prohibited. Options are
annotated with a new attribute `rustc_lint_opt_deny_field_access` which
can specify the error message (i.e. "use this other function instead")
to be emitted.
A simpler alternative would be to simply rename the options in the
option type so that it is clear they should not be used, however this
doesn't prevent uses, just discourages them. Another alternative would
be to make the option fields private, and adding accessor functions on
the option types, however the wrapper functions sometimes rely on
additional state from `Session` or `TyCtxt` which wouldn't be available
in an function on the option type, so the accessor would simply make the
field available and its use would be discouraged too.
Signed-off-by: David Wood <david.wood@huawei.com>
If an internal lint uses `typeck_results` or similar queries then that
can result in rustdoc checking code that it shouldn't (e.g. from other
platforms) and emit compilation errors.
Signed-off-by: David Wood <david.wood@huawei.com>
Add support for LLVM ShadowCallStack.
LLVMs ShadowCallStack provides backward edge control flow integrity protection by using a separate shadow stack to store and retrieve a function's return address.
LLVM currently only supports this for AArch64 targets. The x18 register is used to hold the pointer to the shadow stack, and therefore this only works on ABIs which reserve x18. Further details are available in the [LLVM ShadowCallStack](https://clang.llvm.org/docs/ShadowCallStack.html) docs.
# Usage
`-Zsanitizer=shadow-call-stack`
# Comments/Caveats
* Currently only enabled for the aarch64-linux-android target
* Requires the platform to define a runtime to initialize the shadow stack, see the [LLVM docs](https://clang.llvm.org/docs/ShadowCallStack.html) for more detail.
Allow to disable thinLTO buffer to support lto-embed-bitcode lld feature
Hello
This change is to fix issue (https://github.com/rust-lang/rust/issues/84395) in which passing "-lto-embed-bitcode=optimized" to lld when linking rust code via linker-plugin-lto doesn't produce the expected result.
Instead of emitting a single unified module into a llvmbc section of the linked elf, it emits multiple submodules.
This is caused because rustc emits the BC modules after running llvm `createWriteThinLTOBitcodePass` pass.
Which in turn triggers a thinLTO linkage and causes the said issue.
This patch allows via compiler flag (-Cemit-thin-lto=<bool>) to select between running `createWriteThinLTOBitcodePass` and `createBitcodeWriterPass`.
Note this pattern of selecting between those 2 passes is common inside of LLVM code.
The default is to match the old behavior.
Adding the option to control from rustc CLI
if the resulted ".o" bitcode module files are with
thinLTO info or regular LTO info.
Allows using "-lto-embed-bitcode=optimized" during linkage
correctly.
Signed-off-by: Ziv Dunkelman <ziv.dunkelman@nextsilicon.com>
Keep unstable target features for asm feature checking
Inline assembly uses the target features to determine which registers
are available on the current target. However it needs to be able to
access unstable target features for this.
Fixes#99071
Inline assembly uses the target features to determine which registers
are available on the current target. However it needs to be able to
access unstable target features for this.
Fixes#99071
DWARF version 5 brings a number of improvements over version 4. Quoting from
the announcement [1]:
> Version 5 incorporates improvements in many areas: better data compression,
> separation of debugging data from executable files, improved description of
> macros and source files, faster searching for symbols, improved debugging
> optimized code, as well as numerous improvements in functionality and
> performance.
On platforms where DWARF version 5 is supported (Linux, primarily), this commit
adds support for it behind a new `-Z dwarf-version=5` flag.
[1]: https://dwarfstd.org/Public_Review.php
Migrate some diagnostics from `rustc_const_eval` to `SessionDiagnostic`
I'm still trying to get the hang of this, so it doesn't migrate _all_ of `rustc_const_eval`. Working on that later.
r? `@davidtwco`
Add a `-Zdump-drop-tracking-cfg` debugging flag
This is useful for debugging drop-tracking; previously, you had to recompile
rustc from source and manually add a call to `write_graph_to_file`. This
makes the option more discoverable and configurable at runtime.
I also took the liberty of making the labels for the CFG nodes much easier to read:
previously, they looked like `id(2), local_id: 48`, now they look like
```
expr from_config (hir_id=HirId { owner: DefId(0:10 ~ default_struct_update[79f9]::foo), local_id: 2})
```
r? ``@eholk``
Rename the `--output-width` flag to `--diagnostic-width` as this appears
to be the preferred name within the compiler team.
Signed-off-by: David Wood <david.wood@huawei.com>
Rename the `--terminal-width` flag to `--output-width` as the behaviour
doesn't just apply to terminals (and so is slightly less accurate).
Signed-off-by: David Wood <david.wood@huawei.com>
Formerly `-Zterminal-width`, `--terminal-width` allows the user or build
tool to inform rustc of the width of the terminal so that diagnostics
can be truncated.
Signed-off-by: David Wood <david.wood@huawei.com>
The `rustc_lint_diagnostics` attribute is used by the diagnostic
translation/struct migration lints to identify calls where
non-translatable diagnostics or diagnostics outwith impls are being
created. Any function used in creating a diagnostic should be annotated
with this attribute so this commit adds the attribute to many more
functions.
Signed-off-by: David Wood <david.wood@huawei.com>
This is useful for debugging drop-tracking; previously, you had to recompile
rustc from source and manually add a call to `write_graph_to_file`. This
makes the option more discoverable and configurable at runtime.
I also took the liberty of making the labels for the CFG nodes much easier to read:
previously, they looked like `id(2), local_id: 48`, now they look like
```
expr from_config (hir_id=HirId { owner: DefId(0:10 ~ default_struct_update[79f9]::foo), local_id: 2})
```
Support lint expectations for `--force-warn` lints (RFC 2383)
Rustc has a `--force-warn` flag, which overrides lint level attributes and forces the diagnostics to always be warn. This means, that for lint expectations, the diagnostic can't be suppressed as usual. This also means that the expectation would not be fulfilled, even if a lint had been triggered in the expected scope.
This PR now also tracks the expectation ID in the `ForceWarn` level. I've also made some minor adjustments, to possibly catch more bugs and make the whole implementation more robust.
This will probably conflict with https://github.com/rust-lang/rust/pull/97718. That PR should ideally be reviewed and merged first. The conflict itself will be trivial to fix.
---
r? `@wesleywiser`
cc: `@flip1995` since you've helped with the initial review and also discussed this topic with me. 🙃
Follow-up of: https://github.com/rust-lang/rust/pull/87835
Issue: https://github.com/rust-lang/rust/issues/85549
Yeah, and that's it.
Integrate measureme's hardware performance counter support.
*Note: this is a companion to https://github.com/rust-lang/measureme/pull/143, and duplicates some information with it for convenience*
**(much later) EDIT**: take any numbers with a grain of salt, they may have changed since initial PR open.
## Credits
I'd like to start by thanking `@alyssais,` `@cuviper,` `@edef1c,` `@glandium,` `@jix,` `@Mark-Simulacrum,` `@m-ou-se,` `@mystor,` `@nagisa,` `@puckipedia,` and `@yorickvP,` for all of their help with testing, and valuable insight and suggestions.
Getting here wouldn't have been possible without you!
(If I've forgotten anyone please let me know, I'm going off memory here, plus some discussion logs)
## Summary
This PR adds support to `-Z self-profile` for counting hardware events such as "instructions retired" (as opposed to being limited to time measurements), using the `rdpmc` instruction on `x86_64` Linux.
While other OSes may eventually be supported, preliminary research suggests some kind of kernel extension/driver is required to enable this, whereas on Linux any user can profile (at least) their own threads.
Supporting Linux on architectures other than x86_64 should be much easier (provided the hardware supports such performance counters), and was mostly not done due to a lack of readily available test hardware.
That said, 32-bit `x86` (aka `i686`) would be almost trivial to add and test once we land the initial `x86_64` version (as all the CPU detection code can be reused).
A new flag `-Z self-profile-counter` was added, to control which of the named `measureme` counters is used, and which defaults to `wall-time`, in order to keep `-Z self-profile`'s current functionality unchanged (at least for now).
The named counters so far are:
* `wall-time`: the existing time measurement
* name chosen for consistency with `perf.rust-lang.org`
* continues to use `std::time::Instant` for a nanosecond-precision "monotonic clock"
* `instructions:u`: the hardware performance counter usually referred to as "Instructions retired"
* here "retired" (roughly) means "fully executed"
* the `:u` suffix is from the Linux `perf` tool and indicates the counter only runs while userspace code is executing, and therefore counts no kernel instructions
* *see [Caveats/Subtracting IRQs](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Subtracting-IRQs) for why this isn't entirely true and why `instructions-minus-irqs:u` should be preferred instead*
* `instructions-minus-irqs:u`: same as `instructions:u`, except the count of hardware interrupts ("IRQs" here for brevity) is subtracted
* *see [Caveats/Subtracting IRQs](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Subtracting-IRQs) for why this should be preferred over `instructions:u`*
* `instructions-minus-r0420:u`: experimental counter, same as `instructions-minus-irqs:u` but subtracting an undocumented counter (`r0420:u`) instead of IRQs
* the `rXXXX` notation is again from Linux `perf`, and indicates a "raw" counter, with a hex representation of the low-level counter configuration - this was picked because we still don't *really* know what it is
* this only exists for (future) testing and isn't included/used in any comparisons/data we've put together so far
* *see [Challenges/Zen's undocumented 420 counter](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Epilogue-Zen’s-undocumented-420-counter) for details on how this counter was found and what it does*
---
There are also some additional commits:
* ~~see [Challenges/Rebasing *shouldn't* affect the results, right?](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Rebasing-*shouldn’t*-affect-the-results,-right) for details on the changes to `rustc_parse` and `rustc_trait_section` (the latter far more dubious, and probably shouldn't be merged, or not as-is)~~
* **EDIT**: the effects of these are no long quantifiable, the PR includes reverts for them
* ~~see [Challenges/`jemalloc`: purging will commence in ten seconds](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#jemalloc-purging-will-commence-in-ten-seconds) for details on the `jemalloc` change~~
* this is also separately found in #77162, and we probably want to avoid doing it by default, ideally we'd use the runtime control API `jemalloc` offers (assuming that can stop the timer that's already running, which I'm not sure about)
* **EDIT**: until we can do this based on `-Z` flags, this commit has also been reverted
* the `proc_macro` change was to avoid randomized hashing and therefore ASLR-like effects
---
**(much later) EDIT**: take any numbers with a grain of salt, they may have changed since initial PR open.
#### Write-up / report
Because of how extensive the full report ended up being, I've kept most of it [on `hackmd.io`](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view), but for convenient access, here are all the sections (with individual links):
<sup>(someone suggested I'd make a backup, so [here it is on the wayback machine](http://web.archive.org/web/20201127164748/https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view) - I'll need to remember to update that if I have to edit the write-up)</sup>
* [**Motivation**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Motivation)
* [**Results**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Results)
* [**Overhead**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Overhead)
*Preview (see the report itself for more details):*
|Counter|Total<br>`instructions-minus-irqs:u`|Overhead from "Baseline"<br>(for all 1903881<br>counter reads)|Overhead from "Baseline"<br>(per each counter read)|
|-|-|-|-|
|Baseline|63637621286 ±6||
|`instructions:u`|63658815885 ±2| +21194599 ±8| +11|
|`instructions-minus-irqs:u`|63680307361 ±13| +42686075 ±19| +22|
|`wall-time`|63951958376 ±10275|+314337090 ±10281|+165|
* [**"Macro" noise (self time)**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#“Macro”-noise-(self-time))
*Preview (see the report itself for more details):*
|| `wall-time` (ns) | `instructions:u` | `instructions-minus-irqs:u`
-: | -: | -: | -:
`typeck` | 5478261360 ±283933373 (±~5.2%) | 17350144522 ±6392 (±~0.00004%) | 17351035832.5 ±4.5 (±~0.00000003%)
`expand_crate` | 2342096719 ±110465856 (±~4.7%) | 8263777916 ±2937 (±~0.00004%) | 8263708389 ±0 (±~0%)
`mir_borrowck` | 2216149671 ±119458444 (±~5.4%) | 8340920100 ±2794 (±~0.00003%) | 8341613983.5 ±2.5 (±~0.00000003%)
`mir_built` | 1269059734 ±91514604 (±~7.2%) | 4454959122 ±1618 (±~0.00004%) | 4455303811 ±1 (±~0.00000002%)
`resolve_crate` | 942154987.5 ±53068423.5 (±~5.6%) | 3951197709 ±39 (±~0.000001%) | 3951196865 ±0 (±~0%)
* [**"Micro" noise (individual sampling intervals)**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#“Micro”-noise-(individual-sampling-intervals))
* [**Caveats**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Caveats)
* [**Disabling ASLR**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Disabling-ASLR)
* [**Non-deterministic proc macros**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Non-deterministic-proc-macros)
* [**Subtracting IRQs**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Subtracting-IRQs)
* [**Lack of support for multiple threads**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Lack-of-support-for-multiple-threads)
* [**Challenges**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Challenges)
* [**How do we even read hardware performance counters?**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#How-do-we-even-read-hardware-performance-counters)
* [**ASLR: it's free entropy**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#ASLR-it’s-free-entropy)
* [**The serializing instruction**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#The-serializing-instruction)
* [**Getting constantly interrupted**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Getting-constantly-interrupted)
* [**AMD patented time-travel and dubbed it `SpecLockMap`<br><sup> or: "how we accidentally unlocked `rr` on AMD Zen"</sup>**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#AMD-patented-time-travel-and-dubbed-it-SpecLockMapnbspnbspnbspnbspnbspnbspnbspnbspor-“how-we-accidentally-unlocked-rr-on-AMD-Zen”)
* [**`jemalloc`: purging will commence in ten seconds**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#jemalloc-purging-will-commence-in-ten-seconds)
* [**Rebasing *shouldn't* affect the results, right?**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Rebasing-*shouldn’t*-affect-the-results,-right)
* [**Epilogue: Zen's undocumented 420 counter**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Epilogue-Zen’s-undocumented-420-counter)
Adds the virtual-function-elimination unstable compiler flag and a check
that this flag is only used in combination with -Clto. LLVM can only
apply this optimization with fat LTO.
lint: add diagnostic translation migration lints
Introduce allow-by-default lints for checking whether diagnostics are written in
`SessionDiagnostic` or `AddSubdiagnostic` impls and whether diagnostics are translatable. These lints can be denied for modules once they are fully migrated to impls and translation.
These lints are intended to be temporary - once all diagnostics have been changed then we can just change the APIs we have and that will enforce these constraints thereafter.
r? `````@oli-obk`````
Fix#71363's test by adding `-Z translate-remapped-path-to-local-path=no`
The test relies on `library/std/src/error.rs` not corresponding to a local path, but remapping might still find the related local file of a remapped path. To fix the test, this PR adds a new `-Z` flag to disable finding the corresponding local path of a remapped path.
Introduce allow-by-default lints for checking whether diagnostics are
written in `SessionDiagnostic`/`AddSubdiagnostic` impls and whether
diagnostics are translatable. These lints can be denied for modules once
they are fully migrated to impls and translation.
Signed-off-by: David Wood <david.wood@huawei.com>
The test relies on library/std/src/error.rs not corresponding to a local
path, but remapping might still find the related local file of a
remapped path. To fix the test, this adds a new -Z flag to disable
finding the corresponding local path of a remapped path.
It looks like the last time had left some remaining cfg's -- which made me think
that the stage0 bump was actually successful. This brings us to a released 1.62
beta though.
Currently, the only API for creating errors from a diagnostic derive
will emit it immediately. This makes it difficult to add subdiagnostics
to diagnostics from the derive, so add `create_{err,warning}` functions
that return the diagnostic without emitting it.
Signed-off-by: David Wood <david.wood@huawei.com>
rustc_ast: Harmonize delimiter naming with `proc_macro::Delimiter`
Compiler cannot reuse `proc_macro::Delimiter` directly due to extra impls, but can at least use the same naming.
After this PR the only difference between these two enums is that `proc_macro::Delimiter::None` is turned into `token::Delimiter::Invisible`.
It's my mistake that the invisible delimiter is called `None` on stable, during the stabilization I audited the naming and wrote the docs, but missed the fact that the `None` naming gives a wrong and confusing impression about what this thing is.
cc https://github.com/rust-lang/rust/pull/96421
r? ``@nnethercote``
Make sure `-Dunused-crate-dependencies --json unused-externs` makes rustc exit with error status
This PR:
- fixes compiletest to understand unused extern notifications
- adds tests for `--json unused-externs`
- makes sure that deny-level unused externs notifications are treated as compile errors
- refactors the `emit_unused_externs` callstack to plumb through the level as an enum as a string, and adds `Level::is_error`
Update: adds `--json unused-externs-silent` with the original behaviour since Cargo needs it. Should address `@est31's` concerns.
Fixes: https://github.com/rust-lang/rust/issues/96068
Add missing `target_feature` to the list of well known cfg names
This PR adds the missing `target_feature` cfg name to the list of well known cfg names.
It was notice missing in https://github.com/rust-lang/rust/issues/96472 thanks to `@bjorn3,` the reason being that `--check-cfg=names()` automatically inherit the names passed by `--cfg` (or internal to `rustc`) and is seems that the vast majority of targets have at least one target feature leading to `target_feature` being a well known name in most target but it should always be a well known name so this PR add it unconditionally to list.
r? `@petrochenkov`
Since Cargo wants to do its own fatal error handling for unused
dependencies, add the option `--json unused-externs-silent` which
has the original behaviour of not indicating non-zero exit status for
`deny`/`forbid`-level unused dependencies.
Implement MIR opt unit tests
This implements rust-lang/compiler-team#502 .
There's not much to say here, this implementation does everything as proposed. I also added the flag to a bunch of existing tests (mostly those to which I could add it without causing huge diffs due to changes in line numbers). Summarizing the changes to test outputs:
- Every time an `MirPatch` is created, it adds a cleanup block to the body if it did not exist already. If this block is unused (as is usually the case), it usually gets removed soon after by some pass calling `SimplifyCFG` for unrelated reasons (in many cases this cycle happens quite a few times for a single body). We now run `SimplifyCFG` less often, so those blocks end up in some of our outputs. I looked at changing `MirPatch` to not do this, but that seemed too complicated for this PR. I may still do that in a follow-up.
- The `InstCombine` test had set `-C opt-level=0` in its flags and so there were no storage markers. I don't really see a good motivation for doing this, so bringing it back in line with what everything else does seems correct.
- One of the `EarlyOtherwiseBranch` tests had `UnreachableProp` running on it. Preventing that kind of thing is the goal of this feature, so this seems fine.
For the remaining tests for which this feature might be useful, we can gradually migrate them as opportunities present themselves.
In terms of documentation, I plan on submitting a PR to the rustc dev guide in the near future documenting this and other recent changes to MIR. If there's any other places to update, do let me know
r? `@nagisa`
This adds `nounused` to the set of extern flags:
`--extern nounused:core=/path/to/core/libcore.rlib`.
The effect of this flag is to suppress `unused-crate-dependencies`
warnings relating to the crate.
Remove `--extern-location` and all associated code
`--extern-location` was an experiment to investigate the best way to
generate useful diagnostics for unused dependency warnings by enabling a
build system to identify the corresponding build config.
While I did successfully use this, I've since been convinced the
alternative `--json unused-externs` mechanism is the way to go, and
there's no point in having two mechanisms with basically the same
functionality.
This effectively reverts https://github.com/rust-lang/rust/pull/72603
Create (unstable) 2024 edition
[On Zulip](https://rust-lang.zulipchat.com/#narrow/stream/213817-t-lang/topic/Deprecating.20macro.20scoping.20shenanigans/near/272860652), there was a small aside regarding creating the 2024 edition now as opposed to later. There was a reasonable amount of support and no stated opposition.
This change creates the 2024 edition in the compiler and creates a prelude for the 2024 edition. There is no current difference between the 2021 and 2024 editions. Cargo and other tools will need to be updated separately, as it's not in the same repository. This change permits the vast majority of work towards the next edition to proceed _now_ instead of waiting until 2024.
For sanity purposes, I've merged the "hello" UI tests into a single file with multiple revisions. Otherwise we'd end up with a file per edition, despite them being essentially identical.
````@rustbot```` label +T-lang +S-waiting-on-review
Not sure on the relevant team, to be honest.
Stabilize `derive_default_enum`
This stabilizes `#![feature(derive_default_enum)]`, as proposed in [RFC 3107](https://github.com/rust-lang/rfcs/pull/3107) and tracked in #87517. In short, it permits you to `#[derive(Default)]` on `enum`s, indicating what the default should be by placing a `#[default]` attribute on the desired variant (which must be a unit variant in the interest of forward compatibility).
```````@rustbot``````` label +S-waiting-on-review +T-lang
`--extern-location` was an experiment to investigate the best way to
generate useful diagnostics for unused dependency warnings by enabling a
build system to identify the corresponding build config.
While I did successfully use this, I've since been convinced the
alternative `--json unused-externs` mechanism is the way to go, and
there's no point in having two mechanisms with basically the same
functionality.
This effectively reverts https://github.com/rust-lang/rust/pull/72603
Loading the fallback bundle in compilation sessions that won't go on to
emit any errors unnecessarily degrades compile time performance, so
lazily create the Fluent bundle when it is first required.
Signed-off-by: David Wood <david.wood@huawei.com>
