Tools, tests, and experimenting with MIR-derived coverage counters
Adds a new mir_dump output file in HTML/CSS to visualize code regions
and the MIR features that they came from (including overlapping spans).
See example below:
Includes a basic, MIR-block-based implementation of coverage injection,
available via `-Zexperimental-coverage`. This implementation has known
flaws and omissions, but is simple enough to validate the new tools and
tests.
The existing `-Zinstrument-coverage` option currently enables
function-level coverage only, which at least appears to generate
accurate coverage reports at that level.
Experimental coverage is not accurate at this time. When branch coverage
works as intended, the `-Zexperimental-coverage` option should be
removed.
This PR replaces the bulk of PR #75828, with the remaining parts of
that PR distributed among other separate and indentpent PRs.
This PR depends on three of those other PRs: #76000, #76002, and
Rust compiler MCP rust-lang/compiler-team#278
Relevant issue: #34701 - Implement support for LLVMs code coverage
instrumentation
2020-08-27 14:13:04 -07:00
|
|
|
# needs-profiler-support
|
2021-02-02 21:53:30 -05:00
|
|
|
# min-llvm-version: 11.0
|
Tools, tests, and experimenting with MIR-derived coverage counters
Adds a new mir_dump output file in HTML/CSS to visualize code regions
and the MIR features that they came from (including overlapping spans).
See example below:
Includes a basic, MIR-block-based implementation of coverage injection,
available via `-Zexperimental-coverage`. This implementation has known
flaws and omissions, but is simple enough to validate the new tools and
tests.
The existing `-Zinstrument-coverage` option currently enables
function-level coverage only, which at least appears to generate
accurate coverage reports at that level.
Experimental coverage is not accurate at this time. When branch coverage
works as intended, the `-Zexperimental-coverage` option should be
removed.
This PR replaces the bulk of PR #75828, with the remaining parts of
that PR distributed among other separate and indentpent PRs.
This PR depends on three of those other PRs: #76000, #76002, and
Rust compiler MCP rust-lang/compiler-team#278
Relevant issue: #34701 - Implement support for LLVMs code coverage
instrumentation
2020-08-27 14:13:04 -07:00
|
|
|
|
2020-10-05 09:26:30 -07:00
|
|
|
-include ../coverage/coverage_tools.mk
|
Tools, tests, and experimenting with MIR-derived coverage counters
Adds a new mir_dump output file in HTML/CSS to visualize code regions
and the MIR features that they came from (including overlapping spans).
See example below:
Includes a basic, MIR-block-based implementation of coverage injection,
available via `-Zexperimental-coverage`. This implementation has known
flaws and omissions, but is simple enough to validate the new tools and
tests.
The existing `-Zinstrument-coverage` option currently enables
function-level coverage only, which at least appears to generate
accurate coverage reports at that level.
Experimental coverage is not accurate at this time. When branch coverage
works as intended, the `-Zexperimental-coverage` option should be
removed.
This PR replaces the bulk of PR #75828, with the remaining parts of
that PR distributed among other separate and indentpent PRs.
This PR depends on three of those other PRs: #76000, #76002, and
Rust compiler MCP rust-lang/compiler-team#278
Relevant issue: #34701 - Implement support for LLVMs code coverage
instrumentation
2020-08-27 14:13:04 -07:00
|
|
|
|
2020-11-30 23:37:49 -08:00
|
|
|
BASEDIR=../coverage-llvmir
|
Tools, tests, and experimenting with MIR-derived coverage counters
Adds a new mir_dump output file in HTML/CSS to visualize code regions
and the MIR features that they came from (including overlapping spans).
See example below:
Includes a basic, MIR-block-based implementation of coverage injection,
available via `-Zexperimental-coverage`. This implementation has known
flaws and omissions, but is simple enough to validate the new tools and
tests.
The existing `-Zinstrument-coverage` option currently enables
function-level coverage only, which at least appears to generate
accurate coverage reports at that level.
Experimental coverage is not accurate at this time. When branch coverage
works as intended, the `-Zexperimental-coverage` option should be
removed.
This PR replaces the bulk of PR #75828, with the remaining parts of
that PR distributed among other separate and indentpent PRs.
This PR depends on three of those other PRs: #76000, #76002, and
Rust compiler MCP rust-lang/compiler-team#278
Relevant issue: #34701 - Implement support for LLVMs code coverage
instrumentation
2020-08-27 14:13:04 -07:00
|
|
|
|
|
|
|
|
ifeq ($(UNAME),Darwin)
|
|
|
|
|
INSTR_PROF_DATA_SUFFIX=,regular,live_support
|
|
|
|
|
DATA_SECTION_PREFIX=__DATA,
|
|
|
|
|
LLVM_COV_SECTION_PREFIX=__LLVM_COV,
|
2020-11-23 12:56:07 -08:00
|
|
|
COMDAT_IF_SUPPORTED=
|
Tools, tests, and experimenting with MIR-derived coverage counters
Adds a new mir_dump output file in HTML/CSS to visualize code regions
and the MIR features that they came from (including overlapping spans).
See example below:
Includes a basic, MIR-block-based implementation of coverage injection,
available via `-Zexperimental-coverage`. This implementation has known
flaws and omissions, but is simple enough to validate the new tools and
tests.
The existing `-Zinstrument-coverage` option currently enables
function-level coverage only, which at least appears to generate
accurate coverage reports at that level.
Experimental coverage is not accurate at this time. When branch coverage
works as intended, the `-Zexperimental-coverage` option should be
removed.
This PR replaces the bulk of PR #75828, with the remaining parts of
that PR distributed among other separate and indentpent PRs.
This PR depends on three of those other PRs: #76000, #76002, and
Rust compiler MCP rust-lang/compiler-team#278
Relevant issue: #34701 - Implement support for LLVMs code coverage
instrumentation
2020-08-27 14:13:04 -07:00
|
|
|
else
|
|
|
|
|
INSTR_PROF_DATA_SUFFIX=
|
|
|
|
|
DATA_SECTION_PREFIX=
|
|
|
|
|
LLVM_COV_SECTION_PREFIX=
|
2020-11-23 12:56:07 -08:00
|
|
|
COMDAT_IF_SUPPORTED=, comdat
|
Tools, tests, and experimenting with MIR-derived coverage counters
Adds a new mir_dump output file in HTML/CSS to visualize code regions
and the MIR features that they came from (including overlapping spans).
See example below:
Includes a basic, MIR-block-based implementation of coverage injection,
available via `-Zexperimental-coverage`. This implementation has known
flaws and omissions, but is simple enough to validate the new tools and
tests.
The existing `-Zinstrument-coverage` option currently enables
function-level coverage only, which at least appears to generate
accurate coverage reports at that level.
Experimental coverage is not accurate at this time. When branch coverage
works as intended, the `-Zexperimental-coverage` option should be
removed.
This PR replaces the bulk of PR #75828, with the remaining parts of
that PR distributed among other separate and indentpent PRs.
This PR depends on three of those other PRs: #76000, #76002, and
Rust compiler MCP rust-lang/compiler-team#278
Relevant issue: #34701 - Implement support for LLVMs code coverage
instrumentation
2020-08-27 14:13:04 -07:00
|
|
|
endif
|
|
|
|
|
|
coverage bug fixes and optimization support
Adjusted LLVM codegen for code compiled with `-Zinstrument-coverage` to
address multiple, somewhat related issues.
Fixed a significant flaw in prior coverage solution: Every counter
generated a new counter variable, but there should have only been one
counter variable per function. This appears to have bloated .profraw
files significantly. (For a small program, it increased the size by
about 40%. I have not tested large programs, but there is anecdotal
evidence that profraw files were way too large. This is a good fix,
regardless, but hopefully it also addresses related issues.
Fixes: #82144
Invalid LLVM coverage data produced when compiled with -C opt-level=1
Existing tests now work up to at least `opt-level=3`. This required a
detailed analysis of the LLVM IR, comparisons with Clang C++ LLVM IR
when compiled with coverage, and a lot of trial and error with codegen
adjustments.
The biggest hurdle was figuring out how to continue to support coverage
results for unused functions and generics. Rust's coverage results have
three advantages over Clang's coverage results:
1. Rust's coverage map does not include any overlapping code regions,
making coverage counting unambiguous.
2. Rust generates coverage results (showing zero counts) for all unused
functions, including generics. (Clang does not generate coverage for
uninstantiated template functions.)
3. Rust's unused functions produce minimal stubbed functions in LLVM IR,
sufficient for including in the coverage results; while Clang must
generate the complete LLVM IR for each unused function, even though
it will never be called.
This PR removes the previous hack of attempting to inject coverage into
some other existing function instance, and generates dedicated instances
for each unused function. This change, and a few other adjustments
(similar to what is required for `-C link-dead-code`, but with lower
impact), makes it possible to support LLVM optimizations.
Fixes: #79651
Coverage report: "Unexecuted instantiation:..." for a generic function
from multiple crates
Fixed by removing the aforementioned hack. Some "Unexecuted
instantiation" notices are unavoidable, as explained in the
`used_crate.rs` test, but `-Zinstrument-coverage` has new options to
back off support for either unused generics, or all unused functions,
which avoids the notice, at the cost of less coverage of unused
functions.
Fixes: #82875
Invalid LLVM coverage data produced with crate brotli_decompressor
Fixed by disabling the LLVM function attribute that forces inlining, if
`-Z instrument-coverage` is enabled. This attribute is applied to
Rust functions with `#[inline(always)], and in some cases, the forced
inlining breaks coverage instrumentation and reports.
2021-03-15 16:32:45 -07:00
|
|
|
DEFINE_INTERNAL=define hidden
|
Tools, tests, and experimenting with MIR-derived coverage counters
Adds a new mir_dump output file in HTML/CSS to visualize code regions
and the MIR features that they came from (including overlapping spans).
See example below:
Includes a basic, MIR-block-based implementation of coverage injection,
available via `-Zexperimental-coverage`. This implementation has known
flaws and omissions, but is simple enough to validate the new tools and
tests.
The existing `-Zinstrument-coverage` option currently enables
function-level coverage only, which at least appears to generate
accurate coverage reports at that level.
Experimental coverage is not accurate at this time. When branch coverage
works as intended, the `-Zexperimental-coverage` option should be
removed.
This PR replaces the bulk of PR #75828, with the remaining parts of
that PR distributed among other separate and indentpent PRs.
This PR depends on three of those other PRs: #76000, #76002, and
Rust compiler MCP rust-lang/compiler-team#278
Relevant issue: #34701 - Implement support for LLVMs code coverage
instrumentation
2020-08-27 14:13:04 -07:00
|
|
|
|
|
|
|
|
ifdef IS_WINDOWS
|
|
|
|
|
LLVM_FILECHECK_OPTIONS=\
|
|
|
|
|
-check-prefixes=CHECK,WINDOWS \
|
|
|
|
|
-DPRIVATE_GLOBAL='internal global' \
|
|
|
|
|
-DDEFINE_INTERNAL='$(DEFINE_INTERNAL)' \
|
2020-11-23 12:56:07 -08:00
|
|
|
-DCOMDAT_IF_SUPPORTED='$(COMDAT_IF_SUPPORTED)' \
|
Tools, tests, and experimenting with MIR-derived coverage counters
Adds a new mir_dump output file in HTML/CSS to visualize code regions
and the MIR features that they came from (including overlapping spans).
See example below:
Includes a basic, MIR-block-based implementation of coverage injection,
available via `-Zexperimental-coverage`. This implementation has known
flaws and omissions, but is simple enough to validate the new tools and
tests.
The existing `-Zinstrument-coverage` option currently enables
function-level coverage only, which at least appears to generate
accurate coverage reports at that level.
Experimental coverage is not accurate at this time. When branch coverage
works as intended, the `-Zexperimental-coverage` option should be
removed.
This PR replaces the bulk of PR #75828, with the remaining parts of
that PR distributed among other separate and indentpent PRs.
This PR depends on three of those other PRs: #76000, #76002, and
Rust compiler MCP rust-lang/compiler-team#278
Relevant issue: #34701 - Implement support for LLVMs code coverage
instrumentation
2020-08-27 14:13:04 -07:00
|
|
|
-DINSTR_PROF_DATA='.lprfd$$M' \
|
|
|
|
|
-DINSTR_PROF_NAME='.lprfn$$M' \
|
|
|
|
|
-DINSTR_PROF_CNTS='.lprfc$$M' \
|
|
|
|
|
-DINSTR_PROF_VALS='.lprfv$$M' \
|
|
|
|
|
-DINSTR_PROF_VNODES='.lprfnd$$M' \
|
|
|
|
|
-DINSTR_PROF_COVMAP='.lcovmap$$M' \
|
2020-11-23 12:56:07 -08:00
|
|
|
-DINSTR_PROF_COVFUN='.lcovfun$$M' \
|
Tools, tests, and experimenting with MIR-derived coverage counters
Adds a new mir_dump output file in HTML/CSS to visualize code regions
and the MIR features that they came from (including overlapping spans).
See example below:
Includes a basic, MIR-block-based implementation of coverage injection,
available via `-Zexperimental-coverage`. This implementation has known
flaws and omissions, but is simple enough to validate the new tools and
tests.
The existing `-Zinstrument-coverage` option currently enables
function-level coverage only, which at least appears to generate
accurate coverage reports at that level.
Experimental coverage is not accurate at this time. When branch coverage
works as intended, the `-Zexperimental-coverage` option should be
removed.
This PR replaces the bulk of PR #75828, with the remaining parts of
that PR distributed among other separate and indentpent PRs.
This PR depends on three of those other PRs: #76000, #76002, and
Rust compiler MCP rust-lang/compiler-team#278
Relevant issue: #34701 - Implement support for LLVMs code coverage
instrumentation
2020-08-27 14:13:04 -07:00
|
|
|
-DINSTR_PROF_ORDERFILE='.lorderfile$$M'
|
|
|
|
|
else
|
|
|
|
|
LLVM_FILECHECK_OPTIONS=\
|
|
|
|
|
-check-prefixes=CHECK \
|
|
|
|
|
-DPRIVATE_GLOBAL='private global' \
|
|
|
|
|
-DDEFINE_INTERNAL='$(DEFINE_INTERNAL)' \
|
2020-11-23 12:56:07 -08:00
|
|
|
-DCOMDAT_IF_SUPPORTED='$(COMDAT_IF_SUPPORTED)' \
|
Tools, tests, and experimenting with MIR-derived coverage counters
Adds a new mir_dump output file in HTML/CSS to visualize code regions
and the MIR features that they came from (including overlapping spans).
See example below:
Includes a basic, MIR-block-based implementation of coverage injection,
available via `-Zexperimental-coverage`. This implementation has known
flaws and omissions, but is simple enough to validate the new tools and
tests.
The existing `-Zinstrument-coverage` option currently enables
function-level coverage only, which at least appears to generate
accurate coverage reports at that level.
Experimental coverage is not accurate at this time. When branch coverage
works as intended, the `-Zexperimental-coverage` option should be
removed.
This PR replaces the bulk of PR #75828, with the remaining parts of
that PR distributed among other separate and indentpent PRs.
This PR depends on three of those other PRs: #76000, #76002, and
Rust compiler MCP rust-lang/compiler-team#278
Relevant issue: #34701 - Implement support for LLVMs code coverage
instrumentation
2020-08-27 14:13:04 -07:00
|
|
|
-DINSTR_PROF_DATA='$(DATA_SECTION_PREFIX)__llvm_prf_data$(INSTR_PROF_DATA_SUFFIX)' \
|
|
|
|
|
-DINSTR_PROF_NAME='$(DATA_SECTION_PREFIX)__llvm_prf_names' \
|
|
|
|
|
-DINSTR_PROF_CNTS='$(DATA_SECTION_PREFIX)__llvm_prf_cnts' \
|
|
|
|
|
-DINSTR_PROF_VALS='$(DATA_SECTION_PREFIX)__llvm_prf_vals' \
|
|
|
|
|
-DINSTR_PROF_VNODES='$(DATA_SECTION_PREFIX)__llvm_prf_vnds' \
|
|
|
|
|
-DINSTR_PROF_COVMAP='$(LLVM_COV_SECTION_PREFIX)__llvm_covmap' \
|
2020-11-23 12:56:07 -08:00
|
|
|
-DINSTR_PROF_COVFUN='$(LLVM_COV_SECTION_PREFIX)__llvm_covfun' \
|
Tools, tests, and experimenting with MIR-derived coverage counters
Adds a new mir_dump output file in HTML/CSS to visualize code regions
and the MIR features that they came from (including overlapping spans).
See example below:
Includes a basic, MIR-block-based implementation of coverage injection,
available via `-Zexperimental-coverage`. This implementation has known
flaws and omissions, but is simple enough to validate the new tools and
tests.
The existing `-Zinstrument-coverage` option currently enables
function-level coverage only, which at least appears to generate
accurate coverage reports at that level.
Experimental coverage is not accurate at this time. When branch coverage
works as intended, the `-Zexperimental-coverage` option should be
removed.
This PR replaces the bulk of PR #75828, with the remaining parts of
that PR distributed among other separate and indentpent PRs.
This PR depends on three of those other PRs: #76000, #76002, and
Rust compiler MCP rust-lang/compiler-team#278
Relevant issue: #34701 - Implement support for LLVMs code coverage
instrumentation
2020-08-27 14:13:04 -07:00
|
|
|
-DINSTR_PROF_ORDERFILE='$(DATA_SECTION_PREFIX)__llvm_orderfile'
|
|
|
|
|
endif
|
|
|
|
|
|
2020-11-24 11:50:24 -08:00
|
|
|
all: test_llvm_ir
|
|
|
|
|
|
|
|
|
|
test_llvm_ir:
|
Tools, tests, and experimenting with MIR-derived coverage counters
Adds a new mir_dump output file in HTML/CSS to visualize code regions
and the MIR features that they came from (including overlapping spans).
See example below:
Includes a basic, MIR-block-based implementation of coverage injection,
available via `-Zexperimental-coverage`. This implementation has known
flaws and omissions, but is simple enough to validate the new tools and
tests.
The existing `-Zinstrument-coverage` option currently enables
function-level coverage only, which at least appears to generate
accurate coverage reports at that level.
Experimental coverage is not accurate at this time. When branch coverage
works as intended, the `-Zexperimental-coverage` option should be
removed.
This PR replaces the bulk of PR #75828, with the remaining parts of
that PR distributed among other separate and indentpent PRs.
This PR depends on three of those other PRs: #76000, #76002, and
Rust compiler MCP rust-lang/compiler-team#278
Relevant issue: #34701 - Implement support for LLVMs code coverage
instrumentation
2020-08-27 14:13:04 -07:00
|
|
|
# Compile the test program with non-experimental coverage instrumentation, and generate LLVM IR
|
|
|
|
|
$(RUSTC) $(BASEDIR)/testprog.rs \
|
|
|
|
|
-Zinstrument-coverage \
|
|
|
|
|
--emit=llvm-ir
|
|
|
|
|
|
2020-11-24 11:50:24 -08:00
|
|
|
cat "$(TMPDIR)"/testprog.ll | \
|
|
|
|
|
"$(LLVM_FILECHECK)" $(BASEDIR)/filecheck.testprog.txt $(LLVM_FILECHECK_OPTIONS)
|
