rust/tests/run-make
bors 5b20c45999 Auto merge of #128849 - estebank:issue-89143, r=jackh726
Tweak detection of multiple crate versions to be more encompassing

Previously, we only emitted the additional context if the type was in the same crate as the trait that appeared multiple times in the dependency tree. Now, we look at all traits looking for two with the same name in different crates with the same crate number, and we are more flexible looking for the types involved. This will work even if the type that implements the wrong trait version is from a different crate entirely.

```
error[E0277]: the trait bound `CustomErrorHandler: ErrorHandler` is not satisfied because the trait comes from a different crate version
 --> src/main.rs:5:17
  |
5 |     cnb_runtime(CustomErrorHandler {});
  |                 ^^^^^^^^^^^^^^^^^^^^^ the trait `ErrorHandler` is not implemented for `CustomErrorHandler`
  |
note: there are multiple different versions of crate `c` in the dependency graph
 --> /home/gh-estebank/testcase-rustc-crate-version-mismatch/c-v0.2/src/lib.rs:1:1
  |
1 | pub trait ErrorHandler {}
  | ^^^^^^^^^^^^^^^^^^^^^^ this is the required trait
  |
 ::: src/main.rs:1:5
  |
1 | use b::CustomErrorHandler;
  |     - one version of crate `c` is used here, as a dependency of crate `b`
2 | use c::cnb_runtime;
  |     - one version of crate `c` is used here, as a direct dependency of the current crate
  |
 ::: /home/gh-estebank/testcase-rustc-crate-version-mismatch/b/src/lib.rs:1:1
  |
1 | pub struct CustomErrorHandler {}
  | ----------------------------- this type doesn't implement the required trait
  |
 ::: /home/gh-estebank/testcase-rustc-crate-version-mismatch/c-v0.1/src/lib.rs:1:1
  |
1 | pub trait ErrorHandler {}
  | ---------------------- this is the found trait
  = note: two types coming from two different versions of the same crate are different types even if they look the same
  = help: you can use `cargo tree` to explore your dependency tree
```

Fix #89143.
2024-11-08 00:34:48 +00:00
..
a-b-a-linker-guard
alloc-no-oom-handling
alloc-no-rc
alloc-no-sync
allocator-shim-circular-deps
allow-warnings-cmdline-stability
apple-deployment-target Rollup merge of #129367 - madsmtm:fix-apple-aarch64-deployment-targets, r=jieyouxu 2024-09-12 20:37:15 +10:00
apple-sdk-version Do not specify an SDK version in object files 2024-09-29 14:45:09 +02:00
archive-duplicate-names
arguments-non-c-like-enum
artifact-incr-cache
artifact-incr-cache-no-obj
atomic-lock-free
avr-rjmp-offset Use rust-lld instead of avr-gcc as the linker 2024-10-17 09:00:08 +02:00
bare-outfile
bin-emit-no-symbols
box-struct-no-segfault
branch-protection-check-IBT
broken-pipe-no-ice Ignore broken-pipe-no-ice on apple for now 2024-10-09 05:34:49 +00:00
c-dynamic-dylib
c-dynamic-rlib
c-link-to-rust-dylib
c-link-to-rust-staticlib
c-link-to-rust-va-list-fn Reformat using the new identifier sorting from rustfmt 2024-09-22 19:11:29 -04:00
c-static-dylib
c-static-rlib
c-unwind-abi-catch-lib-panic Reformat using the new identifier sorting from rustfmt 2024-09-22 19:11:29 -04:00
c-unwind-abi-catch-panic Reformat using the new identifier sorting from rustfmt 2024-09-22 19:11:29 -04:00
cat-and-grep-sanity-check
cdylib
cdylib-dylib-linkage
cdylib-fewer-symbols
checksum-freshness Restore prior behavior with less duplicate info in dep file 2024-10-01 23:05:24 -06:00
clear-error-blank-output tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
codegen-options-parsing
comment-section tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
compile-stdin tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
compiler-builtins Update run-make tests to use cargo wrapper cmd 2024-09-24 19:04:51 +08:00
compiler-lookup-paths
compiler-lookup-paths-2
compiler-rt-works-on-mingw
compressed-debuginfo
compressed-debuginfo-zstd Reformat using the new identifier sorting from rustfmt 2024-09-22 19:11:29 -04:00
const_fn_mir
const-prop-lint
core-no-fp-fmt-parse
core-no-oom-handling
cpp-global-destructors
crate-circular-deps-link
crate-data-smoke
crate-hash-rustc-version
crate-loading Remove less relevant info from diagnostic 2024-11-07 20:18:00 +00:00
crate-name-priority
cross-lang-lto
cross-lang-lto-clang ci: add aarch64-gnu-debug job 2024-10-24 16:49:12 +01:00
cross-lang-lto-pgo-smoketest-clang Rename directive needs-profiler-support to needs-profiler-runtime 2024-10-09 20:58:27 +11:00
cross-lang-lto-riscv-abi tests/run-make: fix cross-lang-lto-riscv-abi 2024-10-22 19:43:22 +08:00
cross-lang-lto-upstream-rlibs
CURRENT_RUSTC_VERSION
debug-assertions
debugger-visualizer-dep-info
dep-graph
dep-info
deref-impl-rustdoc-ice
doctests-keep-binaries
doctests-keep-binaries-2024
doctests-merge Rename doctest attribute standalone-crate into standalone_crate for coherency 2024-09-29 13:01:41 +02:00
doctests-runtool
dos-device-input Use path instead of Path in some run-make tests 2024-08-29 10:15:17 -04:00
dump-ice-to-disk
dump-mono-stats
duplicate-output-flavors
dylib-chain
dylib-soname Only add an automatic SONAME for Rust dylibs 2024-09-27 15:53:26 -07:00
embed-source-dwarf Update the minimum external LLVM to 18 2024-09-18 13:53:31 -07:00
emit
emit-named-files
emit-path-unhashed
emit-shared-files Use path instead of Path in some run-make tests 2024-08-29 10:15:17 -04:00
emit-stack-sizes
emit-to-stdout Migrate emit-to-stdout to new run-make 2024-10-07 16:30:41 +00:00
env-dep-info
error-found-staticlib-instead-crate
error-writing-dependencies
exit-code
export-executable-symbols
extern-diff-internal-name
extern-flag-disambiguates
extern-flag-fun
extern-flag-pathless
extern-flag-rename-transitive
extern-fn-explicit-align Reformat using the new identifier sorting from rustfmt 2024-09-22 19:11:29 -04:00
extern-fn-generic
extern-fn-mangle
extern-fn-reachable
extern-fn-slice-no-ice
extern-fn-struct-passing-abi
extern-fn-with-extern-types
extern-fn-with-packed-struct
extern-fn-with-union
extern-multiple-copies
extern-multiple-copies2
extern-overrides-distribution
external-crate-panic-handle-no-lint
extra-filename-with-temp-outputs
fmt-write-bloat
forced-unwind-terminate-pof
foreign-double-unwind
foreign-exceptions Reformat using the new identifier sorting from rustfmt 2024-09-22 19:11:29 -04:00
foreign-rust-exceptions
glibc-staticlib-args
ice-dep-cannot-find-dep
ice-static-mir
import-macro-verbatim
inaccessible-temp-dir
include-all-symbols-linking
include-bytes-deps
incr-add-rust-src-component
incr-foreign-head-span
incr-prev-body-beyond-eof
incr-test-moved-file
incremental-debugger-visualizer
incremental-session-fail
inline-always-many-cgu
interdependent-c-libraries
intrinsic-unreachable
invalid-library
invalid-so
invalid-staticlib
invalid-symlink-search-path tests/run-make: update for symlink helper changes 2024-09-16 21:31:41 +08:00
invalid-tmpdir-env-var
issue-84395-lto-embed-bitcode tests/run-make: port issue-84395-lto-embed-bitcode to rmake.rs 2024-10-22 19:43:22 +08:00
issue-107495-archive-permissions
issue-125484-used-dependencies
jobserver-error
json-error-no-offset
lib-trait-for-trait-no-ice
libs-through-symlinks
libtest-json
libtest-junit tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
libtest-padding
libtest-thread-limit
link-arg
link-args-order
link-cfg
link-dedup
link-eh-frame-terminator
link-framework
link-native-static-lib-to-dylib
link-path-order
linkage-attr-framework port tests/ui/linkage-attr/framework to run-make 2024-10-27 21:23:28 -04:00
linkage-attr-on-static
llvm-ident tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
llvm-outputs tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
long-linker-command-lines
long-linker-command-lines-cmd-exe
longjmp-across-rust Fix longjmp-across-rust test 2024-10-10 15:02:47 +01:00
ls-metadata
lto-avoid-object-duplication
lto-dylib-dep
lto-empty
lto-linkage-used-attr
lto-no-link-whole-rlib
lto-readonly-lib
lto-smoke
lto-smoke-c
macos-fat-archive
manual-crate-name Use path instead of Path in some run-make tests 2024-08-29 10:15:17 -04:00
manual-link
many-crates-but-no-match
metadata-dep-info
metadata-flag-frobs-symbols
metadata-only-crate-no-ice
min-global-align
mingw-export-call-convention
mismatching-target-triples
missing-crate-dependency
mixing-deps
mixing-formats
mixing-libs
moved-src-dir-fingerprint-ice
msvc-lld-thinlto-imp-symbols add non-regression test for issue 81408 2024-09-22 11:40:00 +02:00
msvc-wholearchive
mte-ffi
multiple-emits
naked-symbol-visibility various fixes for naked_asm! implementation 2024-10-06 19:00:09 +02:00
native-lib-alt-naming
native-lib-load-order
native-link-modifier-bundle
native-link-modifier-verbatim-linker
native-link-modifier-verbatim-rustc
native-link-modifier-whole-archive
no-alloc-shim
no-builtins-attribute tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
no-builtins-lto
no-cdylib-as-rdylib
no-duplicate-libs
no-input-file
no-intermediate-extras
non-pie-thread-local
non-unicode-env Fix bug where option_env! would return None when env var is present but not valid Unicode 2024-10-13 02:10:19 +01:00
non-unicode-in-incremental-dir
notify-all-emit-artifacts
obey-crate-type-flag
optimization-remarks-dir
optimization-remarks-dir-pgo Rename directive needs-profiler-support to needs-profiler-runtime 2024-10-09 20:58:27 +11:00
output-filename-conflicts-with-directory
output-filename-overwrites-input
output-type-permutations
output-with-hyphens
override-aliased-flags
overwrite-input
panic-abort-eh_frame
panic-impl-transitive
parallel-rustc-no-overwrite
pass-linker-flags
pass-linker-flags-flavor
pass-linker-flags-from-dep
pass-non-c-like-enum-to-c
pdb-alt-path
pdb-buildinfo-cl-cmd Add test for S_OBJNAME and update test for LF_BUILDINFO cl and cmd for 2024-09-10 17:23:05 +02:00
pdb-sobjname Add test for S_OBJNAME and update test for LF_BUILDINFO cl and cmd for 2024-09-10 17:23:05 +02:00
pgo-branch-weights Rename directive needs-profiler-support to needs-profiler-runtime 2024-10-09 20:58:27 +11:00
pgo-gen Rename directive needs-profiler-support to needs-profiler-runtime 2024-10-09 20:58:27 +11:00
pgo-gen-lto Rename directive needs-profiler-support to needs-profiler-runtime 2024-10-09 20:58:27 +11:00
pgo-gen-no-imp-symbols
pgo-indirect-call-promotion Rename directive needs-profiler-support to needs-profiler-runtime 2024-10-09 20:58:27 +11:00
pgo-use Rename directive needs-profiler-support to needs-profiler-runtime 2024-10-09 20:58:27 +11:00
pointer-auth-link-with-c rustc_codegen_llvm: Add a new 'pc' option to branch-protection 2024-10-31 11:59:17 +00:00
pointer-auth-link-with-c-lto-clang tests: add pac-ret + cross-language lto test 2024-10-24 16:49:12 +01:00
prefer-dylib
prefer-rlib
pretty-print-to-file
pretty-print-with-dep-file Use path instead of Path in some run-make tests 2024-08-29 10:15:17 -04:00
print-cfg
print-check-cfg
print-native-static-libs
print-target-cpus-native Add a run-make test for same-arch --print=target-cpus 2024-11-02 23:39:29 +11:00
print-target-list
print-to-output
proc-macro-init-order
proc-macro-three-crates
prune-link-args
raw-dylib-alt-calling-convention
raw-dylib-c
raw-dylib-cross-compilation
raw-dylib-custom-dlltool
raw-dylib-import-name-type
raw-dylib-inline-cross-dylib
raw-dylib-link-ordinal
raw-dylib-stdcall-ordinal
raw-fn-pointer-opt-undefined-behavior
reachable-extern-fn-available-lto
redundant-libs
relocation-model
relro-levels
remap-path-prefix
remap-path-prefix-dwarf
remove-dir-all-race
repr128-dwarf
reproducible-build
reproducible-build-2
reset-codegen-1 Use path instead of Path in some run-make tests 2024-08-29 10:15:17 -04:00
resolve-rename
return-non-c-like-enum
return-non-c-like-enum-from-c
rlib-chain
rlib-format-packed-bundled-libs
rlib-format-packed-bundled-libs-2
rlib-format-packed-bundled-libs-3
rmeta-preferred
run-in-tmpdir-self-test
rust-lld
rust-lld-by-default-beta-stable
rust-lld-by-default-nightly
rust-lld-custom-target
rust-lld-link-script-provide Add a test case for #131164 2024-10-23 19:01:38 +08:00
rustc-crates-on-stable Add rustc_abi and rustc_parse_format to rustc-crates-on-stable test 2024-10-21 14:57:41 +02:00
rustc-macro-dep-files
rustdoc-cfgspec-parsing
rustdoc-default-output rfc#3662 changes under unstable flags 2024-09-07 18:59:01 -04:00
rustdoc-determinism Use path instead of Path in some run-make tests 2024-08-29 10:15:17 -04:00
rustdoc-error-lines
rustdoc-io-error
rustdoc-map-file
rustdoc-output-path Use path instead of Path in some run-make tests 2024-08-29 10:15:17 -04:00
rustdoc-output-stdout Actually parse stdout json, instead of using hacky contains logic. 2024-08-31 22:08:38 +00:00
rustdoc-scrape-examples-invalid-expr
rustdoc-scrape-examples-macros
rustdoc-scrape-examples-multiple
rustdoc-scrape-examples-ordering
rustdoc-scrape-examples-remap
rustdoc-scrape-examples-test
rustdoc-scrape-examples-whitespace
rustdoc-shared-flags Reformat using the new identifier sorting from rustfmt 2024-09-22 19:11:29 -04:00
rustdoc-target-spec-json-path
rustdoc-test-args
rustdoc-themes
rustdoc-verify-output-files
rustdoc-with-out-dir-option
rustdoc-with-output-option
rustdoc-with-short-out-dir-option
sanitizer-cdylib-link
sanitizer-dylib-link
sanitizer-staticlib-link
separate-link tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
separate-link-fail
sepcomp-cci-copies
sepcomp-inlining
sepcomp-separate
share-generics-dylib
share-generics-export-again
short-ice
silly-file-names
simd-ffi Ban non-array SIMD 2024-09-09 19:39:43 -07:00
split-debuginfo Reformat using the new identifier sorting from rustfmt 2024-09-22 19:11:29 -04:00
stable-symbol-names
static-dylib-by-default
static-extern-type
static-pie tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
static-unwinding
staticlib-blank-lib
staticlib-dylib-linkage
staticlib-thin-archive
std-core-cycle
stdin-rustc tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
stdin-rustdoc tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
suspicious-library
symbol-mangling-hashed
symbol-visibility
symbols-include-type-name
symlinked-extern tests/run-make: update for symlink helper changes 2024-09-16 21:31:41 +08:00
symlinked-libraries tests/run-make: update for symlink helper changes 2024-09-16 21:31:41 +08:00
symlinked-rlib tests/run-make: update for symlink helper changes 2024-09-16 21:31:41 +08:00
sysroot-crates-are-unstable tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
target-cpu-native
target-specs Remove is_builtin target spec field 2024-11-03 21:06:49 +00:00
target-without-atomic-cas
test-benches
test-harness
textrel-on-minimal-lib
thumb-none-cortex-m Update run-make tests to use cargo wrapper cmd 2024-09-24 19:04:51 +08:00
thumb-none-qemu Update run-make tests to use cargo wrapper cmd 2024-09-24 19:04:51 +08:00
track-path-dep-info
track-pgo-dep-info Rename directive needs-profiler-support to needs-profiler-runtime 2024-10-09 20:58:27 +11:00
translation
type-mismatch-same-crate-name Reformat using the new identifier sorting from rustfmt 2024-09-22 19:11:29 -04:00
unknown-mod-stdin tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
unstable-flag-required
use-suggestions-rust-2018
used
used-cdylib-macos
version-verbose-commit-hash
volatile-intrinsics
wasm-custom-section
wasm-custom-sections-opt
wasm-exceptions-nostd
wasm-export-all-symbols Reformat using the new identifier sorting from rustfmt 2024-09-22 19:11:29 -04:00
wasm-import-module
wasm-override-linker
wasm-panic-small
wasm-spurious-import
wasm-stringify-ints-small
wasm-symbols-different-module
wasm-symbols-not-exported
wasm-symbols-not-imported
weird-output-filenames
windows-binary-no-external-deps
windows-safeseh
windows-spawn
windows-ws2_32
x86_64-fortanix-unknown-sgx-lvi tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
zero-extend-abi-param-passing
README.md Fix unnecessary nesting in run-make test output directories 2024-10-16 12:40:25 +11:00
tools.mk

The run-make test suite

The run-make test suite contains tests which are the most flexible out of all the rust-lang/rust test suites. run-make tests can basically contain arbitrary code, and are supported by the run_make_support library.

Infrastructure

There are two kinds of run-make tests:

  1. The new rmake.rs version: this allows run-make tests to be written in Rust (with rmake.rs as the main test file).
  2. The legacy Makefile version: this is what run-make tests were written with before support for rmake.rs was introduced.

The implementation for collecting and building the rmake.rs recipes (or Makefiles) are in src/tools/compiletest/src/runtest.rs, in run_rmake_v2_test and run_rmake_legacy_test.

Rust-based run-make tests: rmake.rs

The setup for the rmake.rs version is a 3-stage process:

  1. First, we build the run_make_support library in bootstrap as a tool lib.

  2. Then, we compile the rmake.rs "recipe" linking the support library and its dependencies in, and provide a bunch of env vars. We setup a directory structure within build/<target>/test/run-make/

    <test-name>/
        rmake.exe              # recipe binary
        rmake_out/             # sources from test sources copied over
    

    and copy non-rmake.rs input support files over to rmake_out/. The support library is made available as an extern prelude.

  3. Finally, we run the recipe binary and set rmake_out/ as the working directory.

Formatting

Note that files under tests/ are not formatted by ./x fmt, use rustfmt tests/path/to/file.rs to format a specific file if desired.