rust/tests/run-make
bors 5b20c45999 Auto merge of #128849 - estebank:issue-89143, r=jackh726
Tweak detection of multiple crate versions to be more encompassing

Previously, we only emitted the additional context if the type was in the same crate as the trait that appeared multiple times in the dependency tree. Now, we look at all traits looking for two with the same name in different crates with the same crate number, and we are more flexible looking for the types involved. This will work even if the type that implements the wrong trait version is from a different crate entirely.

```
error[E0277]: the trait bound `CustomErrorHandler: ErrorHandler` is not satisfied because the trait comes from a different crate version
 --> src/main.rs:5:17
  |
5 |     cnb_runtime(CustomErrorHandler {});
  |                 ^^^^^^^^^^^^^^^^^^^^^ the trait `ErrorHandler` is not implemented for `CustomErrorHandler`
  |
note: there are multiple different versions of crate `c` in the dependency graph
 --> /home/gh-estebank/testcase-rustc-crate-version-mismatch/c-v0.2/src/lib.rs:1:1
  |
1 | pub trait ErrorHandler {}
  | ^^^^^^^^^^^^^^^^^^^^^^ this is the required trait
  |
 ::: src/main.rs:1:5
  |
1 | use b::CustomErrorHandler;
  |     - one version of crate `c` is used here, as a dependency of crate `b`
2 | use c::cnb_runtime;
  |     - one version of crate `c` is used here, as a direct dependency of the current crate
  |
 ::: /home/gh-estebank/testcase-rustc-crate-version-mismatch/b/src/lib.rs:1:1
  |
1 | pub struct CustomErrorHandler {}
  | ----------------------------- this type doesn't implement the required trait
  |
 ::: /home/gh-estebank/testcase-rustc-crate-version-mismatch/c-v0.1/src/lib.rs:1:1
  |
1 | pub trait ErrorHandler {}
  | ---------------------- this is the found trait
  = note: two types coming from two different versions of the same crate are different types even if they look the same
  = help: you can use `cargo tree` to explore your dependency tree
```

Fix #89143.
2024-11-08 00:34:48 +00:00
..
a-b-a-linker-guard
alloc-no-oom-handling
alloc-no-rc
alloc-no-sync
allocator-shim-circular-deps
allow-warnings-cmdline-stability
apple-deployment-target Rollup merge of #129367 - madsmtm:fix-apple-aarch64-deployment-targets, r=jieyouxu 2024-09-12 20:37:15 +10:00
apple-sdk-version Do not specify an SDK version in object files 2024-09-29 14:45:09 +02:00
archive-duplicate-names
arguments-non-c-like-enum remove unused imports from rmake tests 2024-08-10 18:06:10 +00:00
artifact-incr-cache
artifact-incr-cache-no-obj
atomic-lock-free
avr-rjmp-offset Use rust-lld instead of avr-gcc as the linker 2024-10-17 09:00:08 +02:00
bare-outfile
bin-emit-no-symbols
box-struct-no-segfault
branch-protection-check-IBT
broken-pipe-no-ice Ignore broken-pipe-no-ice on apple for now 2024-10-09 05:34:49 +00:00
c-dynamic-dylib
c-dynamic-rlib
c-link-to-rust-dylib
c-link-to-rust-staticlib remove unused imports from rmake tests 2024-08-10 18:06:10 +00:00
c-link-to-rust-va-list-fn Reformat using the new identifier sorting from rustfmt 2024-09-22 19:11:29 -04:00
c-static-dylib
c-static-rlib
c-unwind-abi-catch-lib-panic Reformat using the new identifier sorting from rustfmt 2024-09-22 19:11:29 -04:00
c-unwind-abi-catch-panic Reformat using the new identifier sorting from rustfmt 2024-09-22 19:11:29 -04:00
cat-and-grep-sanity-check
cdylib
cdylib-dylib-linkage
cdylib-fewer-symbols
checksum-freshness Restore prior behavior with less duplicate info in dep file 2024-10-01 23:05:24 -06:00
clear-error-blank-output tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
codegen-options-parsing
comment-section tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
compile-stdin tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
compiler-builtins Update run-make tests to use cargo wrapper cmd 2024-09-24 19:04:51 +08:00
compiler-lookup-paths
compiler-lookup-paths-2
compiler-rt-works-on-mingw
compressed-debuginfo remove unused imports from rmake tests 2024-08-10 18:06:10 +00:00
compressed-debuginfo-zstd Reformat using the new identifier sorting from rustfmt 2024-09-22 19:11:29 -04:00
const_fn_mir remove unused imports from rmake tests 2024-08-10 18:06:10 +00:00
const-prop-lint
core-no-fp-fmt-parse
core-no-oom-handling
cpp-global-destructors
crate-circular-deps-link
crate-data-smoke
crate-hash-rustc-version
crate-loading Remove less relevant info from diagnostic 2024-11-07 20:18:00 +00:00
crate-name-priority
cross-lang-lto
cross-lang-lto-clang ci: add aarch64-gnu-debug job 2024-10-24 16:49:12 +01:00
cross-lang-lto-pgo-smoketest-clang Rename directive needs-profiler-support to needs-profiler-runtime 2024-10-09 20:58:27 +11:00
cross-lang-lto-riscv-abi tests/run-make: fix cross-lang-lto-riscv-abi 2024-10-22 19:43:22 +08:00
cross-lang-lto-upstream-rlibs rewrite long-linker-command-lines-cmd-exe to rmake 2024-08-06 11:03:18 -04:00
CURRENT_RUSTC_VERSION remove unused imports from rmake tests 2024-08-10 18:06:10 +00:00
debug-assertions
debugger-visualizer-dep-info Rename foo.py to my_gdb_script.py 2024-08-22 11:52:21 +10:00
dep-graph
dep-info coalesce dep-info-spaces and dep-info-doesnt-run-much into dep-info 2024-08-15 10:16:30 -04:00
deref-impl-rustdoc-ice About rmake tests 2024-08-15 15:44:29 +02:00
doctests-keep-binaries
doctests-keep-binaries-2024 Ignore cross compile check for tests/run-make/doctests-keep-binaries-2024 test 2024-08-13 20:14:57 +02:00
doctests-merge Rename doctest attribute standalone-crate into standalone_crate for coherency 2024-09-29 13:01:41 +02:00
doctests-runtool
dos-device-input Use path instead of Path in some run-make tests 2024-08-29 10:15:17 -04:00
dump-ice-to-disk tests: disable dump-ice-to-disk for i686-mingw 2024-08-18 09:13:44 +00:00
dump-mono-stats
duplicate-output-flavors
dylib-chain
dylib-soname Only add an automatic SONAME for Rust dylibs 2024-09-27 15:53:26 -07:00
embed-source-dwarf Update the minimum external LLVM to 18 2024-09-18 13:53:31 -07:00
emit
emit-named-files
emit-path-unhashed
emit-shared-files Use path instead of Path in some run-make tests 2024-08-29 10:15:17 -04:00
emit-stack-sizes
emit-to-stdout Migrate emit-to-stdout to new run-make 2024-10-07 16:30:41 +00:00
env-dep-info
error-found-staticlib-instead-crate
error-writing-dependencies
exit-code About rmake tests 2024-08-15 15:44:29 +02:00
export-executable-symbols
extern-diff-internal-name
extern-flag-disambiguates remove unused imports from rmake tests 2024-08-10 18:06:10 +00:00
extern-flag-fun
extern-flag-pathless
extern-flag-rename-transitive
extern-fn-explicit-align Reformat using the new identifier sorting from rustfmt 2024-09-22 19:11:29 -04:00
extern-fn-generic
extern-fn-mangle
extern-fn-reachable
extern-fn-slice-no-ice
extern-fn-struct-passing-abi
extern-fn-with-extern-types
extern-fn-with-packed-struct
extern-fn-with-union
extern-multiple-copies
extern-multiple-copies2
extern-overrides-distribution
external-crate-panic-handle-no-lint
extra-filename-with-temp-outputs
fmt-write-bloat Update reason why fmt-write-bloat ignores windows 2024-08-09 21:21:43 +00:00
forced-unwind-terminate-pof
foreign-double-unwind
foreign-exceptions Reformat using the new identifier sorting from rustfmt 2024-09-22 19:11:29 -04:00
foreign-rust-exceptions
glibc-staticlib-args
ice-dep-cannot-find-dep remove unused imports from rmake tests 2024-08-10 18:06:10 +00:00
ice-static-mir
import-macro-verbatim
inaccessible-temp-dir
include-all-symbols-linking
include-bytes-deps
incr-add-rust-src-component
incr-foreign-head-span
incr-prev-body-beyond-eof
incr-test-moved-file remove unused imports from rmake tests 2024-08-10 18:06:10 +00:00
incremental-debugger-visualizer remove unused imports from rmake tests 2024-08-10 18:06:10 +00:00
incremental-session-fail
inline-always-many-cgu
interdependent-c-libraries
intrinsic-unreachable
invalid-library
invalid-so
invalid-staticlib
invalid-symlink-search-path tests/run-make: update for symlink helper changes 2024-09-16 21:31:41 +08:00
invalid-tmpdir-env-var
issue-84395-lto-embed-bitcode tests/run-make: port issue-84395-lto-embed-bitcode to rmake.rs 2024-10-22 19:43:22 +08:00
issue-107495-archive-permissions
issue-125484-used-dependencies
jobserver-error
json-error-no-offset
lib-trait-for-trait-no-ice
libs-through-symlinks
libtest-json Port run-make/libtest-json/validate_json.py to Rust 2024-08-17 18:15:38 +10:00
libtest-junit tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
libtest-padding
libtest-thread-limit Add needs-unwind compiletest directive to libtest-thread-limit 2024-08-29 10:13:48 -04:00
link-arg
link-args-order
link-cfg
link-dedup
link-eh-frame-terminator
link-framework
link-native-static-lib-to-dylib
link-path-order
linkage-attr-framework port tests/ui/linkage-attr/framework to run-make 2024-10-27 21:23:28 -04:00
linkage-attr-on-static
llvm-ident tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
llvm-outputs tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
long-linker-command-lines rewrite long-linker-command-lines-cmd-exe to rmake 2024-08-06 11:03:18 -04:00
long-linker-command-lines-cmd-exe rewrite long-linker-command-lines-cmd-exe to rmake 2024-08-06 11:03:18 -04:00
longjmp-across-rust Fix longjmp-across-rust test 2024-10-10 15:02:47 +01:00
ls-metadata
lto-avoid-object-duplication
lto-dylib-dep
lto-empty
lto-linkage-used-attr
lto-no-link-whole-rlib
lto-readonly-lib remove unused imports from rmake tests 2024-08-10 18:06:10 +00:00
lto-smoke
lto-smoke-c
macos-fat-archive
manual-crate-name Use path instead of Path in some run-make tests 2024-08-29 10:15:17 -04:00
manual-link
many-crates-but-no-match
metadata-dep-info
metadata-flag-frobs-symbols
metadata-only-crate-no-ice
min-global-align rewrite no-alloc-shim to rmake 2024-08-14 10:33:56 -04:00
mingw-export-call-convention
mismatching-target-triples
missing-crate-dependency
mixing-deps
mixing-formats
mixing-libs
moved-src-dir-fingerprint-ice
msvc-lld-thinlto-imp-symbols add non-regression test for issue 81408 2024-09-22 11:40:00 +02:00
msvc-wholearchive Test wholearchive on rust staticlib 2024-08-19 18:26:55 +00:00
mte-ffi
multiple-emits remove unused imports from rmake tests 2024-08-10 18:06:10 +00:00
naked-symbol-visibility various fixes for naked_asm! implementation 2024-10-06 19:00:09 +02:00
native-lib-alt-naming linker: Better support alternative static library naming on MSVC 2024-08-27 22:13:31 +03:00
native-lib-load-order
native-link-modifier-bundle rewrite native-link-modifier-bundle to rmake 2024-08-15 10:17:38 -04:00
native-link-modifier-verbatim-linker
native-link-modifier-verbatim-rustc
native-link-modifier-whole-archive
no-alloc-shim rewrite no-alloc-shim to rmake 2024-08-14 10:33:56 -04:00
no-builtins-attribute tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
no-builtins-lto
no-cdylib-as-rdylib
no-duplicate-libs
no-input-file
no-intermediate-extras
non-pie-thread-local
non-unicode-env Fix bug where option_env! would return None when env var is present but not valid Unicode 2024-10-13 02:10:19 +01:00
non-unicode-in-incremental-dir remove other warnings from rmake tests 2024-08-10 18:16:15 +00:00
notify-all-emit-artifacts
obey-crate-type-flag
optimization-remarks-dir
optimization-remarks-dir-pgo Rename directive needs-profiler-support to needs-profiler-runtime 2024-10-09 20:58:27 +11:00
output-filename-conflicts-with-directory
output-filename-overwrites-input
output-type-permutations remove other warnings from rmake tests 2024-08-10 18:16:15 +00:00
output-with-hyphens
override-aliased-flags
overwrite-input
panic-abort-eh_frame
panic-impl-transitive
parallel-rustc-no-overwrite
pass-linker-flags
pass-linker-flags-flavor
pass-linker-flags-from-dep
pass-non-c-like-enum-to-c
pdb-alt-path
pdb-buildinfo-cl-cmd Add test for S_OBJNAME and update test for LF_BUILDINFO cl and cmd for 2024-09-10 17:23:05 +02:00
pdb-sobjname Add test for S_OBJNAME and update test for LF_BUILDINFO cl and cmd for 2024-09-10 17:23:05 +02:00
pgo-branch-weights Rename directive needs-profiler-support to needs-profiler-runtime 2024-10-09 20:58:27 +11:00
pgo-gen Rename directive needs-profiler-support to needs-profiler-runtime 2024-10-09 20:58:27 +11:00
pgo-gen-lto Rename directive needs-profiler-support to needs-profiler-runtime 2024-10-09 20:58:27 +11:00
pgo-gen-no-imp-symbols
pgo-indirect-call-promotion Rename directive needs-profiler-support to needs-profiler-runtime 2024-10-09 20:58:27 +11:00
pgo-use Rename directive needs-profiler-support to needs-profiler-runtime 2024-10-09 20:58:27 +11:00
pointer-auth-link-with-c rustc_codegen_llvm: Add a new 'pc' option to branch-protection 2024-10-31 11:59:17 +00:00
pointer-auth-link-with-c-lto-clang tests: add pac-ret + cross-language lto test 2024-10-24 16:49:12 +01:00
prefer-dylib
prefer-rlib
pretty-print-to-file
pretty-print-with-dep-file Use path instead of Path in some run-make tests 2024-08-29 10:15:17 -04:00
print-cfg tests/run-make: add missing needs-llvm-components 2024-08-26 18:20:32 +08:00
print-check-cfg remove unused imports from rmake tests 2024-08-10 18:06:10 +00:00
print-native-static-libs remove unused imports from rmake tests 2024-08-10 18:06:10 +00:00
print-target-cpus-native Add a run-make test for same-arch --print=target-cpus 2024-11-02 23:39:29 +11:00
print-target-list tests/run-make: add missing needs-llvm-components 2024-08-26 18:20:32 +08:00
print-to-output tests/run-make: add missing needs-llvm-components 2024-08-26 18:20:32 +08:00
proc-macro-init-order
proc-macro-three-crates
prune-link-args
raw-dylib-alt-calling-convention
raw-dylib-c
raw-dylib-cross-compilation
raw-dylib-custom-dlltool
raw-dylib-import-name-type
raw-dylib-inline-cross-dylib
raw-dylib-link-ordinal
raw-dylib-stdcall-ordinal
raw-fn-pointer-opt-undefined-behavior
reachable-extern-fn-available-lto
redundant-libs remove unused imports from rmake tests 2024-08-10 18:06:10 +00:00
relocation-model
relro-levels
remap-path-prefix
remap-path-prefix-dwarf rewrite remap-path-prefix-dwarf to rmake 2024-08-12 10:27:20 -04:00
remove-dir-all-race fix: fs::remove_dir_all: treat ENOENT as success 2024-08-22 14:18:42 -04:00
repr128-dwarf
reproducible-build massive refactor of reproducible-build test 2024-08-16 10:10:25 -04:00
reproducible-build-2 remove unused imports from rmake tests 2024-08-10 18:06:10 +00:00
reset-codegen-1 Use path instead of Path in some run-make tests 2024-08-29 10:15:17 -04:00
resolve-rename
return-non-c-like-enum
return-non-c-like-enum-from-c
rlib-chain
rlib-format-packed-bundled-libs rewrite native-link-modifier-bundle to rmake 2024-08-15 10:17:38 -04:00
rlib-format-packed-bundled-libs-2
rlib-format-packed-bundled-libs-3 remove unused imports from rmake tests 2024-08-10 18:06:10 +00:00
rmeta-preferred
run-in-tmpdir-self-test remove unused imports from rmake tests 2024-08-10 18:06:10 +00:00
rust-lld remove unused imports from rmake tests 2024-08-10 18:06:10 +00:00
rust-lld-by-default-beta-stable
rust-lld-by-default-nightly remove unused imports from rmake tests 2024-08-10 18:06:10 +00:00
rust-lld-custom-target remove unused imports from rmake tests 2024-08-10 18:06:10 +00:00
rust-lld-link-script-provide Add a test case for #131164 2024-10-23 19:01:38 +08:00
rustc-crates-on-stable Add rustc_abi and rustc_parse_format to rustc-crates-on-stable test 2024-10-21 14:57:41 +02:00
rustc-macro-dep-files
rustdoc-cfgspec-parsing
rustdoc-default-output rfc#3662 changes under unstable flags 2024-09-07 18:59:01 -04:00
rustdoc-determinism Use path instead of Path in some run-make tests 2024-08-29 10:15:17 -04:00
rustdoc-error-lines
rustdoc-io-error About rmake tests 2024-08-15 15:44:29 +02:00
rustdoc-map-file Reexport serde_json crate from run-make-support to give it access to run-make tests 2024-08-16 16:32:07 +02:00
rustdoc-output-path Use path instead of Path in some run-make tests 2024-08-29 10:15:17 -04:00
rustdoc-output-stdout Actually parse stdout json, instead of using hacky contains logic. 2024-08-31 22:08:38 +00:00
rustdoc-scrape-examples-invalid-expr
rustdoc-scrape-examples-macros About rmake tests 2024-08-15 15:44:29 +02:00
rustdoc-scrape-examples-multiple
rustdoc-scrape-examples-ordering
rustdoc-scrape-examples-remap About rmake tests 2024-08-15 15:44:29 +02:00
rustdoc-scrape-examples-test
rustdoc-scrape-examples-whitespace
rustdoc-shared-flags Reformat using the new identifier sorting from rustfmt 2024-09-22 19:11:29 -04:00
rustdoc-target-spec-json-path About rmake tests 2024-08-15 15:44:29 +02:00
rustdoc-test-args
rustdoc-themes About rmake tests 2024-08-15 15:44:29 +02:00
rustdoc-verify-output-files
rustdoc-with-out-dir-option About rmake tests 2024-08-15 15:44:29 +02:00
rustdoc-with-output-option
rustdoc-with-short-out-dir-option
sanitizer-cdylib-link
sanitizer-dylib-link
sanitizer-staticlib-link
separate-link tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
separate-link-fail
sepcomp-cci-copies remove unused imports from rmake tests 2024-08-10 18:06:10 +00:00
sepcomp-inlining remove unused imports from rmake tests 2024-08-10 18:06:10 +00:00
sepcomp-separate remove unused imports from rmake tests 2024-08-10 18:06:10 +00:00
share-generics-dylib
share-generics-export-again
short-ice
silly-file-names
simd-ffi Ban non-array SIMD 2024-09-09 19:39:43 -07:00
split-debuginfo Reformat using the new identifier sorting from rustfmt 2024-09-22 19:11:29 -04:00
stable-symbol-names
static-dylib-by-default
static-extern-type
static-pie tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
static-unwinding
staticlib-blank-lib
staticlib-dylib-linkage run-make: enable msvc for staticlib-dylib-linkage 2024-08-08 12:17:15 +00:00
staticlib-thin-archive Apply some suggestions to the test rmake file 2024-08-14 16:44:05 +00:00
std-core-cycle
stdin-rustc tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
stdin-rustdoc tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
suspicious-library
symbol-mangling-hashed
symbol-visibility
symbols-include-type-name
symlinked-extern tests/run-make: update for symlink helper changes 2024-09-16 21:31:41 +08:00
symlinked-libraries tests/run-make: update for symlink helper changes 2024-09-16 21:31:41 +08:00
symlinked-rlib tests/run-make: update for symlink helper changes 2024-09-16 21:31:41 +08:00
sysroot-crates-are-unstable tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
target-cpu-native
target-specs Remove is_builtin target spec field 2024-11-03 21:06:49 +00:00
target-without-atomic-cas tests/run-make: add missing needs-llvm-components 2024-08-26 18:20:32 +08:00
test-benches
test-harness
textrel-on-minimal-lib
thumb-none-cortex-m Update run-make tests to use cargo wrapper cmd 2024-09-24 19:04:51 +08:00
thumb-none-qemu Update run-make tests to use cargo wrapper cmd 2024-09-24 19:04:51 +08:00
track-path-dep-info
track-pgo-dep-info Rename directive needs-profiler-support to needs-profiler-runtime 2024-10-09 20:58:27 +11:00
translation
type-mismatch-same-crate-name Reformat using the new identifier sorting from rustfmt 2024-09-22 19:11:29 -04:00
unknown-mod-stdin tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
unstable-flag-required
use-suggestions-rust-2018
used
used-cdylib-macos
version-verbose-commit-hash
volatile-intrinsics
wasm-custom-section
wasm-custom-sections-opt
wasm-exceptions-nostd
wasm-export-all-symbols Reformat using the new identifier sorting from rustfmt 2024-09-22 19:11:29 -04:00
wasm-import-module
wasm-override-linker
wasm-panic-small
wasm-spurious-import
wasm-stringify-ints-small
wasm-symbols-different-module
wasm-symbols-not-exported
wasm-symbols-not-imported
weird-output-filenames
windows-binary-no-external-deps
windows-safeseh
windows-spawn
windows-ws2_32
x86_64-fortanix-unknown-sgx-lvi tests: use renamed stdin_buf 2024-09-05 08:43:38 +00:00
zero-extend-abi-param-passing
README.md Fix unnecessary nesting in run-make test output directories 2024-10-16 12:40:25 +11:00
tools.mk

The run-make test suite

The run-make test suite contains tests which are the most flexible out of all the rust-lang/rust test suites. run-make tests can basically contain arbitrary code, and are supported by the run_make_support library.

Infrastructure

There are two kinds of run-make tests:

  1. The new rmake.rs version: this allows run-make tests to be written in Rust (with rmake.rs as the main test file).
  2. The legacy Makefile version: this is what run-make tests were written with before support for rmake.rs was introduced.

The implementation for collecting and building the rmake.rs recipes (or Makefiles) are in src/tools/compiletest/src/runtest.rs, in run_rmake_v2_test and run_rmake_legacy_test.

Rust-based run-make tests: rmake.rs

The setup for the rmake.rs version is a 3-stage process:

  1. First, we build the run_make_support library in bootstrap as a tool lib.

  2. Then, we compile the rmake.rs "recipe" linking the support library and its dependencies in, and provide a bunch of env vars. We setup a directory structure within build/<target>/test/run-make/

    <test-name>/
        rmake.exe              # recipe binary
        rmake_out/             # sources from test sources copied over
    

    and copy non-rmake.rs input support files over to rmake_out/. The support library is made available as an extern prelude.

  3. Finally, we run the recipe binary and set rmake_out/ as the working directory.

Formatting

Note that files under tests/ are not formatted by ./x fmt, use rustfmt tests/path/to/file.rs to format a specific file if desired.