Nicholas Nethercote 925f7fad57 Improve print_tts by changing tokenstream::Spacing. ... `tokenstream::Spacing` appears on all `TokenTree::Token` instances, both punct and non-punct. Its current usage: - `Joint` means "can join with the next token *and* that token is a punct". - `Alone` means "cannot join with the next token *or* can join with the next token but that token is not a punct". The fact that `Alone` is used for two different cases is awkward. This commit augments `tokenstream::Spacing` with a new variant `JointHidden`, resulting in: - `Joint` means "can join with the next token *and* that token is a punct". - `JointHidden` means "can join with the next token *and* that token is a not a punct". - `Alone` means "cannot join with the next token". This *drastically* improves the output of `print_tts`. For example, this: ``` stringify!(let a: Vec<u32> = vec![];) ``` currently produces this string: ``` let a : Vec < u32 > = vec! [] ; ``` With this PR, it now produces this string: ``` let a: Vec<u32> = vec![] ; ``` (The space after the `]` is because `TokenTree::Delimited` currently doesn't have spacing information. The subsequent commit fixes this.) The new `print_tts` doesn't replicate original code perfectly. E.g. multiple space characters will be condensed into a single space character. But it's much improved. `print_tts` still produces the old, uglier output for code produced by proc macros. Because we have to translate the generated code from `proc_macro::Spacing` to the more expressive `token::Spacing`, which results in too much `proc_macro::Along` usage and no `proc_macro::JointHidden` usage. So `space_between` still exists and is used by `print_tts` in conjunction with the `Spacing` field. This change will also help with the removal of `Token::Interpolated`. Currently interpolated tokens are pretty-printed nicely via AST pretty printing. `Token::Interpolated` removal will mean they get printed with `print_tts`. Without this change, that would result in much uglier output for code produced by decl macro expansions. With this change, AST pretty printing and `print_tts` produce similar results. The commit also tweaks the comments on `proc_macro::Spacing`. In particular, it refers to "compound tokens" rather than "multi-char operators" because lifetimes aren't operators.		2023-12-11 09:19:09 +11:00
..
a-b-a-linker-guard
alloc-no-oom-handling
alloc-no-rc
alloc-no-sync
allocator-shim-circular-deps
allow-non-lint-warnings-cmdline
allow-warnings-cmdline-stability
archive-duplicate-names
arguments-non-c-like-enum
atomic-lock-free
bare-outfile
branch-protection-check-IBT
c-dynamic-dylib
c-dynamic-rlib
c-link-to-rust-dylib
c-link-to-rust-staticlib
c-link-to-rust-va-list-fn
c-static-dylib
c-static-rlib
c-unwind-abi-catch-lib-panic
c-unwind-abi-catch-panic
cat-and-grep-sanity-check
cdylib
cdylib-dylib-linkage
cdylib-fewer-symbols
codegen-options-parsing
comment-section
compile-stdin
compiler-lookup-paths
compiler-lookup-paths-2
compiler-rt-works-on-mingw
compressed-debuginfo	Update the minimum external LLVM to 16.	2023-11-21 22:40:16 +01:00
const_fn_mir	tests/run-make: Use RUSTC_TEST_OP in more places	2023-10-07 08:30:40 +02:00
const-prop-lint
core-no-fp-fmt-parse
core-no-oom-handling
coverage-llvmir	coverage: Emit the filenames section before encoding per-function mappings	2023-10-22 23:17:15 +11:00
crate-data-smoke
crate-hash-rustc-version	Remove support for alias -Z symbol-mangling-version	2023-11-02 18:41:33 +11:00
crate-name-priority
cross-lang-lto
cross-lang-lto-clang
cross-lang-lto-pgo-smoketest
cross-lang-lto-upstream-rlibs
CURRENT_RUSTC_VERSION
debug-assertions
debugger-visualizer-dep-info
dep-graph
dep-info
dep-info-doesnt-run-much
dep-info-spaces
doctests-keep-binaries
doctests-runtool
dump-ice-to-disk	Use YYYY-MM-DDTHH_MM_SS as datetime format for ICE dump files	2023-10-17 17:31:47 +00:00
dump-mono-stats
duplicate-output-flavors
dylib-chain
emit
emit-named-files
emit-path-unhashed	Fix tests	2023-09-10 15:50:48 +00:00
emit-shared-files
emit-stack-sizes	Automatically enable cross-crate inlining for small functions	2023-10-17 19:53:51 -04:00
emit-to-stdout	Bless run-make tests	2023-11-24 19:15:52 +01:00
env-dep-info
error-found-staticlib-instead-crate
error-writing-dependencies
exit-code
export-executable-symbols
extern-diff-internal-name
extern-flag-disambiguates
extern-flag-fun
extern-flag-pathless
extern-flag-rename-transitive
extern-fn-explicit-align
extern-fn-generic
extern-fn-mangle
extern-fn-reachable
extern-fn-struct-passing-abi
extern-fn-with-extern-types
extern-fn-with-packed-struct
extern-fn-with-union
extern-multiple-copies
extern-multiple-copies2
extern-overrides-distribution
extra-filename-with-temp-outputs
fmt-write-bloat
forced-unwind-terminate-pof
foreign-double-unwind
foreign-exceptions
foreign-rust-exceptions
fpic
glibc-staticlib-args
hir-tree
inaccessible-temp-dir
include_bytes_deps
incr-add-rust-src-component
incr-foreign-head-span	Validate crate name in CLI option --extern	2023-09-20 18:50:40 +02:00
incr-prev-body-beyond-eof
incremental-debugger-visualizer
incremental-session-fail
inline-always-many-cgu
interdependent-c-libraries
intrinsic-unreachable	Automatically enable cross-crate inlining for small functions	2023-10-17 19:53:51 -04:00
invalid-library
invalid-so
invalid-staticlib
issue64319
issue-7349
issue-10971-temps-dir
issue-11908
issue-14500
issue-14698
issue-15460
issue-18943
issue-20626
issue-22131
issue-24445
issue-25581
issue-26006
issue-26092
issue-28595
issue-28766
issue-30063
issue-33329
issue-35164
issue-36710
issue-37839
issue-37893
issue-38237
issue-40535
issue-46239
issue-47384
issue-47551
issue-51671
issue-53964
issue-64153
issue-68794-textrel-on-minimal-lib
issue-69368
issue-83045
issue-83112-incr-test-moved-file
issue-84395-lto-embed-bitcode
issue-85019-moved-src-dir
issue-85401-static-mir
issue-85441
issue-88756-default-output	Delete obsolete --disable-per-crate-search rustdoc flag	2023-09-20 22:09:48 -07:00
issue-97463-abi-param-passing
issue-107094
issue-109934-lto-debuginfo
jobserver-error	1. fix jobserver GLOBAL_CLIENT_CHECKED uninitialized before use	2023-12-08 09:50:28 +08:00
libs-through-symlinks
libtest-json
libtest-junit
libtest-thread-limit
link-arg
link-args-order
link-cfg
link-dedup
link-path-order
linkage-attr-on-static
llvm-ident
llvm-outputs
long-linker-command-lines
long-linker-command-lines-cmd-exe
longjmp-across-rust
ls-metadata	Fix tests	2023-09-10 15:50:48 +00:00
lto-dylib-dep
lto-empty
lto-linkage-used-attr	Add a test for #108030	2023-09-11 22:01:55 +08:00
lto-no-link-whole-rlib
lto-readonly-lib
lto-smoke
lto-smoke-c
macos-deployment-target	Raise minimum supported macOS to 10.12	2023-09-23 19:14:25 -05:00
macos-fat-archive
manual-crate-name
manual-link
many-crates-but-no-match
metadata-dep-info	tests/run-make: Move RUSTC_TEST_OP to tools.mk	2023-10-07 08:30:40 +02:00
metadata-flag-frobs-symbols
min-global-align
mingw-export-call-convention
mismatching-target-triples
missing-crate-dependency
mixing-deps
mixing-formats
mixing-libs
msvc-opt-minsize
multiple-emits
native-link-modifier-bundle
native-link-modifier-verbatim-linker
native-link-modifier-verbatim-rustc
native-link-modifier-whole-archive
no-alloc-shim
no-builtins-attribute
no-builtins-lto	Put $(LLVM_BIN_DIR) in quotes to prevent missing backslashes	2023-12-02 04:40:12 +08:00
no-builtins-symbols	Avoid adding compiler-used functions to symbols.o	2023-12-04 22:28:00 +08:00
no-cdylib-as-rdylib
no-duplicate-libs
no-input-file
no-intermediate-extras
obey-crate-type-flag
optimization-remarks-dir
optimization-remarks-dir-pgo
output-filename-conflicts-with-directory
output-filename-overwrites-input	Fix tests	2023-09-10 15:50:48 +00:00
output-type-permutations
output-with-hyphens
override-aliased-flags
overwrite-input	Bless run-make tests	2023-11-24 19:15:52 +01:00
panic-abort-eh_frame
panic-impl-transitive
pass-linker-flags	Enable link-arg link kind inside of #[link] attribute	2023-11-30 08:26:13 -08:00
pass-linker-flags-flavor	Enable link-arg link kind inside of #[link] attribute	2023-11-30 08:26:13 -08:00
pass-linker-flags-from-dep	Enable link-arg link kind inside of #[link] attribute	2023-11-30 08:26:13 -08:00
pass-non-c-like-enum-to-c
pdb-buildinfo-cl-cmd	Add missing Debuginfo to PDB debug file on windows.	2023-09-08 00:28:40 +02:00
pgo-branch-weights
pgo-gen
pgo-gen-lto
pgo-gen-no-imp-symbols
pgo-indirect-call-promotion
pgo-use
pointer-auth-link-with-c
prefer-dylib
prefer-rlib
pretty-print-to-file
pretty-print-with-dep-file
print-calling-conventions
print-cfg	Stabilize PATH option for --print KIND=PATH	2023-09-07 15:07:30 +02:00
print-native-static-libs
print-target-list
profile
prune-link-args
raw-dylib-alt-calling-convention	tests/run-make: Use RUSTC_TEST_OP in more places	2023-10-07 08:30:40 +02:00
raw-dylib-c
raw-dylib-cross-compilation
raw-dylib-custom-dlltool
raw-dylib-import-name-type	tests/run-make: Use RUSTC_TEST_OP in more places	2023-10-07 08:30:40 +02:00
raw-dylib-inline-cross-dylib	tests/run-make: Use RUSTC_TEST_OP in more places	2023-10-07 08:30:40 +02:00
raw-dylib-link-ordinal	tests/run-make: Use RUSTC_TEST_OP in more places	2023-10-07 08:30:40 +02:00
raw-dylib-stdcall-ordinal	tests/run-make: Use RUSTC_TEST_OP in more places	2023-10-07 08:30:40 +02:00
redundant-libs
relocation-model
relro-levels
remap-path-prefix	Test -Zremap-path-scope=split-debuginfo with -Csplit-debuginfo=packed	2023-10-17 10:11:31 +02:00
remap-path-prefix-dwarf	Adjust tests for MacOS having different -Csplit-debuginfo default	2023-10-17 10:11:31 +02:00
repr128-dwarf
reproducible-build
reproducible-build-2
resolve-rename
return-non-c-like-enum
return-non-c-like-enum-from-c
rlib-chain
rlib-format-packed-bundled-libs
rlib-format-packed-bundled-libs-2
rlib-format-packed-bundled-libs-3
rmeta-preferred
rust-lld
rust-lld-custom-target	add end-to-end test of custom target using rust-lld	2023-10-18 11:33:40 +00:00
rustc-macro-dep-files	Improve print_tts by changing tokenstream::Spacing.	2023-12-11 09:19:09 +11:00
rustdoc-determinism
rustdoc-error-lines
rustdoc-io-error
rustdoc-map-file
rustdoc-output-path
rustdoc-scrape-examples-invalid-expr
rustdoc-scrape-examples-macros
rustdoc-scrape-examples-multiple
rustdoc-scrape-examples-ordering
rustdoc-scrape-examples-remap
rustdoc-scrape-examples-test
rustdoc-scrape-examples-whitespace
rustdoc-shared-flags
rustdoc-target-spec-json-path
rustdoc-themes	rustdoc: merge theme css into rustdoc.css	2023-09-15 07:40:17 -07:00
rustdoc-verify-output-files
rustdoc-with-out-dir-option
rustdoc-with-output-option
rustdoc-with-short-out-dir-option
sanitizer-cdylib-link
sanitizer-dylib-link
sanitizer-staticlib-link
separate-link
separate-link-fail
sepcomp-cci-copies
sepcomp-inlining
sepcomp-separate
share-generics-dylib
short-ice
silly-file-names	tests/run-make: Move RUSTC_TEST_OP to tools.mk	2023-10-07 08:30:40 +02:00
simd-ffi
simple-dylib
simple-rlib
split-debuginfo	[RFC 3127 - Trim Paths]: Adjust tests for -Zremap-path-scope	2023-10-17 10:11:31 +02:00
stable-symbol-names
static-dylib-by-default
static-extern-type
static-pie
static-unwinding
staticlib-blank-lib
staticlib-dylib-linkage
std-core-cycle
stdin-non-utf8
suspicious-library
symbol-visibility
symbols-include-type-name
symlinked-extern
symlinked-libraries
symlinked-rlib
sysroot-crates-are-unstable
target-cpu-native
target-specs
target-without-atomic-cas
test-benches
test-harness
thumb-none-cortex-m
thumb-none-qemu
track-path-dep-info
track-pgo-dep-info
translation
type-mismatch-same-crate-name
unknown-mod-stdin	Bless run-make tests	2023-11-24 19:15:52 +01:00
unstable-flag-required
use-suggestions-rust-2018
used
used-cdylib-macos
valid-print-requests	Sort PRINT_KINDS.	2023-11-30 17:28:29 +11:00
version
volatile-intrinsics
wasm-abi
wasm-builtins-import	Update tests/run-make/wasm-builtins-import/main.rs	2023-10-17 11:31:30 -04:00
wasm-custom-section
wasm-custom-sections-opt
wasm-exceptions-nostd
wasm-export-all-symbols
wasm-import-module
wasm-override-linker	add test for wasm linker override=clang	2023-10-09 11:42:23 +09:00
wasm-panic-small
wasm-stringify-ints-small
wasm-symbols-different-module
wasm-symbols-not-exported
wasm-symbols-not-imported
weird-output-filenames
windows-binary-no-external-deps
windows-safeseh	Mark .rmeta files as /SAFESEH on x86 Windows.	2023-10-24 13:42:28 -07:00
windows-spawn
windows-subsystem
x86_64-fortanix-unknown-sgx-lvi
git_clone_sha1.sh
tools.mk	tests/run-make: Move RUSTC_TEST_OP to tools.mk	2023-10-07 08:30:40 +02:00