TokenMap -> SpanMap rewrite
Opening early so I can have an overview over the full diff more easily, still very unfinished and lots of work to be done.
The gist of what this PR does is move away from assigning IDs to tokens in arguments and expansions and instead gives the subtrees the text ranges they are sourced from (made relative to some item for incrementality). This means we now only have a single map per expension, opposed to map for expansion and arguments.
A few of the things that are not done yet (in arbitrary order):
- [x] generally clean up the current mess
- [x] proc-macros, have been completely ignored so far
- [x] syntax fixups, has been commented out for the time being needs to be rewritten on top of some marker SyntaxContextId
- [x] macro invocation syntax contexts are not properly passed around yet, so $crate hygiene does not work in all cases (but most)
- [x] builtin macros do not set spans properly, $crate basically does not work with them rn (which we use)
~~- [ ] remove all uses of dummy spans (or if that does not work, change the dummy entries for dummy spans so that tests will not silently pass due to havin a file id for the dummy file)~~
- [x] de-queryfy `macro_expand`, the sole caller of it is `parse_macro_expansion`, and both of these are lru-cached with the same limit so having it be a query is pointless
- [x] docs and more docs
- [x] fix eager macro spans and other stuff
- [x] simplify include! handling
- [x] Figure out how to undo the sudden `()` expression wrapping in expansions / alternatively prioritize getting invisible delimiters working again
- [x] Simplify InFile stuff and HirFIleId extensions
~~- [ ] span crate containing all the file ids, span stuff, ast ids. Then remove the dependency injection generics from tt and mbe~~
Fixes https://github.com/rust-lang/rust-analyzer/issues/10300
Fixes https://github.com/rust-lang/rust-analyzer/issues/15685
Resolve inlay hint data
Part of https://github.com/rust-lang/rust-analyzer/issues/13962
Support https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#inlayHint_resolve better, by omitting all inlay hint fields specified in the client hint resolve capabilities.
Current list of all capabilities possible to resolve later:
```
"textEdits"
"tooltip"
"label.tooltip"
"label.location"
"label.command"
```
and every one specified in the client capabilities is now resolved by r-a, being omitted in the initial response.
--------------
When editing `inlay_hints.rs` file around line `457` with no resolve capabilities, I get
<details>
<summary>resolved json, 10803 characters</summary>
```json
{"jsonrpc":"2.0","id":55,"result":[{"position":{"line":477,"character":1},"label":[{"value":"fn inlay_hints","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/ide/src/inlay_hints.rs","range":{"start":{"line":445,"character":14},"end":{"line":445,"character":25}}}}],"paddingLeft":true,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":451,"character":10},"label":[{"value":": "},{"value":"ProfileSpan","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/profile/src/hprof.rs","range":{"start":{"line":85,"character":11},"end":{"line":85,"character":22}}}},{"value":""}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":451,"character":27},"label":[{"value":"label:","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/profile/src/hprof.rs","range":{"start":{"line":60,"character":12},"end":{"line":60,"character":17}}}}],"kind":2,"paddingLeft":false,"paddingRight":true,"data":{"file_id":0}},{"position":{"line":452,"character":12},"label":[{"value":": "},{"value":"Semantics","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/hir/src/semantics.rs","range":{"start":{"line":108,"character":11},"end":{"line":108,"character":20}}}},{"value":"<'_, "},{"value":"RootDatabase","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/ide-db/src/lib.rs","range":{"start":{"line":75,"character":11},"end":{"line":75,"character":23}}}},{"value":">"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":453,"character":12},"label":[{"value":": "},{"value":"SourceFile","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/syntax/src/ast/generated/nodes.rs","range":{"start":{"line":223,"character":11},"end":{"line":223,"character":21}}}},{"value":""}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":454,"character":12},"label":[{"value":": &"},{"value":"SyntaxNode","location":{"uri":"file:///Users/someonetoignore/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rowan-0.15.11/src/api.rs","range":{"start":{"line":15,"character":11},"end":{"line":15,"character":21}}}},{"value":"<"},{"value":"RustLanguage","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/syntax/src/syntax_node.rs","range":{"start":{"line":15,"character":9},"end":{"line":15,"character":21}}}},{"value":">"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":456,"character":12},"label":": i32","kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":458,"character":15},"label":[{"value":": "},{"value":"Vec","location":{"uri":"file:///Users/someonetoignore/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/alloc/src/vec/mod.rs","range":{"start":{"line":395,"character":11},"end":{"line":395,"character":14}}}},{"value":"<"},{"value":"InlayHint","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/ide/src/inlay_hints.rs","range":{"start":{"line":149,"character":11},"end":{"line":149,"character":20}}}},{"value":">"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":460,"character":21},"label":[{"value":": "},{"value":"SemanticsScope","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/hir/src/semantics.rs","range":{"start":{"line":1651,"character":11},"end":{"line":1651,"character":25}}}},{"value":"<'_>"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":460,"character":36},"label":[{"value":"node:","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/hir/src/semantics.rs","range":{"start":{"line":482,"character":24},"end":{"line":482,"character":28}}}}],"kind":2,"paddingLeft":false,"paddingRight":true,"data":{"file_id":0}},{"position":{"line":461,"character":23},"label":[{"value":": "},{"value":"FamousDefs","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/ide-db/src/famous_defs.rs","range":{"start":{"line":20,"character":11},"end":{"line":20,"character":21}}}},{"value":"<'_, '_>"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":463,"character":17},"label":[{"value":": impl FnMut("},{"value":"SyntaxNode","location":{"uri":"file:///Users/someonetoignore/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rowan-0.15.11/src/api.rs","range":{"start":{"line":15,"character":11},"end":{"line":15,"character":21}}}},{"value":"<"},{"value":"RustLanguage","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/syntax/src/syntax_node.rs","range":{"start":{"line":15,"character":9},"end":{"line":15,"character":21}}}},{"value":">)"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":463,"character":25},"label":[{"value":": "},{"value":"SyntaxNode","location":{"uri":"file:///Users/someonetoignore/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rowan-0.15.11/src/api.rs","range":{"start":{"line":15,"character":11},"end":{"line":15,"character":21}}}},{"value":"<"},{"value":"RustLanguage","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/syntax/src/syntax_node.rs","range":{"start":{"line":15,"character":9},"end":{"line":15,"character":21}}}},{"value":">"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":463,"character":33},"label":[{"value":"hints:","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/ide/src/inlay_hints.rs","range":{"start":{"line":480,"character":4},"end":{"line":480,"character":9}}}}],"kind":2,"paddingLeft":false,"paddingRight":true,"data":{"file_id":0}},{"position":{"line":465,"character":22},"label":[{"value":": "},{"value":"TextRange","location":{"uri":"file:///Users/someonetoignore/.cargo/registry/src/index.crates.io-6f17d22bba15001f/text-size-1.1.0/src/range.rs","range":{"start":{"line":14,"character":11},"end":{"line":14,"character":20}}}},{"value":""}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":467,"character":35},"label":[{"value":": "},{"value":"SyntaxNode","location":{"uri":"file:///Users/someonetoignore/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rowan-0.15.11/src/api.rs","range":{"start":{"line":15,"character":11},"end":{"line":15,"character":21}}}},{"value":"<"},{"value":"RustLanguage","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/syntax/src/syntax_node.rs","range":{"start":{"line":15,"character":9},"end":{"line":15,"character":21}}}},{"value":">"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":469,"character":92},"label":[{"value":"impl "},{"value":"Iterator","location":{"uri":"file:///Users/someonetoignore/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/iter/traits/iterator.rs","range":{"start":{"line":72,"character":10},"end":{"line":72,"character":18}}}},{"value":"<"},{"value":"Item","location":{"uri":"file:///Users/someonetoignore/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/iter/traits/iterator.rs","range":{"start":{"line":76,"character":9},"end":{"line":76,"character":13}}}},{"value":" = "},{"value":"SyntaxNode","location":{"uri":"file:///Users/someonetoignore/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rowan-0.15.11/src/api.rs","range":{"start":{"line":15,"character":11},"end":{"line":15,"character":21}}}},{"value":"<"},{"value":"RustLanguage","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/syntax/src/syntax_node.rs","range":{"start":{"line":15,"character":9},"end":{"line":15,"character":21}}}},{"value":">>"}],"kind":1,"paddingLeft":true,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":468,"character":34},"label":[{"value":"impl "},{"value":"Iterator","location":{"uri":"file:///Users/someonetoignore/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/iter/traits/iterator.rs","range":{"start":{"line":72,"character":10},"end":{"line":72,"character":18}}}},{"value":"<"},{"value":"Item","location":{"uri":"file:///Users/someonetoignore/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/iter/traits/iterator.rs","range":{"start":{"line":76,"character":9},"end":{"line":76,"character":13}}}},{"value":" = "},{"value":"SyntaxNode","location":{"uri":"file:///Users/someonetoignore/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rowan-0.15.11/src/api.rs","range":{"start":{"line":15,"character":11},"end":{"line":15,"character":21}}}},{"value":"<"},{"value":"RustLanguage","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/syntax/src/syntax_node.rs","range":{"start":{"line":15,"character":9},"end":{"line":15,"character":21}}}},{"value":">>"},{"value":""}],"kind":1,"paddingLeft":true,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":467,"character":41},"label":[{"value":""},{"value":"SyntaxNode","location":{"uri":"file:///Users/someonetoignore/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rowan-0.15.11/src/api.rs","range":{"start":{"line":15,"character":11},"end":{"line":15,"character":21}}}},{"value":"<"},{"value":"RustLanguage","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/syntax/src/syntax_node.rs","range":{"start":{"line":15,"character":9},"end":{"line":15,"character":21}}}},{"value":">"}],"kind":1,"paddingLeft":true,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":469,"character":40},"label":" -> bool","kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":469,"character":39},"label":[{"value":": &"},{"value":"SyntaxNode","location":{"uri":"file:///Users/someonetoignore/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rowan-0.15.11/src/api.rs","range":{"start":{"line":15,"character":11},"end":{"line":15,"character":21}}}},{"value":"<"},{"value":"RustLanguage","location":{"uri":"file:///Users/someonetoignore/work/rust-analyzer/crates/syntax/src/syntax_node.rs","range":{"start":{"line":15,"character":9},"end":{"line":15,"character":21}}}},{"value":">"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}}]}
```
</details>
for the visible editor range alone, pretty much repeated on every consequent edit.
With this patch and all inlay hint resolve capabilities enabled, for the same example I observe quite a footprint reduction:
<details>
<summary>unresolved json, 4142 characters</summary>
```json
{"jsonrpc":"2.0","id":49,"result":[{"position":{"line":477,"character":1},"label":[{"value":"fn inlay_hints"}],"paddingLeft":true,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":451,"character":10},"label":[{"value":": "},{"value":"ProfileSpan"},{"value":""}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":451,"character":27},"label":[{"value":"label:"}],"kind":2,"paddingLeft":false,"paddingRight":true,"data":{"file_id":0}},{"position":{"line":452,"character":12},"label":[{"value":": "},{"value":"Semantics"},{"value":"<'_, "},{"value":"RootDatabase"},{"value":">"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":453,"character":12},"label":[{"value":": "},{"value":"SourceFile"},{"value":""}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":454,"character":12},"label":[{"value":": &"},{"value":"SyntaxNode"},{"value":"<"},{"value":"RustLanguage"},{"value":">"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":456,"character":12},"label":": i32","kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":458,"character":15},"label":[{"value":": "},{"value":"Vec"},{"value":"<"},{"value":"InlayHint"},{"value":">"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":460,"character":21},"label":[{"value":": "},{"value":"SemanticsScope"},{"value":"<'_>"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":460,"character":36},"label":[{"value":"node:"}],"kind":2,"paddingLeft":false,"paddingRight":true,"data":{"file_id":0}},{"position":{"line":461,"character":23},"label":[{"value":": "},{"value":"FamousDefs"},{"value":"<'_, '_>"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":463,"character":17},"label":[{"value":": impl FnMut("},{"value":"SyntaxNode"},{"value":"<"},{"value":"RustLanguage"},{"value":">)"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":463,"character":25},"label":[{"value":": "},{"value":"SyntaxNode"},{"value":"<"},{"value":"RustLanguage"},{"value":">"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":463,"character":33},"label":[{"value":"hints:"}],"kind":2,"paddingLeft":false,"paddingRight":true,"data":{"file_id":0}},{"position":{"line":465,"character":22},"label":[{"value":": "},{"value":"TextRange"},{"value":""}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":467,"character":35},"label":[{"value":": "},{"value":"SyntaxNode"},{"value":"<"},{"value":"RustLanguage"},{"value":">"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":469,"character":92},"label":[{"value":"impl "},{"value":"Iterator"},{"value":"<"},{"value":"Item"},{"value":" = "},{"value":"SyntaxNode"},{"value":"<"},{"value":"RustLanguage"},{"value":">>"}],"kind":1,"paddingLeft":true,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":468,"character":34},"label":[{"value":"impl "},{"value":"Iterator"},{"value":"<"},{"value":"Item"},{"value":" = "},{"value":"SyntaxNode"},{"value":"<"},{"value":"RustLanguage"},{"value":">>"},{"value":""}],"kind":1,"paddingLeft":true,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":467,"character":41},"label":[{"value":""},{"value":"SyntaxNode"},{"value":"<"},{"value":"RustLanguage"},{"value":">"}],"kind":1,"paddingLeft":true,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":469,"character":40},"label":" -> bool","kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}},{"position":{"line":469,"character":39},"label":[{"value":": &"},{"value":"SyntaxNode"},{"value":"<"},{"value":"RustLanguage"},{"value":">"}],"kind":1,"paddingLeft":false,"paddingRight":false,"data":{"file_id":0}}]}
```
</details>
with all unresolved parts needing only for navigation, hover or applying the hint edit — dynamic parts that are made after mouse hover or similar events, that resolve the hint data.
On type format '(', by adding closing ')' automatically
If I understand right, `()` can surround pretty much the same `{}` can, so add another on type formatting pair for convenience: sometimes it's not that pleasant to write parenthesis in `Some(2).map(|i| (i, i+1))` cases and I would prefer r-a to do that for me.
One note: currently, b06503b6ec/crates/rust-analyzer/src/handlers/request.rs (L357) fires always.
Should we remove the assertion entirely now, since apparently things work in release despite that check?
Fix signature help of methods from macros
Currently the receiver type is copied from AST instead re-formatting through `HirDisplay`. Macro generated functions seem to have no spaces and their signature help are rendered like `fn foo(&'amutself)` instead of `fn foo(&'a mut self)`.
Support doc links that resolve to fields
Fixes#15331
Also removes `Resolver::resolve_module_path_in_trait_assoc_items()` and reimplements it in hir with other `Resolver` methods to decouple things a bit.
Handle `#[cfg]`s on generic parameters
Records attributes on generic parameters in the item tree and filters out generic parameters disabled by `#[cfg]`s in `generic_params_query`.
Closes#11756
SCIP requires symbols to be unique, but multiple functions may have a
parameter with the same name. Qualify parameters according to the
containing function.
Added remove unused imports assist
This resolves the most important part of #5131. I needed to make a couple of cosmetic changes to the search infrastructure to do this.
A few open questions:
* Should imports that don't resolve to anything be considered unused? I figured probably not, but it would be a trivial change to make if we want it.
* Is there a cleaner way to make the edits to the use list?
* Is there a cleaner way to get the list of uses that intersect the current selection?
* Is the performance acceptable? When testing this on itself, it takes a good couple seconds to perform the assist.
* Is there a way to hide the rustc diagnostics that overlap with this functionality?
internal: Defer structured snippet rendering to allow escaping snippet bits
Since we know exactly where snippets are, we can transparently escape snippet bits to the exact text edits that need it, and not have to do it for anything other text edits.
Also will eventually fix#11006 once all assists are migrated. This comes as a side-effect of text edits that don't have snippets get marked as having no insert formatting at all.
internal: Format let-else
As nightly finally got support for it I went ahead and formatted r-a with the latest nightly, then with the latest stable (in case other stuff changed)
Fix runnable detection for `#[tokio::test]`
fix#15141
It is hacky, and it wouldn't work for e.g. this case:
```Rust
use ::core::prelude;
#[prelude::v1::test]
fn foo() {
}
```
But it works for the tokio case. We should use the name resolution here somehow, and after that we should probably also get rid of the ast based `test_related_attribute` function.
internal: add `library` fixture meta
Currently, there is no way to specify `CrateOrigin` of a file fixture ([this] might be a bug?). This PR adds `library` meta to explicitly specify the fixture to be `CrateOrigin::Library` and also makes sure crates that belong to a library source root are set `CrateOrigin::Library`.
(`library` isn't really the best name. It essentially means that the crate is outside workspace but `non_workspace_member` feels a bit too long. Suggestions for the better name would be appreciated)
Additionally:
- documents the fixture meta syntax as thoroughly as possible
- refactors relevant code
[this]: 4b06d3c595/crates/base-db/src/fixture.rs (L450)
internal: Add run-tests command
This command is similar to `cargo test` except that it uses r-a to run tests instead of compiling and running them with rustc. This is slower than `cargo test` and it is only useful for me to see a bird view of what needs to be fixed. The current output is:
```
48 passed, 5028 failed, 2 ignored
All tests 174.74s, 648ginstr
```
48 is very low, but higher than what I originally thought.
Now that there is some passing tests, I can show the plan:
https://github.com/rust-lang/rust-analyzer/assets/45197576/76d7d777-1843-4ca4-b7fe-e463bdade6cb
That is, at the end, I want to be able to immediately re run every test after every change. (0.5s is not really immediate, but it's not finished yet, and it is way better than 8s that running a typical test in r-a will take on my system)
Added a test near positive extermes and two test near negative
extermes as well one for 0.
Added a test using the `as` cast and one with comparison with 0.
fix: Fix nav target calculation discarding file ids from differing macro upmapping
Fixes https://github.com/rust-lang/rust-analyzer/issues/14792
Turns out there was the assumption that upmapping from a macro will always end in the same root file, which is no longer the case thanks to `include!`
Add signature help for tuple patterns and expressions
~~These are somewhat wonky since their signature changes as you type depending on context but they help out nevertheless.~~ should be less wonky now with added parser and lowering recoveries
fix: Don't duplicate sysroot crates in rustc workspace
Since we handle `library` as the sysroot source directly in the rustc workspace, we now duplicate the crates there, once as sysroot and once as just plain workspace crate. This causes a variety of issues for `vec!` macros and similar that emit `$crate` tokens across crates.
