fix: Don't expand macros in the same expansion tree after overflow
This patch fixes 2 bugs:
- In `Expander::enter_expand_id()` (and in code paths it's called), we never check whether we've reached the recursion limit. Although it hasn't been reported as far as I'm aware, this may cause hangs or stack overflows if some malformed attribute macro is used on associated items.
- We keep expansion even when recursion limit is reached. Take the following for example:
```rust
macro_rules! foo { () => {{ foo!(); foo!(); }} }
fn main() { foo!(); }
```
We keep expanding the first `foo!()` in each expansion and would reach the limit at some point, *after which* we would try expanding the second `foo!()` in each expansion until it hits the limit again. This will (by default) lead to ~2^128 expansions.
This is essentially what's happening in #14074. Unlike rustc, we don't just stop expanding macros when we fail as long as it produces some tokens so that we can provide completions and other services in incomplete macro calls.
This patch provides a method that takes care of recursion depths (`Expander::within_limit()`) and stops macro expansions in the whole macro expansion tree once it detects recursion depth overflow. To be honest, I'm not really satisfied with this fix because it can still be used in unintended ways to bypass overflow checks, and I'm still seeking ways such that misuses are caught by the compiler by leveraging types or something.
Fixes#14074
Support multi-character punct tokens in MBE
Fixes#11497
In the context of MBE, consecutive puncts are parsed as multi-character punct tokens whenever possible. For example, `:::` is parsed as ``[Punct(`::`), Punct(`:`)]`` and shouldn't get matched to patterns like `: : :` or `: ::`.
We have implemented this behavior only for when we match puncts against `tt` fragments, but not when we match puncts literally. This PR extracts the multi-character punct handling procedure into a separate method and extends its support for literal matching.
For good measure, this PR adds support for `<-` token, which is still [considered as one token in rustc](e396186407/compiler/rustc_ast/src/token.rs (L249)) despite the placement syntax having been removed.
Use `rustc_safe_intrinsic` attribute to check for intrinsic safety
Instead of maintaining a list that is poorly kept in sync we can just use the attribute.
This will make new RA versions unusable with old toolchains that don't have the attribute yet. Should we keep maintaining the list as a fallback or just don't care?
This makes code more readale and concise,
moving all format arguments like `format!("{}", foo)`
into the more compact `format!("{foo}")` form.
The change was automatically created with, so there are far less change
of an accidental typo.
```
cargo clippy --fix -- -A clippy::all -W clippy::uninlined_format_args
```
Seems like these can be safely fixed. With one, I was particularly
surprised -- `Some(pats) => &**pats,` in body.rs?
```
cargo clippy --fix -- -A clippy::all -D clippy::explicit_auto_deref
```
I am not certain if this will improve performance,
but it seems having a .clone() without any need should be removed.
This was done with clippy, and manually reviewed:
```
cargo clippy --fix -- -A clippy::all -D clippy::redundant_clone
```
Compute data layout of types
cc #4091
Things that aren't working:
* Closures
* Generators (so no support for `Future` I think)
* Opaque types
* Type alias and associated types which may need normalization
Things that show wrong result:
* ~Enums with explicit discriminant~
* SIMD types
* ~`NonZero*` and similar standard library items which control layout with special attributes~
At the user level, I didn't put much work, since I wasn't confident about what is the best way to present this information. Currently it shows size and align for ADTs, and size, align, offset for struct fields, in the hover, similar to clangd. I used it some days and I feel I liked it, but we may consider it too noisy and move it to an assist or command.
Mega-sync from `rust-lang/rust`
This essentially implements `@oli-obk's` suggestion here https://github.com/rust-lang/rust-analyzer/pull/13459#issuecomment-1297285607, with `@eddyb's` help.
This PR is equivalent to 14 syncs (back and forth) between `rust-lang/rust` and `rust-lang/rust-analyzer`.
Working from this list (from bottom to top):
```
(x) a2a1d9954⬆️ rust-analyzer
(x) 79923c382⬆️ rust-analyzer
(x) c60b1f641⬆️ rust-analyzer
(x) 8807fc4cc⬆️ rust-analyzer
(x) a99a48e78⬆️ rust-analyzer
(x) 4f55ebbd4⬆️ rust-analyzer
(x) f5fde4df4⬆️ rust-analyzer
(x) 459bbb422⬆️ rust-analyzer
(x) 65e1dc4d9⬆️ rust-analyzer
(x) 3e358a682⬆️ rust-analyzer
(x) 31519bb39⬆️ rust-analyzer
(x) 8231fee46⬆️ rust-analyzer
(x) 22c8c9c40⬆️ rust-analyzer
(x) 9d2cb42a4⬆️ rust-analyzer
```
(This listed was assembled by doing a `git subtree push`, which made a branch, and looking at the new commits in that branch, picking only those that were `⬆️ rust-analyzer` commits)
We used the following commands to simulate merges in both directions:
```shell
TO_MERGE=22c8c9c40 # taken from the list above, bottom to top
git merge --no-edit --no-ff $TO_MERGE
git merge --no-edit --no-ff $(git -C ../rust log --pretty=format:'%cN | %s | %ad => %P' | rg -m1 -F "$(git show --no-patch --pretty=format:%ad $TO_MERGE)" | tee /dev/stderr | rg '.* => \S+ (\S+)$' --replace '$1')
```
We encountered no merge conflicts that Git wasn't able to solve by doing it this way.
Here's what the commit graph looks like (as shown in the Git Lens VSCode extension):
<img width="1345" alt="image" src="https://user-images.githubusercontent.com/7998310/203984523-7c1a690a-8224-416c-8015-ed6e49667066.png">
This PR closes#13459
## Does this unbreak `rust->ra` syncs?
Yes, here's how we tried:
In `rust-analyzer`:
* check out `subtree-fix` (this PR's branch)
* make a new branch off of it: `git checkout -b subtree-fix-merge-test`
* simulate this PR getting merged with `git merge master`
In `rust`:
* pull latest master
* make a new branch: `git checkout -b test-change`
* mess with rust-analyzer (I added a comment to `src/tools/rust-analyzer/Cargo.toml`)
* commit
* run `git subtree push -P src/tools/rust-analyzer ra-local final-sync` (this follows the [Clippy sync guide](https://doc.rust-lang.org/nightly/clippy/development/infrastructure/sync.html))
This created a `final-sync` branch in `rust-analyzer`.
In `rust-analyzer`:
* `git merge --no-ff final-sync` (this follows the [Clippy sync guide](https://doc.rust-lang.org/nightly/clippy/development/infrastructure/sync.html))
Now `git log` in `rust-analyzer` shows this:
```
commit 460128387e46ddfc2b95921b2d7f6e913a3d2b9f (HEAD -> subtree-fix-merge-test)
Merge: 0513fc02a 9ce6a734f
Author: Amos Wenger <amoswenger@gmail.com>
Date: Fri Nov 25 13:28:24 2022 +0100
Merge branch 'final-sync' into subtree-fix-merge-test
commit 0513fc02a08ea9de952983624bd0a00e98044b36
Merge: 38c98d1ff6918009fe
Author: Amos Wenger <amoswenger@gmail.com>
Date: Fri Nov 25 13:28:02 2022 +0100
Merge branch 'master' into subtree-fix-merge-test
commit 9ce6a734f37ef8e53689f1c6f427a9efafe846bd (final-sync)
Author: Amos Wenger <amoswenger@gmail.com>
Date: Fri Nov 25 13:26:26 2022 +0100
Mess with rust-analyzer just for fun
```
And `git diff 0513fc02a08ea9de952983624bd0a00e98044b36` shows this:
```patch
diff --git a/Cargo.toml b/Cargo.toml
index 286ef1e7d..c9e24cd19 100644
--- a/Cargo.toml
+++ b/Cargo.toml
`@@` -32,3 +32,5 `@@` debug = 0
# ungrammar = { path = "../ungrammar" }
# salsa = { path = "../salsa" }
+
+# lol, hi
```
## Does this unbreak `ra->rust` syncs?
Yes, here's how we tried.
From `rust`:
* `git checkout -b sync-from-ra`
* `git subtree pull -P src/tools/rust-analyzer ra-local subtree-fix-merge-test` (this is adapted from the [Clippy sync guide](https://doc.rust-lang.org/nightly/clippy/development/infrastructure/sync.html#performing-the-sync-from-clippy-to-rust-langrust), you would normally use `ra-upstream master` but we're simulating things here)
A commit editor pops up, there was no merge conflicts.
## How do we prevent this from happening again?
Like `@bjorn3` said in https://github.com/rust-lang/rust-analyzer/pull/13459#issuecomment-1293587848
> Whenever syncing from rust-analyzer -> rust you have to immediately sync the merge commit from rust -> rust-analyzer to prevent merge conflicts in the future.
But if we get it wrong again, at least now we have a not-so-painful way to fix it.
fix: check visibility of each path segment
Upon path resolution, we have not been checking if every def pointed to by each segment of the path is visible from the original module. This leads to incorrect import resolutions, in particular when one uses glob imports and names collide.
There is decent amount of changes in this PR because:
- some of our tests were not correct in terms of visibility
- I left several basic nameres tests as-is (with expect test updated) since I thought it would be nice to ensure we don't resolve defs that are not visible.
- `fix_visibility` assist relied on `Semantics::resolve_path()`, which uses the name resolution procedure I'm fixing and wouldn't be able to "see through" the items with strict visibility with this patch
The first commit is the gist of the fix itself.
Fixes#10991Fixes#11473Fixes#13252
Fix `tt::Punct`'s spacing calculation
Fixes#13499
We currently set a `tt::Punct`'s spacing to `Spacing::Joint` unless its next token is a trivia (i.e. whitespaces or comment). As I understand it, rustc only [sets `Spacing::Joint` if the next token is an operator](5b3e909075/compiler/rustc_parse/src/lexer/tokentrees.rs (L77-L78)) and we should follow it to guarantee the consistent behavior of proc macros.
Use $crate instead of std for panic builtin_fn_macro
This should be closer to the expected output and gets rid of a few type mismatches in rustc/library
feat: type inference for generators
This PR implements basic type inference for generator and yield expressions.
Things not included in this PR:
- Generator upvars and generator witnesses are not implemented. They are only used to determine auto trait impls, so basic type inference should be fine without them, but method resolutions with auto trait bounds may not be resolved correctly.
Open questions:
- I haven't (yet) implemented `HirDisplay` for `TyKind::Generator`, so generator types are just shown as "{{generator}}" (in tests, inlay hints, hovers, etc.), which is not really nice. How should we show them?
- I added moderate amount of stuffs to minicore. I especially didn't want to add `impl<T> Deref for &T` and `impl<T> Deref for &mut T` exclusively for tests for generators; should I move them into the test fixtures or can they be placed in minicore?
cc #4309
feat: Display the value of enum variant on hover
fixes#12955
This PR adds const eval support for enums, as well as showing their value on hover, just as consts currently have.
I developed these two things at the same time, but I've realized now that they are separate. However since the hover is just a 10 line change (not including tests), I figured I may as well put them in the same PR. Though if you want them split up into "enum const eval support" and "show enum variant value on hover", I think that's reasonable too.
Since this adds const eval support for enums this also allows consts that reference enums to have their values computed now too.
The const evaluation itself is quite rudimentary, it doesn't keep track of the actual type of the enum, but it turns out that Rust doesn't actually either, and `E::A as u8` is valid regardless of the `repr` on `E`.
It also doesn't really care about what expression the enum variant contains, it could for example be a string, despite that not being allowed, but I guess it's up to the `cargo check` diagnostics to inform of such issues anyway?
A Resolver *always* has a module scope at the end of its scope stack,
instead of encoding this as an invariant we can just lift this scope
out into a field, allowing us to skip going through the scope vec
indirection entirely.
feat: Implement `feature(exhaustive_patterns)` from unstable Rust
Closes#12753
Recognize Rust's unstable `#![feature(exhaustive_patterns)]` (RFC 1872). Allow omitting visibly uninhabited variants from `match` expressions when the feature is on.
This adjusts match checking to the current implementation of the postponed RFC 1872 in rustc.
This hir expression isn't needed and only existed as it was simpler to
deal with at first as it gave us a direct mapping for the ast version of
the same construct. This PR removes it, properly handling the statements
that are introduced by macro call expressions.
This PR will fix some typos detected by [typos].
There are also some other typos in the function names, variable names, and file
names, which I leave as they are. I'm more certain that typos in comments
should be fixed.
[typos]: https://github.com/crate-ci/typos
fix: Fix incorrect type mismatch with `cfg_if!` and other macros in expression position
Fixes https://github.com/rust-lang/rust-analyzer/issues/12940
This is a bit of a hack, ideally `MacroStmts` would not exist at all after HIR lowering, but that requires changing how the lowering code works.
fix: make `concat!` work with char
Fixes#12921
- I avoided making `unquote_str()` take char literals as well because it's depended on by another function `parse_string()` that's only supposed to take strings.
- Even with this patch, we don't output `\0` as `\u{0}` which #12921 pointed out ~~, but we're not actually responsible for serializing it but rowan is~~. They are functionally equivalent and I don't think it'd cause any confusion, but we *could* try escaping them before serialization (for reference, `rustc -Zunpretty=expanded`, which `cargo expand` uses under the hood, [makes use of `str::escape_default()`](3830ecaa8d/compiler/rustc_ast/src/util/literal.rs (L161)).
internal: Use ItemTree for variant, field and module attribute collection in attrs_query
Less parsing = very good, should speed up lang item collection as that basically probes attributes of all enum variants which currently triggers parsing
Not fond of how this is searching for the correct index, ideally we'd map between HIR and item tree Id here but I am not sure how, storing the item tree ids in the HIR version doesn't work due to the usage of `Trace`...
internal: Record all macro definitions in ItemScope
Fixes https://github.com/rust-lang/rust-analyzer/issues/12100
Doesn't resolve the shadowing issues though, fixing those is gonna be really tricky I believe unless we can come up with a nice scheme to "order" item tree items (using syntax ranges and file ids would be a pain and also a bad idea since that'll require us to potentially reparse files in collection).
fix: Simplify macro statement expansion handling
I only meant to fix https://github.com/rust-lang/rust-analyzer/issues/12644 but that somehow turned into a rewrite of the statement handling ... at least this fixes a few more issues in the IDE layer now
feat: implement destructuring assignment
This is an attempt to implement destructuring assignments, or more specifically, type inference for [assignee expressions](https://doc.rust-lang.org/reference/expressions.html#place-expressions-and-value-expressions).
I'm not sure if this is the right approach, so I don't even expect this to be merged (hence the branch name 😉) but rather want to propose one direction we could choose. I don't mind getting merged if this is good enough though!
Some notes on the implementation choices:
- Assignee expressions are **not** desugared on HIR level unlike rustc, but are inferred directly along with other expressions. This matches the processing of other syntaxes that are desugared in rustc but not in r-a. I find this reasonable because r-a only needs to infer types and it's easier to relate AST nodes and HIR nodes, so I followed it.
- Assignee expressions obviously resemble patterns, so type inference for each kind of pattern and its corresponding assignee expressions share a significant amount of logic. I tried to reuse the type inference functions for patterns by introducing `PatLike` trait which generalizes assignee expressions and patterns.
- This is not the most elegant solution I suspect (and I really don't like the name of the trait!), but it's cleaner and the change is smaller than other ways I experimented, like making the functions generic without such trait, or making them take `Either<ExprId, PatId>` in place of `PatId`.
in case this is merged:
Closes#11532Closes#11839Closes#12322
Distinguish between
- there is no build data (for some reason?)
- there is build data, but the cargo package didn't build a proc macro dylib
- there is a proc macro dylib, but it didn't contain the proc macro we expected
- the name did not resolve to any macro (this is now an
unresolved_macro_call even for attributes)
I changed the handling of disabled attribute macro expansion to
immediately ignore the macro and report an unresolved_proc_macro,
because otherwise they would now result in loud unresolved_macro_call
errors. I hope this doesn't break anything.
Also try to improve error ranges for unresolved_macro_call / macro_error
by reusing the code for unresolved_proc_macro. It's not perfect but
probably better than before.
fix methods in pub trait generated by macro cannot be completed
Fix#12483
Check if the container is trait and inherit the visibility to associate items during collection.
