Today, rust-analyzer (and rustc, and bat, and IntelliJ) fail badly on
some kinds of maliciously constructed code, like a deep sequence of
nested parenthesis.
"Who writes 100k nested parenthesis" you'd ask?
Well, in a language with macros, a run-away macro expansion might do
that (see the added tests)! Such expansion can be broad, rather than
deep, so it bypasses recursion check at the macro-expansion layer, but
triggers deep recursion in parser.
In the ideal world, the parser would just handle deeply nested structs
gracefully. We'll get there some day, but at the moment, let's try to be
simple, and just avoid expanding macros with unbalanced parenthesis in
the first place.
closes#9358
We generally avoid "syntax only" helper wrappers, which don't do much:
they make code easier to write, but harder to read. They also make
investigations harder, as "find_usages" needs to be invoked both for the
wrapped and unwrapped APIs
9260: tree-wide: make rustdoc links spiky so they are clickable r=matklad a=lf-
Rustdoc was complaining about these while I was running with --document-private-items and I figure they should be fixed.
Co-authored-by: Jade <software@lfcode.ca>
The idea here is to eventually get rid of `dyn Diagnostic` and
`DiagnosticSink` infrastructure altogether, and just have a `enum
hir::Diagnostic` instead.
The problem with `dyn Diagnostic` is that it is defined in the lowest
level of the stack (hir_expand), but is used by the highest level (ide).
As a first step, we free hir_expand and hir_def from `dyn Diagnostic`
and kick the can up to `hir_ty`, as an intermediate state. The plan is
then to move DiagnosticSink similarly to the hir crate, and, as final
third step, remove its usage from the ide.
One currently unsolved problem is testing. You can notice that the test
which checks precise diagnostic ranges, unresolved_import_in_use_tree,
was moved to the ide layer. Logically, only IDE should have the infra to
render a specific range.
At the same time, the range is determined with the data produced in
hir_def and hir crates, so this layering is rather unfortunate. Working
on hir_def shouldn't require compiling `ide` for testing.
8776: fix: fix unnecessary recomputations due to macros r=jonas-schievink a=jonas-schievink
This computes a macro's fragment kind eagerly (when the calling file is still available in parsed form) and stores it in the `MacroCallLoc`. This means that during expansion we no longer have to reparse the file containing the macro call, avoiding the unnecessary salsa dependencies (https://github.com/rust-analyzer/rust-analyzer/pull/8746#issuecomment-834776349).
Marking as draft until I manage to find a test for this problem, since for some reason `typing_inside_a_function_should_not_invalidate_expansions` does not catch this (which might indicate that I misunderstand the problem).
I've manually confirmed that this fixes the issue described in https://github.com/rust-analyzer/rust-analyzer/pull/8746#issuecomment-834776349:
```
7ms - parse_query @ FileId(179)
12ms - SourceBinder::to_module_def
12ms - crate_def_map:wait
5ms - item_tree_query (1 calls)
7ms - ???
```
Co-authored-by: Jonas Schievink <jonasschievink@gmail.com>
8280: Borrow text of immutable syntax node r=iDawer a=iDawer
In https://github.com/rust-analyzer/rowan/pull/101 `rowan::SyntaxNode::green` returns `Cow<'_, GreenNodeData>`. It returns borrow of green node of immutable syntax tree node.
Using this we can return borrowed text from `ast::Name::text`.
~~However now it allocates in case of mutable syntax trees.~~ (see next comment)
The idea comes from https://github.com/rust-analyzer/rowan/pull/100#issuecomment-809330325
Co-authored-by: Dawer <7803845+iDawer@users.noreply.github.com>
8201: Fix recursive macro statements expansion r=edwin0cheng a=edwin0cheng
This PR attempts to properly handle macro statement expansion by implementing the following:
1. Merge macro expanded statements to parent scope statements.
2. Add a new hir `Expr::MacroStmts` for handle tail expression infer.
PS : The scope of macro expanded statements are so strange that it took more time than I thought to understand and implement it :(
Fixes #8171
Co-authored-by: Edwin Cheng <edwin0cheng@gmail.com>
8134: Correct the paths of submodules from the include! macro r=edwin0cheng a=sticnarf
This PR should fix#7846. It mostly follows the instructions from @edwin0cheng in that issue.
Co-authored-by: Yilin Chen <sticnarf@gmail.com>
8063: couple clippy::complexity fixes r=matklad a=matthiaskrgr
avoid redundant `.into()` calls to convert T into identical T (`let x: String = String::from("hello").into();`)
use `if let Some(x)` instead of `.is_some()` + `.unwrap()`
don't clone Copy types
remove redundant wrapped ?s: `Some(Some(3)?)` can just be `Some(3)`
use `.map(|x| y)` instead of `and_then(|x| Some(y)` on `Option`s
Co-authored-by: Matthias Krüger <matthias.krueger@famsik.de>
use vec![] instead of Vec::new() + push()
avoid redundant clones
use chars instead of &str for single char patterns in ends_with() and starts_with()
allocate some Vecs with capacity to avoid unneccessary resizing
7994: Speed up mbe matching in heavy recursive cases r=edwin0cheng a=edwin0cheng
In some cases (e.g. #4186), mbe matching is very slow due to a lot of copy and allocation for bindings, this PR try to solve this problem by introduce a semi "link-list" approach for bindings building.
I used this [test case](https://github.com/weiznich/minimal_example_for_rust_81262) (for `features(32-column-tables)`) to run following command to benchmark:
```
time rust-analyzer analysis-stats --load-output-dirs ./
```
Before this PR : 2 mins
After this PR: 3 seconds.
However, for 64-column-tables cases, we still need 4 mins to complete.
I will try to investigate in the following weeks.
bors r+
Co-authored-by: Edwin Cheng <edwin0cheng@gmail.com>
7359: ItemTree: store a mapping from blocks to inner items r=jonas-schievink a=jonas-schievink
To do name resolution within block expressions, we need to know which inner items are located inside each block expression. This adds such a mapping to `ItemTree`, replacing the previous one, which was seemingly unused other than to access all the inner items.
This also assigns `AstId`s to block expressions, which is needed to store the mapping in salsa.
Co-authored-by: Jonas Schievink <jonasschievink@gmail.com>
7145: Proper handling $crate Take 2 [DO NOT MERGE] r=edwin0cheng a=edwin0cheng
Similar to previous PR (#7133) , but improved the following things :
1. Instead of storing the whole `ExpansionInfo`, we store a similar but stripped version `HygieneInfo`.
2. Instread of storing the `SyntaxNode` (because every token we are interested are IDENT), we store the `TextRange` only.
3. Because of 2, we now can put it in Salsa.
4. And most important improvement: Instead of computing the whole frames every single time, we compute it recursively through salsa: (Such that in the best scenario, we only need to compute the first layer of frame)
```rust
let def_site = db.hygiene_frame(info.def.file_id);
let call_site = db.hygiene_frame(info.arg.file_id);
HygieneFrame { expansion: Some(info), local_inner, krate, call_site, def_site }
```
The overall speed compared to previous PR is much faster (65s vs 45s) :
```
[WITH old PR]
Database loaded 644.86ms, 284mi
Crates in this dir: 36
Total modules found: 576
Total declarations: 11153
Total functions: 8715
Item Collection: 15.78s, 91562mi
Total expressions: 240721
Expressions of unknown type: 2635 (1%)
Expressions of partially unknown type: 2064 (0%)
Type mismatches: 865
Inference: 49.84s, 250747mi
Total: 65.62s, 342310mi
rust-analyzer -q analysis-stats . 66.72s user 0.57s system 99% cpu 1:07.40 total
[WITH this PR]
Database loaded 665.83ms, 284mi
Crates in this dir: 36
Total modules found: 577
Total declarations: 11188
Total functions: 8743
Item Collection: 15.28s, 84919mi
Total expressions: 241229
Expressions of unknown type: 2637 (1%)
Expressions of partially unknown type: 2064 (0%)
Type mismatches: 868
Inference: 30.15s, 135293mi
Total: 45.43s, 220213mi
rust-analyzer -q analysis-stats . 46.26s user 0.74s system 99% cpu 47.294 total
```
*HOWEVER*, it is still a perf regression (35s vs 45s):
```
[WITHOUT this PR]
Database loaded 657.42ms, 284mi
Crates in this dir: 36
Total modules found: 577
Total declarations: 11177
Total functions: 8735
Item Collection: 12.87s, 72407mi
Total expressions: 239380
Expressions of unknown type: 2643 (1%)
Expressions of partially unknown type: 2064 (0%)
Type mismatches: 868
Inference: 22.88s, 97889mi
Total: 35.74s, 170297mi
rust-analyzer -q analysis-stats . 36.71s user 0.63s system 99% cpu 37.498 total
```
Co-authored-by: Edwin Cheng <edwin0cheng@gmail.com>
7133: Proper handling $crate and local_inner_macros r=jonas-schievink a=edwin0cheng
This PR introduces `HygineFrames` to store the macro definition/call site hierarchy in hyginee and when resolving `local_inner_macros` and `$crate`, we use the token to look up the corresponding frame and return the correct value.
See also: https://rustc-dev-guide.rust-lang.org/macro-expansion.html#hygiene-and-hierarchies
fixe #6890 and #6788
r? @jonas-schievink
Co-authored-by: Edwin Cheng <edwin0cheng@gmail.com>
6645: Publish diagnostics for macro expansion errors r=matklad a=jonas-schievink
This adds 2 new diagnostics, emitted during name resolution:
* `unresolved-proc-macro`, a weak warning that is emitted when a proc macro is supposed to be expanded, but was not provided by the build system. This usually means that proc macro support is turned off, but may also indicate setup issues when using rust-project.json. Being a weak warning, this should help set expectations when users see it, while not being too obstructive. We do not yet emit this for attribute macros though, just custom derives and `!` macros.
* `macro-error`, which is emitted when any macro (procedural or `macro_rules!`) fails to expand due to some error. This is an error-level diagnostic, but currently still marked as experimental, because there might be spurious errors and this hasn't been tested too well.
This does not yet emit diagnostics when expansion in item bodies fails, just for module-level macros.
Known bug: The "proc macro not found" diagnostic points at the whole item for custom derives, it should just point at the macro's name in the `#[derive]` list, but I haven't found an easy way to do that.
Screenshots:
Co-authored-by: Jonas Schievink <jonasschievink@gmail.com>
6033: Make name resolution resolve proc macros instead of relying purely on the build system r=matklad a=jonas-schievink
This makes name resolution look at proc-macro declaration attributes like `#[proc_macro_derive]` and defines the right proc macro in the macro namespace, fixing unresolved custom derives like `thiserror::Error` (which can cause false positives, now that we emit diagnostics for unresolved imports).
This works even when proc-macro support is turned off, in which case we fall back to a dummy expander that always returns an error. IMO this is the right way to handle at least the name resolution part of proc. macros, while the *expansion* itself should rely on the build system to build and provide the macro DLL. It does mean that they may go out of sync, but we can provide diagnostics if that happens (something like "could not find macro X in crate Y – ensure that all files of crate Y are saved").
I think it is valuable to be able to reason about proc macros even when we can't expand them, since proc macro expansion can break between Rust releases or users might not want to turn it on for performance reasons. It allows us to provide better diagnostics on any proc macro invocation we're not expanding (like a weak warning that informs the user that proc macro support is turned off, or that it has been disabled because the server crashed).
Fixes https://github.com/rust-analyzer/rust-analyzer/issues/5763
Co-authored-by: Jonas Schievink <jonas.schievink@ferrous-systems.com>
