099b63e7c0
This story begins in #8384, where we added a smart test for our syntax highting, which run the algorithm on synthetic files of varying length in order to guesstimate if the complexity is O(N^2) or O(N)-ish. The test turned out to be pretty effective, and flagged #9031 as a change that makes syntax highlighting accidentally quadratic. There was much rejoicing, for the time being. Then, lnicola asked an ominous question[1]: "Are we sure that the time is linear right now?" Of course it turned out that our sophisticated non-linearity detector *was* broken, and that our syntax highlighting *was* quadratic. Investigating that, many brave hearts dug deeper and deeper into the guts of rust-analyzer, only to get lost in a maze of traits delegating to traits delegating to macros. Eventually, matklad managed to peel off all layers of abstraction one by one, until almost nothing was left. In fact, the issue was discovered in the very foundation of the rust-analyzer -- in the syntax trees. Worse, it was not a new problem, but rather a well-know, well-understood and event (almost) well-fixed (!) performance bug. The problem lies within `SyntaxNodePtr` type -- a light-weight "address" of a node in a syntax tree [3]. Such pointers are used by rust-analyzer all other the place to record relationships between IR nodes and the original syntax. Internally, the pointer to a syntax node is represented by node's range. To "dereference" the pointer, you traverse the syntax tree from the root, looking for the node with the right range. The inner loop of this search is finding a node's child whose range contains the specified range. This inner loop was implemented by naive linear search over all the children. For wide trees, dereferencing a single `SyntaxNodePtr` was linear. The problem with wide trees though is that they contain a lot of nodes! And dereferencing pointers to all the nodes is quadratic in the size of the file! The solution to this problem is to speed up the children search -- rather than doing a linear lookup, we can use binary search to locate the child with the desired interval. Doing this optimization was one of the motivations (or rather, side effects) of #6857. That's why `rowan` grew the useful `child_or_token_at_range` method which does exactly this binary search. But looks like we've never actually switch to this method? Oups. Lesson learned: do not leave broken windows in the fundamental infra. Otherwise, you'll have to repeatedly re-investigate the issue, by digging from the top of the Everest down to the foundation! [1]: https://rust-lang.zulipchat.com/#narrow/stream/185405-t-compiler.2Frust-analyzer/topic/.60syntax_highlighting_not_quadratic.60.20failure/near/240811501 [2]: https://rust-lang.zulipchat.com/#narrow/stream/185405-t-compiler.2Frust-analyzer/topic/Syntax.20highlighting.20is.20quadratic [3]: https://rust-lang.zulipchat.com/#narrow/stream/185405-t-compiler.2Frust-analyzer/topic/Syntax.20highlighting.20is.20quadratic/near/243412392
rust-analyzer is a modular compiler frontend for the Rust language. It is a part of a larger rls-2.0 effort to create excellent IDE support for Rust.
Work on rust-analyzer is sponsored by
Quick Start
https://rust-analyzer.github.io/manual.html#installation
Documentation
If you want to contribute to rust-analyzer or are just curious about how things work under the hood, check the ./docs/dev folder.
If you want to use rust-analyzer's language server with your editor of choice, check the manual folder. It also contains some tips & tricks to help you be more productive when using rust-analyzer.
Security and Privacy
See the corresponding sections of the manual.
Communication
For usage and troubleshooting requests, please use "IDEs and Editors" category of the Rust forum:
https://users.rust-lang.org/c/ide/14
For questions about development and implementation, join rust-analyzer working group on Zulip:
https://rust-lang.zulipchat.com/#narrow/stream/185405-t-compiler.2Frust-analyzer
Quick Links
- Website: https://rust-analyzer.github.io/
- Metrics: https://rust-analyzer.github.io/metrics/
- API docs: https://rust-analyzer.github.io/rust-analyzer/ide/
- Changelog: https://rust-analyzer.github.io/thisweek
License
Rust analyzer is primarily distributed under the terms of both the MIT license and the Apache License (Version 2.0).
See LICENSE-APACHE and LICENSE-MIT for details.