Commit Graph

339 Commits

Author SHA1 Message Date
Lukas Wirth
a9c4c6da4c Fix mbe::Shift::new not accounting for non-ident token ids 2021-11-22 18:00:32 +01:00
Lukas Wirth
64cb09ddea Add to macro testing infra to emit token map ids 2021-11-22 16:51:09 +01:00
Aleksey Kladov
5a83d1be66 internal: replace L_DOLLAR/R_DOLLAR with parenthesis hack
The general problem we are dealing with here is this:

```
macro_rules! thrice {
    ($e:expr) => { $e * 3}
}

fn main() {
    let x = thrice!(1 + 2);
}
```

we really want this to print 9 rather than 7.

The way rustc solves this is rather ad-hoc. In rustc, token trees are
allowed to include whole AST fragments, so 1+2 is passed through macro
expansion as a single unit. This is a significant violation of token
tree model.

In rust-analyzer, we intended to handle this in a more elegant way,
using token trees with "invisible" delimiters. The idea was is that we
introduce a new kind of parenthesis, "left $"/"right $", and let the
parser intelligently handle this.

The idea was inspired by the relevant comment in the proc_macro crate:

https://doc.rust-lang.org/stable/proc_macro/enum.Delimiter.html#variant.None

> An implicit delimiter, that may, for example, appear around tokens
> coming from a “macro variable” $var. It is important to preserve
> operator priorities in cases like $var * 3 where $var is 1 + 2.
> Implicit delimiters might not survive roundtrip of a token stream
> through a string.

Now that we are older and wiser, we conclude that the idea doesn't work.

_First_, the comment in the proc-macro crate is wishful thinking. Rustc
currently completely ignores none delimiters. It solves the (1 + 2) * 3
problem by having magical token trees which can't be duplicated:

* https://rust-lang.zulipchat.com/#narrow/stream/185405-t-compiler.2Frust-analyzer/topic/TIL.20that.20token.20streams.20are.20magic
* https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/Handling.20of.20Delimiter.3A.3ANone.20by.20the.20parser

_Second_, it's not like our implementation in rust-analyzer works. We
special-case expressions (as opposed to treating all kinds of $var
captures the same) and we don't know how parser error recovery should
work with these dollar-parenthesis.

So, in this PR we simplify the whole thing away by not pretending that
we are doing something proper and instead just explicitly special-casing
expressions by wrapping them into real `()`.

In the future, to maintain bug-parity with `rustc` what we are going to
do is probably adding an explicit `CAPTURED_EXPR` *token* which we can
explicitly account for in the parser.

If/when rustc starts handling delimiter=none properly, we'll port that
logic as well, in addition to special handling.
2021-10-23 20:44:31 +03:00
Laurențiu Nicola
8457ae34bd Set MSRV 2021-10-23 15:07:11 +03:00
Lukas Wirth
1294bfce86 Migrate to edition 2021 2021-10-21 20:10:40 +02:00
Aleksey Kladov
d4d67406d7 internal: clean up code duplication 2021-10-10 21:08:10 +03:00
Aleksey Kladov
634f047d90 internal: add integrated test for token censoring 2021-10-10 16:52:21 +03:00
Aleksey Kladov
bfc5d8529a drop obsolete tests 2021-10-10 15:11:33 +03:00
Aleksey Kladov
0f849a7a35 move test 2021-10-10 15:06:41 +03:00
Aleksey Kladov
dce41e5a03 move tests 2021-10-10 14:58:25 +03:00
Aleksey Kladov
42fd71e6c8 move tests 2021-10-10 14:40:13 +03:00
Aleksey Kladov
be73cc8f83 move test 2021-10-10 14:28:04 +03:00
Aleksey Kladov
af76db3c36 move tests 2021-10-10 14:26:47 +03:00
Aleksey Kladov
6253213a6e move test 2021-10-10 14:23:52 +03:00
Aleksey Kladov
e55797f59d move tests 2021-10-10 14:21:47 +03:00
Aleksey Kladov
8997d742dc move tests 2021-10-10 14:08:49 +03:00
Aleksey Kladov
5ad502dbdb move test 2021-10-10 13:54:44 +03:00
Aleksey Kladov
c986568cbb move test 2021-10-10 13:26:07 +03:00
Aleksey Kladov
5b44770102 move test 2021-10-10 13:24:48 +03:00
Aleksey Kladov
e255e9577f internal: move test 2021-10-10 13:21:42 +03:00
Aleksey Kladov
8670e83cec move test 2021-10-10 12:57:18 +03:00
Aleksey Kladov
7d92b9f6ff move test 2021-10-10 12:55:31 +03:00
Aleksey Kladov
748e6881fc move tests 2021-10-10 12:52:28 +03:00
Aleksey Kladov
6fd2f1d25b internal: move tests 2021-10-10 12:45:17 +03:00
Aleksey Kladov
5a854a7253 internal: move tests 2021-10-10 12:39:58 +03:00
Aleksey Kladov
c88cda04db move some tests 2021-10-10 11:44:46 +03:00
Aleksey Kladov
a3470a8114 move tests 2021-10-10 11:39:08 +03:00
Aleksey Kladov
7e53a3ce23 move test 2021-10-10 11:29:26 +03:00
Aleksey Kladov
408475a593 move test 2021-10-10 11:26:18 +03:00
Aleksey Kladov
9c819eaa9a move tests 2021-10-10 11:15:42 +03:00
Aleksey Kladov
1c15f47e00 internal: move tests 2021-10-10 11:11:50 +03:00
Aleksey Kladov
c6d5c1c946 dead code 2021-10-10 11:09:16 +03:00
Aleksey Kladov
e9902b92ab internal: move some mbe tests 2021-10-10 11:08:02 +03:00
Aleksey Kladov
f17f5d68f9 move tests 2021-10-10 11:08:02 +03:00
Aleksey Kladov
3a47dba761 fix tests 2021-10-10 11:08:02 +03:00
Aleksey Kladov
de136a5340 move test 2021-10-09 19:11:04 +03:00
Aleksey Kladov
e838da18a9 internal: move tests 2021-10-09 18:54:15 +03:00
Aleksey Kladov
0dc87badd7 internal: move test 2021-10-09 18:51:26 +03:00
Aleksey Kladov
0a32c20142 internal: move test 2021-10-09 18:49:14 +03:00
Aleksey Kladov
993ff1c239 internal: drop duplicated test 2021-10-09 18:47:04 +03:00
Aleksey Kladov
419c234333 internal: move test 2021-10-09 18:46:16 +03:00
Aleksey Kladov
b1cfa51ef5 internal: move tests 2021-10-09 18:43:15 +03:00
Aleksey Kladov
3e8ef943c6 internal: move some tests 2021-10-09 18:18:56 +03:00
Aleksey Kladov
a060b9a4b2 internal: move some macro tests 2021-10-09 18:15:05 +03:00
Aleksey Kladov
c41b7bbe69 internal: allow macro tests to inspect parse tree 2021-10-09 17:58:17 +03:00
Aleksey Kladov
aac23f7832 move tests 2021-10-09 17:43:07 +03:00
Aleksey Kladov
036c0ff8c7 move some tests 2021-10-09 17:27:38 +03:00
Aleksey Kladov
afacdd612d internal: update expect 2021-10-09 17:17:16 +03:00
Aleksey Kladov
959da8caa1 internal: move test 2021-10-09 16:31:26 +03:00
Aleksey Kladov
75b0ce17cf move test 2021-10-09 16:27:19 +03:00
Aleksey Kladov
0dd1b35479 move test 2021-10-09 16:25:37 +03:00
Aleksey Kladov
574df660e4 move test 2021-10-09 16:22:42 +03:00
Aleksey Kladov
b21244e080 internal: move test 2021-10-09 16:19:19 +03:00
Aleksey Kladov
ef1251f696 feat: report errors in macro definition
Reporting macro *definition* error at the macro *call site* is a rather
questionable approach, but at least we don't erase the errors
altogether!
2021-10-09 15:23:55 +03:00
Aleksey Kladov
8e9003447c future proof structure 2021-10-09 14:48:38 +03:00
Aleksey Kladov
5ecda802f1 move test 2021-10-09 14:45:52 +03:00
Aleksey Kladov
f4ee0d736c move tests 2021-10-09 14:39:24 +03:00
Aleksey Kladov
1abe3f8275 internal: move tests 2021-10-09 14:22:49 +03:00
Aleksey Kladov
49f5fecf06 internal: move test 2021-10-09 14:18:53 +03:00
Aleksey Kladov
78ca43ef3d internal: move test 2021-10-09 13:51:02 +03:00
Aleksey Kladov
093f99b809 internal: start new macro test suite
I don't like our macro tests -- they are brittle and don't inspire
confidence. I think the reason for that is that we try to unit-test
them, but that is at odds with reality, where macro expansion
fundamentally depends on name resolution.
2021-10-09 13:42:32 +03:00
Aleksey Kladov
4e352275d1 minor: simplify 2021-10-02 20:38:39 +03:00
Aleksey Kladov
613609cc5e minor: cleanup 2021-10-02 20:38:39 +03:00
Aleksey Kladov
77bf761203 internal: move code to where it's used and reduce visibility 2021-10-02 20:38:39 +03:00
Lukas Wirth
b6ed91a6de Rename *Owner traits to Has* 2021-09-27 12:54:24 +02:00
Aleksey Kladov
2bf81922f7 internal: more reasonable grammar for blocks
Consider these expples

        { 92 }
  async { 92 }
    'a: { 92 }
   #[a] { 92 }

Previously the tree for them were

  BLOCK_EXPR
    { ... }

  EFFECT_EXPR
    async
    BLOCK_EXPR
      { ... }

  EFFECT_EXPR
    'a:
    BLOCK_EXPR
      { ... }

  BLOCK_EXPR
    #[a]
    { ... }

As you see, it gets progressively worse :) The last two items are
especially odd. The last one even violates the balanced curleys
invariant we have () The new approach is to say that the stuff in
`{}` is stmt_list, and the block is stmt_list + optional modifiers

  BLOCK_EXPR
    STMT_LIST
      { ... }

  BLOCK_EXPR
    async
    STMT_LIST
      { ... }

  BLOCK_EXPR
    'a:
    STMT_LIST
      { ... }

  BLOCK_EXPR
    #[a]
    STMT_LIST
      { ... }
2021-09-26 19:16:09 +03:00
Lukas Wirth
d99adc5738 Make hover work for intra doc links in macro invocations 2021-09-23 17:32:39 +02:00
bors[bot]
f1d7f98ed0
Merge
10293: fix: Don't bail on parse errors in macro input for builtin expansion r=Veykril a=Veykril

Fixes https://github.com/rust-analyzer/rust-analyzer/issues/8158

Co-authored-by: Lukas Wirth <lukastw97@gmail.com>
2021-09-19 22:33:42 +00:00
Lukas Wirth
e7e87fc69d Don't bail on parse errors in macro input for builtin expansion 2021-09-20 00:33:13 +02:00
Lukas Wirth
a6dde501df Only strip derive attributes when preparing macro input 2021-09-19 23:38:38 +02:00
Giles Cope
15312aab58
removing seemingly unused dev deps. 2021-09-11 16:26:36 +01:00
Aleksey Kladov
104cd0ce88 internal: make name consistent with usage 2021-09-06 18:34:03 +03:00
Aleksey Kladov
dbb702cfc1 internal: remove accidental code re-use
FragmentKind played two roles:

* entry point to the parser
* syntactic category of a macro call

These are different use-cases, and warrant different types. For example,
macro can't expand to visibility, but we have such fragment today.

This PR introduces `ExpandsTo` enum to separate this two use-cases.

I suspect we might further split `FragmentKind` into `$x:specifier` enum
specific to MBE, and a general parser entry point, but that's for
another PR!
2021-09-05 22:36:36 +03:00
Lukas Wirth
36a5ce9790 minor: fix some clippy lints 2021-09-03 16:00:50 +02:00
Aleksey Kladov
81602f8a5d internal: reduce coupling
tt is a data structure, data structures cant' go wrong, they shouldn't
have the knowledge that the world outside of them has all kinds of
errors.
2021-08-31 19:14:33 +03:00
Dezhi Wu
ba0947dded switch log crate to tracing 2021-08-30 15:11:42 +08:00
Lukas Wirth
c5059e0623 Return all ranges corresponding to a token id in TokenMap 2021-08-29 00:49:57 +02:00
bors[bot]
fae440c32a
Merge
10025: Don't mutate syntax trees when preparing proc-macro input r=Veykril a=Veykril

Fixes 

Co-authored-by: Lukas Wirth <lukastw97@gmail.com>
2021-08-28 14:14:54 +00:00
bors[bot]
97409e5fc8
Merge
9970: feat: Implement attribute input token mapping, fix attribute item token mapping r=Veykril a=Veykril

![image](https://user-images.githubusercontent.com/3757771/130328577-4c1ad72c-51b1-47c3-8d3d-3242ec44a355.png)

The token mapping for items with attributes got overwritten partially by the attributes non-item input, since attributes have two different inputs, the item and the direct input both.
This PR gives attributes a second TokenMap for its direct input. We now shift all normal input IDs by the item input maximum(we maybe wanna swap this see below) similar to what we do for macro-rules/def. For mapping down we then have to figure out whether we are inside the direct attribute input or its item input to pick the appropriate mapping which can be done with some token range comparisons.

Fixes https://github.com/rust-analyzer/rust-analyzer/issues/9867

Co-authored-by: Lukas Wirth <lukastw97@gmail.com>
2021-08-27 19:30:36 +00:00
Lukas Wirth
1195cb50c2 Add simple test for syntax_node_to_token_tree_censored 2021-08-25 19:57:18 +02:00
Lukas Wirth
d6134b6802 Don't mutate syntax trees when preparing proc-macro input 2021-08-25 18:57:24 +02:00
Frank Steffahn
5f5d45468d Fix two more “a”/“an” typos (this time the other way) 2021-08-22 17:36:58 +02:00
Lukas Wirth
5fb8c0ddfd Remove MappedSubtree 2021-08-21 18:19:18 +02:00
Lukas Wirth
177c70128c Map attribute input tokens correctly 2021-08-21 18:13:41 +02:00
Aleksey Kladov
78c7940f5c internal: remove dead code 2021-08-14 20:29:46 +03:00
Aleksey Kladov
9aa6be71a5 internal: remove useless helpers
We generally avoid "syntax only" helper wrappers, which don't do much:
they make code easier to write, but harder to read. They also make
investigations harder, as "find_usages" needs to be invoked both for the
wrapped and unwrapped APIs
2021-08-09 15:58:21 +03:00
Aleksey Kladov
86720f2953 minor: drop dummy authors field 2021-07-05 14:19:41 +03:00
Jonas Schievink
c6669776e1 Rewrite convert_tokens to use an explicit stack 2021-06-23 00:21:11 +02:00
Jonas Schievink
6504c3c32a Move subtree collection out of TokenConvertor 2021-06-23 00:19:54 +02:00
Clemens Wasser
47747cd412 Apply some clippy suggestions 2021-06-21 16:40:21 +02:00
Lukas Wirth
95c8c65139 Nest all the or-patterns! 2021-06-17 17:37:14 +02:00
bors[bot]
5a8ddb4b2d
Merge
9260: tree-wide: make rustdoc links spiky so they are clickable r=matklad a=lf-

Rustdoc was complaining about these while I was running with --document-private-items and I figure they should be fixed.

Co-authored-by: Jade <software@lfcode.ca>
2021-06-14 07:16:48 +00:00
Jade
20b325c7d5 tree-wide: make rustdoc links spiky so they are clickable 2021-06-13 21:58:05 -07:00
Maan2003
aabd41cafc
clippy::redundant_field_names 2021-06-13 09:40:22 +05:30
Maan2003
c9b4ac5be4
clippy::redudant_borrow 2021-06-13 09:24:16 +05:30
Aleksey Kladov
0463d76a1f internal: cross-crate cov-marks 2021-06-12 23:40:52 +03:00
Jonas Schievink
1d6eef1350 Update ungrammar 2021-06-11 18:34:30 +02:00
Clemens Wasser
629e8d1ed0 Apply more clippy suggestions and update generated 2021-06-03 12:46:56 +02:00
Jonas Schievink
4c4bcaebbe Treat pat_param like pat fragments 2021-05-29 18:32:32 +02:00
Jonas Schievink
c8f40b1503 Fixup 2021-05-24 21:47:01 +02:00
Jonas Schievink
489ae7a800 Make TokenTextRange private 2021-05-24 20:29:48 +02:00
Jonas Schievink
27bf62b70e Move TokenMap to its own file 2021-05-24 18:43:42 +02:00
Aleksey Kladov
dc1577d58d Add even more docs 2021-05-22 17:20:22 +03:00
bors[bot]
2ace128dd4
Merge
8560: Escape characters in doc comments in macros correctly r=jonas-schievink a=ChayimFriedman2

Previously they were escaped twice, both by `.escape_default()` and the debug view of strings (`{:?}`). This leads to things like newlines or tabs in documentation comments being `\\n`, but we unescape literals only once, ending up with `\n`.

This was hard to spot because CMark unescaped them (at least for `'` and `"`), but it did not do so in code blocks.

This also was the root cause of . This issue was solved by using `.escape_debug()` instead of `.escape_default()`, but the real issue remained.
We can bring the `.escape_default()` back by now, however I didn't do it because it is probably slower than `.escape_debug()` (more work to do), and also in order to change the code the least.

Example (the keyword and primitive docs are `include!()`d at https://doc.rust-lang.org/src/std/lib.rs.html#570-578, and thus originate from macro):

Before:
![image](https://user-images.githubusercontent.com/24700207/115130096-40544300-9ff5-11eb-847b-969e7034e8a4.png)

After:
![image](https://user-images.githubusercontent.com/24700207/115130143-9cb76280-9ff5-11eb-9281-323746089440.png)


Co-authored-by: Chayim Refael Friedman <chayimfr@gmail.com>
2021-04-18 02:14:27 +00:00
Chayim Refael Friedman
f92be7eaab Escape characters in doc comments in macros correctly
Previously they were escaped twice, both by `.escape_default()` and the debug view of strings (`{:?}`). This leads to things like newlines or tabs in documentation comments being `\\n`, but we unescape literals only once, ending up with `\n`.

This was hard to spot because CMark unescaped them (at least for `'` and `"`), but it did not do so in code blocks.

This also was the root cause of . This issue was solved by using `.escape_debug()` instead of `.escape_default()`, but the real issue remained.
We can bring the `.escape_default()` back by now, however I didn't do it because it is probably slower than `.escape_debug()` (more work to do), and also in order to change the code the least.
2021-04-18 03:16:38 +03:00
Edwin Cheng
c4173bb468 Handle extended key value attr in mbe 2021-04-17 14:31:52 +08:00
Jonas Schievink
3abcdc03ba Make ast_to_token_tree infallible
It could never return `None`, so reflect that in the return type
2021-04-04 01:46:45 +02:00
Edwin Cheng
20d55ce44d Allow include! an empty content file 2021-04-03 12:50:55 +08:00
Jonas Schievink
eaffdae300 Allow , to delimit macro 2.0 rules 2021-04-03 03:08:31 +02:00
Matthias Krüger
202b51bc7b a lot of clippy::style fixes 2021-03-21 16:15:41 +01:00
Matthias Krüger
ae7e55c1dd clippy::complexity simplifications related to Iterators 2021-03-21 13:13:34 +01:00
Kevin Mehall
0a7f28620a Fix and test edge cases of _ as ident 2021-03-20 17:54:57 -06:00
Kevin Mehall
0a0e22235b Make bare underscore token an Ident rather than Punct in proc-macro 2021-03-20 12:28:44 -06:00
Edwin Cheng
49b876de09 Reorganize mbe tests 2021-03-18 17:22:52 +08:00
Matthias Krüger
048dad8c2e don't clone types that are copy (clippy::clone_on_copy) 2021-03-17 01:56:31 +01:00
Matthias Krüger
c5d654d513 use if let Some(x) instead of if x.is_some() and x.unwrap() (clippy::unnecessary-unwrap) 2021-03-17 01:39:58 +01:00
Matthias Krüger
966c23f529 avoid converting types into themselves via .into() (clippy::useless-conversion)
example: let x: String = String::from("hello world").into();
2021-03-17 01:27:56 +01:00
Edwin Cheng
8e07b23b84 Fix macro expansion for statements w/o semicolon 2021-03-16 13:44:50 +08:00
Laurențiu Nicola
88cee24c6c Enable thread-local coverage marks 2021-03-15 16:02:50 +02:00
Matthias Krüger
cad617bba0 some clippy::performance fixes
use vec![] instead of Vec::new() +  push()
avoid redundant clones
use chars instead of &str for single char patterns in ends_with() and starts_with()
allocate some Vecs with capacity to avoid unneccessary resizing
2021-03-15 10:19:59 +01:00
Edwin Cheng
7279749bbb Simpify mbe bindings builder 2021-03-14 11:54:19 +08:00
Edwin Cheng
b3b91046dd Make sure ill-form macro handle propely 2021-03-14 11:24:55 +08:00
bors[bot]
fe4a94fff3
Merge
7994: Speed up mbe matching in heavy recursive cases r=edwin0cheng a=edwin0cheng

In some cases (e.g.  ), mbe matching is very slow due to a lot of copy and allocation for bindings, this PR try to solve this problem by introduce a semi "link-list" approach for bindings building.

I used this [test case](https://github.com/weiznich/minimal_example_for_rust_81262) (for `features(32-column-tables)`) to run following command to benchmark:
```
time rust-analyzer analysis-stats  --load-output-dirs ./ 
```

Before this PR : 2 mins
After this PR: 3 seconds.

However, for 64-column-tables cases, we still need 4 mins to complete. 

I will try to investigate in the following weeks.

bors r+




Co-authored-by: Edwin Cheng <edwin0cheng@gmail.com>
2021-03-13 13:04:45 +00:00
Edwin Cheng
9117148f42 Add bindings builder for speed up matching 2021-03-13 20:52:36 +08:00
Aleksey Kladov
842d8ad9c8 Compilation speed 2021-03-09 22:30:58 +03:00
Laurențiu Nicola
fc9eed4836 Use upstream cov-mark 2021-03-08 22:19:44 +02:00
Edwin Cheng
20eda09712 Fix fail to parse :: for meta in mbe 2021-03-06 04:57:34 +08:00
bors[bot]
91bf5fa827
Merge
7513: NFA parser for mbe matcher r=matklad a=edwin0cheng

Almost straight porting from rustc one, but a little bit slow :(

```
rust-analyzer analysis-stats -q . 
```

From:
```log
Database loaded:     636.11ms, 277minstr
  crates: 36, mods: 594, decls: 11527, fns: 9017
Item Collection:     10.99s, 60ginstr
  exprs: 249618, ??ty: 2699 (1%), ?ty: 2101 (0%), !ty: 932
Inference:           28.94s, 123ginstr
Total:               39.93s, 184ginstr
```

To:
```log
Database loaded:     630.90ms, 277minstr
  crates: 36, mods: 594, decls: 11528, fns: 9018
Item Collection:     13.70s, 77ginstr
  exprs: 249482, ??ty: 2699 (1%), ?ty: 2101 (0%), !ty: 932
Inference:           30.27s, 133ginstr
Total:               43.97s, 211ginstr
```

Fixes 

Co-authored-by: Edwin Cheng <edwin0cheng@gmail.com>
2021-03-02 13:20:47 +00:00
Laurențiu Nicola
a445584029 Use an unversioned profile dependency in mbe 2021-03-01 18:11:12 +02:00
bors[bot]
1c7b2b8c07
Merge
7822: Paperover a bug in cargo-workspace for publish RA r=lnicola a=edwin0cheng

r? @lnicola 

See also https://github.com/pksunkara/cargo-workspaces/issues/39

Co-authored-by: Edwin Cheng <edwin0cheng@gmail.com>
2021-03-01 11:29:47 +00:00
Edwin Cheng
43f52a965c Paperover a bug in cargo-workspace for publish RA 2021-03-01 19:23:45 +08:00
Dániel Buga
2f7b03b01c Remove redundant semicolon 2021-03-01 09:05:14 +01:00
Edwin Cheng
cff2201c30 NFA parser for mbe matcher 2021-03-01 00:11:14 +08:00
Edwin Cheng
bf8bc5c882 Fix non-latin characters doc comment for mbe 2021-02-28 13:49:08 +08:00
Edwin Cheng
f5bf1a9650 Fix builtin macros split exprs on comma 2021-02-28 13:06:17 +08:00
Edwin Cheng
23dbf36c7d Add benchmark test for mbe 2021-02-25 05:47:13 +08:00
Pavan Kumar Sunkara
a89fc070c6
Make sure normal dependencies always have version 2021-02-03 12:51:07 +00:00
Edwin Cheng
438b34dcee Simpilfy mbe parsing 2021-01-30 16:12:30 +08:00
Edwin Cheng
706ac8256d Simplify mbe match error.
Handle parse error in rule parsing instead of match in mbe
2021-01-30 00:21:43 +08:00
Edwin Cheng
4c7d8cbfbf Rename mbe_expander for consistency 2021-01-29 20:23:38 +08:00
Edwin Cheng
ce4eeec1bf Support Macro v2 in mbe 2021-01-26 05:15:47 +08:00
Aleksey Kladov
3429b32ad1 ⬆️ rowan
It now stores text inline with tokens
2021-01-20 14:04:53 +03:00
Aleksey Kladov
46b4f89c92 . 2021-01-20 01:56:11 +03:00
Edwin Cheng
7b3e960d94 Reduce TokenMap size 2021-01-19 19:26:53 +08:00
Jonas Schievink
2d799cf5e0 shrink_to_fit TokenMap's backing storage 2021-01-18 18:40:06 +01:00
yugo-horie
f273995a25 Swap assert_eq_text\!(expected, actual) 2021-01-16 13:30:26 +09:00
bors[bot]
d76143da19
Merge
7211: Fixed expr meta var after path colons in mbe r=matklad a=edwin0cheng

Fixes 

Added `L_DOLLAR` in `ITEM_RECOVERY_SET` , but I don't know whether it is a good idea.

r? @matklad 

Co-authored-by: Edwin Cheng <edwin0cheng@gmail.com>
2021-01-14 10:52:12 +00:00
Edwin Cheng
8d62576a9b Fixed expr meta var after path colons in mbe 2021-01-10 20:52:46 +08:00
Vincent Esche
586f84c730 Fixed typos in public type names 2021-01-09 15:41:29 +01:00
Vincent Esche
5877db28b4 Fixed typos in private type names 2021-01-09 15:41:29 +01:00
Vincent Esche
4702c01553 Fixed typos in tests 2021-01-09 15:41:29 +01:00
Vincent Esche
23f05aa1ad Fixed typos in log::warn! message 2021-01-09 15:41:29 +01:00
Vincent Esche
21f8239ac8 Fixed typos in code comments 2021-01-09 15:41:29 +01:00
Edwin Cheng
bced02c5dc Add test 2021-01-08 14:00:23 +08:00
Edwin Cheng
74a24adc8e Fix bug when $crate in LHS in mbe 2021-01-08 14:00:16 +08:00
bors[bot]
1a29934c37
Merge
7145: Proper handling $crate Take 2 [DO NOT MERGE] r=edwin0cheng a=edwin0cheng

Similar to previous PR () , but improved the following things :

1. Instead of storing the whole `ExpansionInfo`, we store a similar but stripped version `HygieneInfo`.
2. Instread of storing the `SyntaxNode` (because every token we are interested are IDENT), we store the `TextRange` only.
3. Because of 2, we now can put it in Salsa.
4. And most important improvement: Instead of computing the whole frames every single time, we compute it recursively through salsa: (Such that in the best scenario, we only need to compute the first layer of frame)

```rust
        let def_site = db.hygiene_frame(info.def.file_id);
        let call_site = db.hygiene_frame(info.arg.file_id);

        HygieneFrame { expansion: Some(info), local_inner, krate, call_site, def_site }
```

The overall speed compared to previous PR is much faster (65s vs 45s) :
```
[WITH old PR]
Database loaded 644.86ms, 284mi
Crates in this dir: 36
Total modules found: 576
Total declarations: 11153
Total functions: 8715
Item Collection: 15.78s, 91562mi
Total expressions: 240721
Expressions of unknown type: 2635 (1%)
Expressions of partially unknown type: 2064 (0%)
Type mismatches: 865
Inference: 49.84s, 250747mi
Total: 65.62s, 342310mi
rust-analyzer -q analysis-stats .  66.72s user 0.57s system 99% cpu 1:07.40 total

[WITH this PR]
Database loaded 665.83ms, 284mi
Crates in this dir: 36
Total modules found: 577
Total declarations: 11188
Total functions: 8743
Item Collection: 15.28s, 84919mi
Total expressions: 241229
Expressions of unknown type: 2637 (1%)
Expressions of partially unknown type: 2064 (0%)
Type mismatches: 868
Inference: 30.15s, 135293mi
Total: 45.43s, 220213mi   
rust-analyzer -q analysis-stats .  46.26s user 0.74s system 99% cpu 47.294 total
```

*HOWEVER*,  it is still a perf regression (35s vs 45s):
```
[WITHOUT this PR]
Database loaded 657.42ms, 284mi
Crates in this dir: 36
Total modules found: 577
Total declarations: 11177
Total functions: 8735
Item Collection: 12.87s, 72407mi
Total expressions: 239380
Expressions of unknown type: 2643 (1%)
Expressions of partially unknown type: 2064 (0%)
Type mismatches: 868
Inference: 22.88s, 97889mi
Total: 35.74s, 170297mi
rust-analyzer -q analysis-stats .  36.71s user 0.63s system 99% cpu 37.498 total
```



Co-authored-by: Edwin Cheng <edwin0cheng@gmail.com>
2021-01-08 03:57:11 +00:00
Edwin Cheng
76f2b9d2ef Proper handling $crate Take 2 2021-01-07 13:08:32 +08:00
Edwin Cheng
d387bfdc4a Simplify SubtreeTokenSource 2021-01-05 02:00:51 +08:00
Edwin Cheng
af3d75ad2e Refactor TokenBuffer for reduc cloning 2021-01-05 02:00:46 +08:00
Edwin Cheng
f1ffd14922 Reduce string copying 2021-01-05 00:11:56 +08:00
Jonas Schievink
85cc3cfec9
Revert "Proper handling $crate and local_inner_macros" 2021-01-03 11:47:57 +01:00
Edwin Cheng
fe5340d970 Introduce HygieneFrames for proper token hyginee 2021-01-02 20:39:57 +08:00
Edwin Cheng
ede1b11741 Remove unused clones in mbe 2020-12-31 02:58:41 +08:00
Edwin Cheng
266b14d4b5 Refactor mbe parsing code 2020-12-30 02:51:06 +08:00
bors[bot]
77ad203a71
Merge
7060: Fix mbe fail to pass expr with attr r=edwin0cheng a=edwin0cheng

bors r+

Fixes 

Co-authored-by: Edwin Cheng <edwin0cheng@gmail.com>
2020-12-28 12:34:19 +00:00
Edwin Cheng
710407b11d Fix mbe fail to pass expr with attr 2020-12-28 19:53:00 +08:00
Edwin Cheng
b5c29af02a Special case underscore in mbe meta op 2020-12-28 18:51:43 +08:00
Edwin Cheng
b33453f844 Rollback 6929 2020-12-28 18:47:00 +08:00
Edwin Cheng
26e1f7696a Ignore third punct in 2-composite punct 2020-12-27 23:51:00 +08:00
Edwin Cheng
f942d10e80 Use pattern_single instead of pattern in mbe pat 2020-12-20 17:13:40 +08:00
Laurențiu Nicola
75a26f64ff mbe: treat _ as ident 2020-12-20 08:06:17 +02:00
Lukas Wirth
dd496223f5 Node-ify lifetimes 2020-12-16 14:16:09 +01:00
Jonas Schievink
c1cb595382 Move to upstream macro_rules! model 2020-12-15 15:37:37 +01:00
Edwin Cheng
ae29fb0211 Add missing arg for eat_char 2020-12-12 06:14:11 +08:00
Edwin Cheng
175229ab3d negative sign matching in mbe matching for literal 2020-12-11 17:59:04 +08:00
Jonas Schievink
1b26520971 Add dedicated error for "proc macro not found" 2020-11-27 13:50:20 +01:00
Laurențiu Nicola
3aca697511 Simplify error formatting 2020-11-26 21:07:15 +02:00
Jonas Schievink
2c85db8eb6 Implement Display for macro expansion errors 2020-11-26 16:56:22 +01:00
Jonas Schievink
6a9338e979 Use ExpandResult instead of MacroResult
`MacroResult` is redundant
2020-11-26 16:48:17 +01:00
Jonas Schievink
c784c8ec76 Use named fields in ExpandResult 2020-11-26 16:04:23 +01:00
Aleksey Kladov
ba8d6d1e4e Remove more unreachable pubs 2020-11-02 16:58:33 +01:00
Aleksey Kladov
b610118453 Deny unreachable-pub
It's very useful when `pub` is equivalent to "this is crate's public
API", let's enforce this!

Ideally, we should enforce it for local `cargo test`, and only during
CI, but that needs https://github.com/rust-lang/cargo/issues/5034.
2020-11-02 14:07:08 +01:00
Aleksey Kladov
fd8622e1ec Cleanup 2020-10-07 11:55:20 +02:00
Tim
27798ee575
Added unit test for negative number literals in macros. 2020-10-06 22:28:13 +02:00
Tim
8cf9362984
Fixed parsing of negative number literals in macros. 2020-10-06 22:28:13 +02:00
Jean SIMARD
875ad9b5c4
Bump smol_str from 0.1.16 to 0.1.17 2020-09-24 16:39:08 +02:00
Pavan Kumar Sunkara
335add49db Add description for crates that will be published 2020-08-24 13:07:22 +02:00
Pavan Kumar Sunkara
a8fa5cd42e Add version to deps in cargo.toml 2020-08-24 11:10:41 +02:00
Aleksey Kladov
2f45cfc415 Rename ra_mbe -> mbe 2020-08-13 10:36:04 +02:00