This is my first code contribution to Rust, so I'm sure there are some issues with the changes I've made.
I've added the `quote_arg!`, `quote_block!`, `quote_path!`, and `quote_meta_item!` quasiquoting macros. From my experience trying to build AST in compiler plugins, I would like to be able to build any AST piece with a quasiquoting macro (e.g., `quote_struct_field!` or `quote_variant!`) and then use those AST pieces in other quasiquoting macros, but this pull request just adds some of the low-hanging fruit.
I'm not sure if these additions are desirable, and I'm sure these macros can be implemented in an external crate if not.
Stricter checking of stability attributes + enforcement of their invariants at compile time
(+ removed dead file librustc_front/attr.rs)
I intended to enforce use of `reason` for unstable items as well (it normally presents for new items), but it turned out too intrusive, many older unstable items don't have `reason`s.
r? @aturon
I'm studying how stability works and do some refactoring along the way, so it's probably not the last PR.
This PR removes random remaining `Ident`s outside of libsyntax and performs general cleanup
In particular, interfaces of `Name` and `Ident` are tidied up, `Name`s and `Ident`s being small `Copy` aggregates are always passed to functions by value, and `Ident`s are never used as keys in maps, because `Ident` comparisons are tricky.
Although this PR closes https://github.com/rust-lang/rust/issues/6993 there's still work related to it:
- `Name` can be made `NonZero` to compress numerous `Option<Name>`s and `Option<Ident>`s but it requires const unsafe functions.
- Implementation of `PartialEq` on `Ident` should be eliminated and replaced with explicit hygienic, non-hygienic or member-wise comparisons.
- Finally, large parts of AST can potentially be converted to `Name`s in the same way as HIR to clearly separate identifiers used in hygienic and non-hygienic contexts.
r? @nrc
Make sure Name, SyntaxContext and Ident are passed by value
Make sure Idents don't serve as keys (or parts of keys) in maps, Ident comparison is not well defined
This is a [breaking-change] for syntax extension authors. The fix is to use MultiModifier or MultiDecorator, which have the same functionality but are more flexible. Users of syntax extensions are unaffected.
This is a [breaking-change] for syntax extension authors. The fix is to use MultiModifier or MultiDecorator, which have the same functionality but are more flexible. Users of syntax extensions are unaffected.
This is theoretically a breaking change, but GitHub search turns up no
uses of it, and most non-built-in cfg's are passed via cargo features,
which look like `feature = "..."`, and hence can't overlap.
This pull request implements the functionality for [RFC 873](https://github.com/rust-lang/rfcs/blob/master/text/0873-type-macros.md). This is currently just an update of @freebroccolo's branch from January, the corresponding commits are linked in each commit message.
@nikomatsakis and I had talked about updating the macro language to support a lifetime fragment specifier, and it is possible to do that work on this branch as well. If so we can (collectively) talk about it next week during the pre-RustCamp work week.
This commit is an implementation of [RFC 1184][rfc] which tweaks the behavior of
the `#![no_std]` attribute and adds a new `#![no_core]` attribute. The
`#![no_std]` attribute now injects `extern crate core` at the top of the crate
as well as the libcore prelude into all modules (in the same manner as the
standard library's prelude). The `#![no_core]` attribute disables both std and
core injection.
[rfc]: https://github.com/rust-lang/rfcs/pull/1184
Fixes#25022
This adapts the deriving mechanism to not repeat bounds for the same type parameter. To give an example: for the following code:
```rust
#[derive(Clone)]
pub struct FlatMap<I, U: IntoIterator, F> {
iter: I,
f: F,
frontiter: Option<U::IntoIter>,
backiter: Option<U::IntoIter>,
}
```
the latest nightly generates the following impl signature:
```rust
impl <I: ::std::clone::Clone,
U: ::std::clone::Clone + IntoIterator,
F: ::std::clone::Clone>
::std::clone::Clone for FlatMap<I, U, F> where
I: ::std::clone::Clone,
F: ::std::clone::Clone,
U::IntoIter: ::std::clone::Clone,
U::IntoIter: ::std::clone::Clone
```
With these changes, the signature changes to this:
```rust
impl <I, U: IntoIterator, F> ::std::clone::Clone for FlatMap<I, U, F> where
I: ::std::clone::Clone,
F: ::std::clone::Clone,
U::IntoIter: ::std::clone::Clone
```
(Nothing in the body of the impl changes)
Note that the second impl is more permissive, as it doesn't have a `Clone` bound on `U` at all. There was a compile-fail test that failed due to this. I don't understand why we would want the old behaviour (and nobody on IRC could tell me either), so please tell me if there is a good reason that I missed.
`LocalSource` indicated wether a let binding originated from for-loop desugaring to enable specialized error messages, but for-loop expansion has changed and this is now achieved through `MatchSource::ForLoopDesugar`.
This introduces a test for #23389 and improves the error behaviour to treat the malformed LHS as an error, not a compiler bug.
The parse phase that precedes the call to `check_lhs_nt_follows` could possibly be enhanced to police the format itself (which the old code suggests was the original intention), but I'm not sure that's any nicer than just parsing the matcher as generic rust code and then policing the specific requirements for being a macro matcher afterwards (as this does).
Fixes#23389
(Over time the stability checking has gotten more finicky; in
particular one must attach the (whole) span of the original `in PLACE
BLOCK` expression to the injected references to unstable paths, as
noted in the comments.)
call `push_compiler_expansion` during the placement-`in` expansion.
Even after expansion, the generated expressions still track depth of
such pushes (i.e. how often you have "pushed" without a corresponding
"pop"), and we add a rule that in a context with a positive
`push_unsafe!` depth, it is effectively an `unsafe` block context.
(This way, we can inject code that uses `unsafe` features, but still
contains within it a sub-expression that should inherit the outer
safety checking setting, outside of the injected code.)
This is a total hack; it not only needs a feature-gate, but probably
should be feature-gated forever (if possible).
ignore-pretty in test/run-pass/pushpop-unsafe-okay.rs
This commit expands the follow set of the `ty` and `path` macro fragments to
include the semicolon token as well. A semicolon is already allowed after these
tokens, so it's currently a little too restrictive to not have a semicolon
allowed. For example:
extern {
fn foo() -> i32; // semicolon after type
}
fn main() {
struct Foo;
Foo; // semicolon after path
}
This commit expands the follow set of the `ty` and `path` macro fragments to
include the semicolon token as well. A semicolon is already allowed after these
tokens, so it's currently a little too restrictive to not have a semicolon
allowed. For example:
extern {
fn foo() -> i32; // semicolon after type
}
fn main() {
struct Foo;
Foo; // semicolon after path
}
The common pattern `iter::repeat(elt).take(n).collect::<Vec<_>>()` is
exactly equivalent to `vec![elt; n]`, do this replacement in the whole
tree.
(Actually, vec![] is smart enough to only call clone n - 1 times, while
the former solution would call clone n times, and this fact is
virtually irrelevant in practice.)
The new code generated for deriving on enums looks something like this:
```rust
let __self0_vi = unsafe {
std::intrinsics::discriminant_value(&self) } as i32;
let __self1_vi = unsafe {
std::intrinsics::discriminant_value(&__arg1) } as i32;
let __self2_vi = unsafe {
std::intrinsics::discriminant_value(&__arg2) } as i32;
///
if __self0_vi == __self1_vi && __self0_vi == __self2_vi && ... {
match (...) {
(Variant1, Variant1, ...) => Body1
(Variant2, Variant2, ...) => Body2,
...
_ => ::core::intrinsics::unreachable()
}
}
else {
... // catch-all remainder can inspect above variant index values.
}
```
This helps massively for C-like enums since they will be compiled as a
single comparison giving observed speedups of up to 8x. For more complex
enums the speedup is more difficult to measure but it should not be
slower to generate code this way regardless.
Needed to support:
```rust
match X {
pattern if Y ...
}
for pattern in Y {}
```
IMO, this shouldn't require an RFC because it can't interfere with any future language changes (because `pattern if` and `pattern in` are already legal in rust) and can't cause any ambiguity.
This is a port of @eddyb's `const-fn` branch. I rebased it, tweaked a few things, and added tests as well as a feature gate. The set of tests is still pretty rudimentary, I'd appreciate suggestions on new tests to write. Also, a double-check that the feature-gate covers all necessary cases.
One question: currently, the feature-gate allows the *use* of const functions from stable code, just not the definition. This seems to fit our usual strategy, and implies that we might (perhaps) allow some constant functions in libstd someday, even before stabilizing const-fn, if we were willing to commit to the existence of const fns but found some details of their impl unsatisfactory.
r? @pnkfelix
- add feature gate
- add basic tests
- adjust parser to eliminate conflict between `const fn` and associated
constants
- allow `const fn` in traits/trait-impls, but forbid later in type check
- correct some merge conflicts
This allows compiling entire crates from memory or preprocessing source files before they are tokenized.
Minor API refactoring included, which is a [breaking-change] for libsyntax users:
* `ParseSess::{next_node_id, reserve_node_ids}` moved to rustc's `Session`
* `new_parse_sess` -> `ParseSess::new`
* `new_parse_sess_special_handler` -> `ParseSess::with_span_handler`
* `mk_span_handler` -> `SpanHandler::new`
* `default_handler` -> `Handler::new`
* `mk_handler` -> `Handler::with_emitter`
* `string_to_filemap(sess source, path)` -> `sess.codemap().new_filemap(path, source)`
There were still some mentions of `~[T]` and `~T`, mostly in comments and debugging statements. I tried to do my best to preserve meaning, but I might have gotten some wrong-- I'm happy to fix anything :)
I've found that there are still huge amounts of occurrences of `task`s in the documentation. This PR tries to eliminate all of them in favor of `thread`.
An automated script was run against the `.rs` and `.md` files,
subsituting every occurrence of `task` with `thread`. In the `.rs`
files, only the texts in the comment blocks were affected.
There are two interesting kinds of breakage illustrated here:
1. `Box<Trait>` in many contexts is treated as `Box<Trait + 'static>`,
due to [RFC 599]. However, in a type like `&'a Box<Trait>`, the
`Box<Trait>` type will be expanded to `Box<Trait + 'a>`, again due
to [RFC 599]. This, combined with the fix to Issue 25199, leads to
a borrowck problem due the combination of this function signature
(in src/libstd/net/parser.rs):
```rust
fn read_or<T>(&mut self, parsers: &mut [Box<FnMut(&mut Parser) -> Option<T>>]) -> Option<T>;
```
with this call site (again in src/libstd/net/parser.rs):
```rust
fn read_ip_addr(&mut self) -> Option<IpAddr> {
let ipv4_addr = |p: &mut Parser| p.read_ipv4_addr().map(|v4| IpAddr::V4(v4));
let ipv6_addr = |p: &mut Parser| p.read_ipv6_addr().map(|v6| IpAddr::V6(v6));
self.read_or(&mut [Box::new(ipv4_addr), Box::new(ipv6_addr)])
}
```
yielding borrowck errors like:
```
parser.rs:265:27: 265:69 error: borrowed value does not live long enough
parser.rs:265 self.read_or(&mut [Box::new(ipv4_addr), Box::new(ipv6_addr)])
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
(full log at: https://gist.github.com/pnkfelix/e2e80f1a71580f5d3103 )
The issue here is perhaps subtle: the `parsers` argument is
inferred to be taking a slice of boxed objects with the implicit
lifetime bound attached to the `self` parameter to `read_or`.
Meanwhile, the fix to Issue 25199 (added in a forth-coming commit)
is forcing us to assume that each boxed object may have a
destructor that could refer to state of that lifetime, and
*therefore* that inferred lifetime is required to outlive the boxed
object itself.
In this case, the relevant boxed object here is not going to make
any such references; I believe it is just an artifact of how the
expression was built that it is not assigned type:
`Box<FnMut(&mut Parser) -> Option<T> + 'static>`.
(i.e., mucking with the expression is probably one way to fix this
problem).
But the other way to fix it, adopted here, is to change the
`read_or` method type to force make the (presumably-intended)
`'static` bound explicit on the boxed `FnMut` object.
(Note: this is still just the *first* example of breakage.)
2. In `macro_rules.rs`, the `TTMacroExpander` trait defines a method
with signature:
```rust
fn expand<'cx>(&self, cx: &'cx mut ExtCtxt, ...) -> Box<MacResult+'cx>;
```
taking a `&'cx mut ExtCtxt` as an argument and returning a
`Box<MacResult'cx>`.
The fix to Issue 25199 (added in aforementioned forth-coming
commit) assumes that a value of type `Box<MacResult+'cx>` may, in
its destructor, refer to a reference of lifetime `'cx`; thus the
`'cx` lifetime is forced to outlive the returned value.
Meanwhile, within `expand.rs`, the old code was doing:
```rust
match expander.expand(fld.cx, ...).make_pat() { ... => immutable borrow of fld.cx ... }
```
The problem is that the `'cx` lifetime, inferred for the
`expander.expand` call, has now been extended so that it has to
outlive the temporary R-value returned by `expanded.expand`. But
call is also reborrowing `fld.cx` *mutably*, which means that this
reborrow must end before any immutable borrow of `fld.cx`; but
there is one of those within the match body. (Note that the
temporary R-values for the input expression to `match` all live as
long as the whole `match` expression itself (see Issue #3511 and PR
#11585).
To address this, I moved the construction of the pat value into its
own `let`-statement, so that the `Box<MacResult>` will only live
for as long as the initializing expression for the `let`-statement,
and thus allow the subsequent immutable borrow within the `match`.
[RFC 599]: https://github.com/rust-lang/rfcs/blob/master/text/0599-default-object-bound.md
This changes the `ToTokens` implementations for expressions, statements,
etc. with almost-trivial ones that produce `Interpolated(*Nt(...))`
pseudo-tokens. In this way, quasiquote now works the same way as macros
do: already-parsed AST fragments are used as-is, not reparsed.
The `ToSource` trait is removed. Quasiquote no longer involves
pretty-printing at all, which removes the need for the
`encode_with_hygiene` hack. All associated machinery is removed.
A new `Nonterminal` is added, NtArm, which the parser now interpolates.
This is just for quasiquote, not macros (although it could be in the
future).
`ToTokens` is no longer implemented for `Arg` (although this could be
added again) and `Generics` (which I don't think makes sense).
This breaks any compiler extensions that relied on the ability of
`ToTokens` to turn AST fragments back into inspectable token trees. For
this reason, this closes#16987.
As such, this is a [breaking-change].
Fixes#16472.
Fixes#15962.
Fixes#17397.
Fixes#16617.
Changes the style guidelines regarding unit tests to recommend using a
sub-module named "tests" instead of "test" for unit tests as "test"
might clash with imports of libtest.
This is a little bit tricky, since with include_str!, we know that we
are including utf-8 content, so it's safe to store the source as a
String in a FileMap. We don't know that for include_bytes!, but I don't
think we actually need to track the contents anyways, so I'm passing "".
new_filemap does check for the zero length content, and it should be
reasonable, howeven I'm not sure if it would be better to pass None
instead of Some(Rc::new("")) as the src component of a FileMap.
Fixes bug #24348
This is a little bit tricky, since with include_str!, we know that we
are including utf-8 content, so it's safe to store the source as a
String in a FileMap. We don't know that for include_bytes!, but I don't
think we actually need to track the contents anyways, so I'm passing "".
new_filemap does check for the zero length content, and it should be
reasonable, howeven I'm not sure if it would be better to pass None
instead of Some(Rc::new("")) as the src component of a FileMap.
Fixes bug #24348
There are syntax extensions that call `std::rt::begin_unwind` passing it a `usize`. I updated the syntax extension to instead pass `u32`, but for bootstrapping reasons, I needed to create a `#[cfg(stage0)]` version of `std::rt::begin_unwind` and therefore also `panic!`.
`format_args!` uses `#[allow_internal_unstable]` to access internal
functions and structs that are marked unstable. For this to work, the
spans on AST nodes referencing unstable internals must be equal (same
lo/hi values) to the `format_args!` call site, so that the stability
checker can recognize that the AST node was generated by the macro.
* In noop_fold_expr, call new_span in these cases:
- ExprMethodCall's identifier
- ExprField's identifier
- ExprTupField's integer
Calling new_span for ExprMethodCall's identifier is necessary to print
an acceptable diagnostic for write!(&2, ""). We see this error:
<std macros>:2:20: 2:66 error: type `&mut _` does not implement any method in scope named `write_fmt`
<std macros>:2 ( & mut * $ dst ) . write_fmt ( format_args ! ( $ ( $ arg ) * ) ) )
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
With this change, we also see a macro expansion backtrace leading to
the write!(&2, "") call site.
* After fully expanding a macro, we replace the expansion expression's
span with the original span. Call fld.new_span to add a backtrace to
this span. (Note that I'm call new_span after bt.pop(), so the macro
just expanded isn't on the backtrace.)
The motivating example for this change is println!("{}"). The format
string literal is concat!($fmt, "arg") and is inside the libstd macro.
We need to see the backtrace to find the println! call site.
* Add a backtrace to the format_args! format expression span.
Addresses #23459
Statement macros are now treated somewhat like item macros, in that a statement macro can now expand into a series of statements, rather than just a single statement.
This allows statement macros to be nested inside other kinds of macros and expand properly, where previously the expansion would only work when no nesting was present.
See:
- `src/test/run-pass/macro-stmt_macro_in_expr_macro.rs`
- `src/test/run-pass/macro-nested_stmt_macro.rs`
This changes the interface of the MacResult trait. make_stmt has become make_stmts and now returns a vector, rather than a single item. Plugin writers who were implementing MacResult will have breakage, as well as anyone using MacEager::stmt.
See:
- `src/libsyntax/ext/base.rs`
This also causes a minor difference in behavior to the diagnostics produced by certain malformed macros.
See:
- `src/test/compile-fail/macro-incomplete-parse.rs`
Implements pop() on SmallVector, and uses it to expand the final semicolon
in a statement macro expansion more efficiently.
Corrects the placement of the call to fld.cx.bt_pop(). It must run
unconditionally to reverse the corresponding push.
Statement macros are now treated somewhat like item macros, in that a
statement macro can now expand into a series of statements, rather than
just a single statement.
This allows statement macros to be nested inside other kinds of macros and
expand properly, where previously the expansion would only work when no
nesting was present.
See: src/test/run-pass/macro-stmt_macro_in_expr_macro.rs
src/test/run-pass/macro-nested_stmt_macro.rs
This changes the interface of the MacResult trait. make_stmt has become
make_stmts and now returns a vector, rather than a single item. Plugin
writers who were implementing MacResult will have breakage, as well as
anyone using MacEager::stmt.
See: src/libsyntax/ext/base.rs
This also causes a minor difference in behavior to the diagnostics
produced by certain malformed macros.
See: src/test/compile-fail/macro-incomplete-parse.rs
- Functions in parser.rs return PResult<> rather than panicing
- Other functions in libsyntax call panic! explicitly for now if they rely on panicing behaviour.
- 'panictry!' macro added as scaffolding while converting panicing functions.
(This does the same as 'unwrap()' but is easier to grep for and turn into try!())
- Leaves panicing wrappers for the following functions so that the
quote_* macros behave the same:
- parse_expr, parse_item, parse_pat, parse_arm, parse_ty, parse_stmt
RFC pending, but this is the patch that does it.
Totally untested. Likely needs some removed imports. std::collections docs should also be updated to provide better examples.
Closes#23508
