* Reexport io::mem and io::buffered structs directly under io, make mem/buffered
private modules
* Remove with_mem_writer
* Remove DEFAULT_CAPACITY and use DEFAULT_BUF_SIZE (in io::buffered)
cc #11119
* Reexport io::mem and io::buffered structs directly under io, make mem/buffered
private modules
* Remove with_mem_writer
* Remove DEFAULT_CAPACITY and use DEFAULT_BUF_SIZE (in io::buffered)
This is a first pass on support for procedural macros that aren't hardcoded into libsyntax. It is **not yet ready to merge** but I've opened a PR to have a chance to discuss some open questions and implementation issues.
Example
=======
Here's a silly example showing off the basics:
my_synext.rs
```rust
#[feature(managed_boxes, globs, macro_registrar, macro_rules)];
extern mod syntax;
use syntax::ast::{Name, token_tree};
use syntax::codemap::Span;
use syntax::ext::base::*;
use syntax::parse::token;
#[macro_export]
macro_rules! exported_macro (() => (2))
#[macro_registrar]
pub fn macro_registrar(register: |Name, SyntaxExtension|) {
register(token::intern(&"make_a_1"),
NormalTT(@SyntaxExpanderTT {
expander: SyntaxExpanderTTExpanderWithoutContext(expand_make_a_1),
span: None,
} as @SyntaxExpanderTTTrait,
None));
}
pub fn expand_make_a_1(cx: &mut ExtCtxt, sp: Span, tts: &[token_tree]) -> MacResult {
if !tts.is_empty() {
cx.span_fatal(sp, "make_a_1 takes no arguments");
}
MRExpr(quote_expr!(cx, 1i))
}
```
main.rs:
```rust
#[feature(phase)];
#[phase(syntax)]
extern mod my_synext;
fn main() {
assert_eq!(1, make_a_1!());
assert_eq!(2, exported_macro!());
}
```
Overview
=======
Crates that contain syntax extensions need to define a function with the following signature and annotation:
```rust
#[macro_registrar]
pub fn registrar(register: |ast::Name, ext::base::SyntaxExtension|) { ... }
```
that should call the `register` closure with each extension it defines. `macro_rules!` style macros can be tagged with `#[macro_export]` to be exported from the crate as well.
Crates that wish to use externally loadable syntax extensions load them by adding the `#[phase(syntax)]` attribute to an `extern mod`. All extensions registered by the specified crate are loaded with the same scoping rules as `macro_rules!` macros. If you want to use a crate both for syntax extensions and normal linkage, you can use `#[phase(syntax, link)]`.
Open questions
===========
* ~~Does the `macro_crate` syntax make sense? It wraps an entire `extern mod` declaration which looks a bit weird but is nice in the sense that the crate lookup logic can be identical between normal external crates and external macro crates. If the `extern mod` syntax, changes, this will get it for free, etc.~~ Changed to a `phase` attribute.
* ~~Is the magic name `macro_crate_registration` the right way to handle extension registration? It could alternatively be handled by a function annotated with `#[macro_registration]` I guess.~~ Switched to an attribute.
* The crate loading logic lives inside of librustc, which means that the syntax extension infrastructure can't directly access it. I've worked around this by passing a `CrateLoader` trait object from the driver to libsyntax that can call back into the crate loading logic. It should be possible to pull things apart enough that this isn't necessary anymore, but it will be an enormous refactoring project. I think we'll need to create a couple of new libraries: libsynext libmetadata/ty and libmiddle.
* Item decorator extensions can be loaded but the `deriving` decorator itself can't be extended so you'd need to do e.g. `#[deriving_MyTrait] #[deriving(Clone)]` instead of `#[deriving(MyTrait, Clone)]`. Is this something worth bothering with for now?
Remaining work
===========
- [x] ~~There is not yet support for rustdoc downloading and compiling referenced macro crates as it does for other referenced crates. This shouldn't be too hard I think.~~
- [x] ~~This is not testable at stage1 and sketchily testable at stages above that. The stage *n* rustc links against the stage *n-1* libsyntax and librustc. Unfortunately, crates in the test/auxiliary directory link against the stage *n* libstd, libextra, libsyntax, etc. This causes macro crates to fail to properly dynamically link into rustc since names end up being mangled slightly differently. In addition, when rustc is actually installed onto a system, there are actually do copies of libsyntax, libstd, etc: the ones that user code links against and a separate set from the previous stage that rustc itself uses. By this point in the bootstrap process, the two library versions *should probably* be binary compatible, but it doesn't seem like a sure thing. Fixing this is apparently hard, but necessary to properly cross compile as well and is being tracked in #11145.~~ The offending tests are ignored during `check-stage1-rpass` and `check-stage1-cfail`. When we get a snapshot that has this commit, I'll look into how feasible it'll be to get them working on stage1.
- [x] ~~`macro_rules!` style macros aren't being exported. Now that the crate loading infrastructure is there, this should just require serializing the AST of the macros into the crate metadata and yanking them out again, but I'm not very familiar with that part of the compiler.~~
- [x] ~~The `macro_crate_registration` function isn't type-checked when it's loaded. I poked around in the `csearch` infrastructure a bit but didn't find any super obvious ways of checking the type of an item with a certain name. Fixing this may also eliminate the need to `#[no_mangle]` the registration function.~~ Now that the registration function is identified by an attribute, typechecking this will be like typechecking other annotated functions.
- [x] ~~The dynamic libraries that are loaded are never unloaded. It shouldn't require too much work to tie the lifetime of the `DynamicLibrary` object to the `MapChain` that its extensions are loaded into.~~
- [x] ~~The compiler segfaults sometimes when loading external crates. The `DynamicLibrary` reference and code objects from that library are both put into the same hash table. When the table drops, due to the random ordering the library sometimes drops before the objects do. Once #11228 lands it'll be easy to fix this.~~
If a reexport comes from a non-public module, then the documentation for the
reexport will be inlined into the module that exports it, but if the reexport is
targeted at a public type (like the prelude), then it is not inlined but rather
hyperlinked.
Right now the --crate-id and related flags are all process *after* the entire
crate is parsed. This is less than desirable when used with makefiles because it
means that just to learn the output name of the crate you have to parse the
entire crate (unnecessary).
This commit changes the behavior to lift the handling of these flags much sooner
in the compilation process. This allows us to not have to parse the entire crate
and only have to worry about parsing the crate attributes themselves. The
related methods have all been updated to take an array of attributes rather than
a crate.
Additionally, this ceases duplication of the "what output are we producing"
logic in order to correctly handle things in the case of --test.
Finally, this adds tests for all of this functionality to ensure that it does
not regress.
Right now the --crate-id and related flags are all process *after* the entire
crate is parsed. This is less than desirable when used with makefiles because it
means that just to learn the output name of the crate you have to parse the
entire crate (unnecessary).
This commit changes the behavior to lift the handling of these flags much sooner
in the compilation process. This allows us to not have to parse the entire crate
and only have to worry about parsing the crate attributes themselves. The
related methods have all been updated to take an array of attributes rather than
a crate.
Additionally, this ceases duplication of the "what output are we producing"
logic in order to correctly handle things in the case of --test.
Finally, this adds tests for all of this functionality to ensure that it does
not regress.
We were previously reading metadata via `ar p`, but as learned from rustdoc
awhile back, spawning a process to do something is pretty slow. Turns out LLVM
has an Archive class to read archives, but it cannot write archives.
This commits adds bindings to the read-only version of the LLVM archive class
(with a new type that only has a read() method), and then it uses this class
when reading the metadata out of rlibs. When you put this in tandem of not
compressing the metadata, reading the metadata is 4x faster than it used to be
The timings I got for reading metadata from the respective libraries was:
libstd-04ff901e-0.9-pre.dylib => 100ms
libstd-04ff901e-0.9-pre.rlib => 23ms
librustuv-7945354c-0.9-pre.dylib => 4ms
librustuv-7945354c-0.9-pre.rlib => 1ms
librustc-5b94a16f-0.9-pre.dylib => 87ms
librustc-5b94a16f-0.9-pre.rlib => 35ms
libextra-a6ebb16f-0.9-pre.dylib => 63ms
libextra-a6ebb16f-0.9-pre.rlib => 15ms
libsyntax-2e4c0458-0.9-pre.dylib => 86ms
libsyntax-2e4c0458-0.9-pre.rlib => 22ms
In order to always take advantage of these faster metadata read-times, I sort
the files in filesearch based on whether they have an rlib extension or not
(prefer all rlib files first).
Overall, this halved the compile time for a `fn main() {}` crate from 0.185s to
0.095s on my system (when preferring dynamic linking). Reading metadata is still
the slowest pass of the compiler at 0.035s, but it's getting pretty close to
linking at 0.021s! The next best optimization is to just not copy the metadata
from LLVM because that's the most expensive part of reading metadata right now.
By performing this logic very late in the build process, it ended up leading to
bugs like those found in #10973 where certain stages of the build process
expected a particular output format which didn't end up being the case. In order
to fix this, the build output generation is moved very early in the build
process to the absolute first thing in phase 2.
Closes#10973
If it's a trait method, this checks the stability attribute of the
method inside the trait definition. Otherwise, it checks the method
implementation itself.
By performing this logic very late in the build process, it ended up leading to
bugs like those found in #10973 where certain stages of the build process
expected a particular output format which didn't end up being the case. In order
to fix this, the build output generation is moved very early in the build
process to the absolute first thing in phase 2.
Closes#10973
When performing LTO, the rust compiler has an opportunity to completely strip
all landing pads in all dependent libraries. I've modified the LTO pass to
recognize the -Z no-landing-pads option when also running an LTO pass to flag
everything in LLVM as nothrow. I've verified that this prevents any and all
invoke instructions from being emitted.
I believe that this is one of our best options for moving forward with
accomodating use-cases where unwinding doesn't really make sense. This will
allow libraries to be built with landing pads by default but allow usage of them
in contexts where landing pads aren't necessary.
