### Rationale
There is no reason to support more than 2³² nodes or names at this moment, as compiling something that big (even without considering the quadratic space usage of some analysis passes) would take at least **64GB**.
Meanwhile, some can't (or barely can) compile rustc because it requires almost **1.5GB**.
### Potential problems
Can someone confirm this doesn't affect metadata (de)serialization? I can't tell myself, I know nothing about it.
### Results
Some structures have a size reduction of 25% to 50%: [before](https://gist.github.com/luqmana/3a82a51fa9c86d9191fa) - [after](https://gist.github.com/eddyb/5a75f8973d3d8018afd3).
Sadly, there isn't a massive change in the memory used for compiling stage2 librustc (it doesn't go over **1.4GB** as [before](http://huonw.github.io/isrustfastyet/mem/), but I can barely see the difference).
However, my own testcase (previously peaking at **1.6GB** in typeck) shows a reduction of **200**-**400MB**.
Currently, the parser doesn't give any context when it finds an unclosed
delimiter and it's not EOF. Report the most recent unclosed delimiter, to help
the user along.
Closes#10636
This adds bindings to the remaining functions provided by libuv, all of which
are useful operations on files which need to get exposed somehow.
Some highlights:
* Dropped `FileReader` and `FileWriter` and `FileStream` for one `File` type
* Moved all file-related methods to be static methods under `File`
* All directory related methods are still top-level functions
* Created `io::FilePermission` types (backed by u32) that are what you'd expect
* Created `io::FileType` and refactored `FileStat` to use FileType and
FilePermission
* Removed the expanding matrix of `FileMode` operations. The mode of reading a
file will not have the O_CREAT flag, but a write mode will always have the
O_CREAT flag.
Closes#10130Closes#10131Closes#10121
This commit moves all thread-blocking I/O functions from the std::os module.
Their replacements can be found in either std::rt::io::file or in a hidden
"old_os" module inside of native::file. I didn't want to outright delete these
functions because they have a lot of special casing learned over time for each
OS/platform, and I imagine that these will someday get integrated into a
blocking implementation of IoFactory. For now, they're moved to a private module
to prevent bitrot and still have tests to ensure that they work.
I've also expanded the extensions to a few more methods defined on Path, most of
which were previously defined in std::os but now have non-thread-blocking
implementations as part of using the current IoFactory.
The api of io::file is in flux, but I plan on changing it in the next commit as
well.
Closes#10057
The general idea is to remove conditions completely from I/O, so in the meantime
remove the read_error condition to mean the same thing as the io_error condition.
Standardize the is_sep() functions to be the same in both posix and
windows, and re-export from path. Update extra::glob to use this.
Remove the usage of either, as it's going away.
Move the WindowsPath-specific methods out of WindowsPath and make them
top-level functions of path::windows instead. This way you cannot
accidentally write code that will fail to compile on non-windows
architectures without typing ::windows anywhere.
Remove GenericPath::from_c_str() and just impl BytesContainer for
CString instead.
Remove .join_path() and .push_path() and just implement BytesContainer
for Path instead.
Remove FilenameDisplay and add a boolean flag to Display instead.
Remove .each_parent(). It only had one caller, so just inline its
definition there.
Rewrite these methods as methods on Display and FilenameDisplay. This
turns
do path.with_display_str |s| { ... }
into
do path.display().with_str |s| { ... }
Add a new trait BytesContainer that is implemented for both byte vectors
and strings.
Convert Path::from_vec and ::from_str to one function, Path::new().
Remove all the _str-suffixed mutation methods (push, join, with_*,
set_*) and modify the non-suffixed versions to use BytesContainer.
Remove the old path.
Rename path2 to path.
Update all clients for the new path.
Also make some miscellaneous changes to the Path APIs to help the
adoption process.
There's currently a fair amount of code which is being ignored on unnamed blocks
(which are the default now), and I opted to leave it commented out for now. I
intend on very soon revisiting on how we perform linking with extern crates in
an effort to support static linking.
It's unclear to me why these currently aren't allowed, and my best guess is that
a long time ago we didn't strip the ast of cfg nodes before syntax expansion.
Now that this is done, I'm not certain that we should continue to prohibit this
functionality.
This is a step in the right direction towards #5605, because now we can add an
empty `std::macros` module to the documentation with a bunch of empty macros
explaining how they're supposed to be used.
It's unclear to me why these currently aren't allowed, and my best guess is that
a long time ago we didn't strip the ast of cfg nodes before syntax expansion.
Now that this is done, I'm not certain that we should continue to prohibit this
functionality.
This is a step in the right direction towards #5605, because now we can add an
empty `std::macros` module to the documentation with a bunch of empty macros
explaining how they're supposed to be used.
r? anybody It's more helpful to list the span of each open delimiter seen so far
than to print out an error with the span of the last position in the file.
Closes#2354
For the benefit of the pretty printer we want to keep track of how
string literals in the ast were originally represented in the source
code.
This commit changes parser functions so they don't extract strings from
the token stream without at least also returning what style of string
literal it was. This is stored in the resulting ast node for string
literals, obviously, for the package id in `extern mod = r"package id"`
view items, for the inline asm in `asm!()` invocations.
For `asm!()`'s other arguments or for `extern "Rust" fn()` items, I just
the style of string, because it seemed disproportionally cumbersome to
thread that information through the string processing that happens with
those string literals, given the limited advantage raw string literals
would provide in these positions.
The other syntax extensions don't seem to store passed string literals
in the ast, so they also discard the style of strings they parse.
It's more helpful to list the span of each open delimiter seen so far
than to print out an error with the span of the last position in the file.
Closes#2354
Raw string literals are lexed into regular string literals. This is okay
for them to "work" and be usable/testable, but the pretty-printer does
not know about them yet and will just emit regular string literals.
Replaces existing tests for removed obsolete-syntax errors with tests
for the resulting regular errors, adds a test for each of the removed
parser errors to make sure that obsolete forms don't start working
again, removes some obsolete/superfluous tests that were now failing.
Deletes some amount of dead code in the parser, also includes some small
changes to parser error messages to accomodate new tests.
It is simply defined as `f64` across every platform right now.
A use case hasn't been presented for a `float` type defined as the
highest precision floating point type implemented in hardware on the
platform. Performance-wise, using the smallest precision correct for the
use case greatly saves on cache space and allows for fitting more
numbers into SSE/AVX registers.
If there was a use case, this could be implemented as simply a type
alias or a struct thanks to `#[cfg(...)]`.
Closes#6592
The mailing list thread, for reference:
https://mail.mozilla.org/pipermail/rust-dev/2013-July/004632.html
This slurps up everything inside of an 'extern' block into the enclosing module
in order to document them. The documentation must be on the items themselves,
and they'll show up next to everything else on the module index pages.
Closes#5953
There is less implicit removal of various comment styles, and it also removes
extraneous stars occasionally found in docblock comments. It turns out that the
bug for getops was just a differently formatted block.
Closes#9425Closes#9417
This fixes private statics and functions from being usable cross-crates, along
with some bad privacy error messages. This is a reopening of #8365 with all the
privacy checks in privacy.rs instead of resolve.rs (where they should be
anyway).
These maps of exported items will hopefully get used for generating
documentation by rustdoc
Closes#8592
As documented in issue #7945, these literal identifiers are all accepted by rust
today, but they should probably be disallowed (especially `'''`). This changes
all escapable sequences to being *required* to be escaped.
Closes#7945
I wanted to write the tests with more exact spans, but I think #9308 will be fixing that?
As documented in issue #7945, these literal identifiers are all accepted by rust
today, but they should probably be disallowed (especially `'''`). This changes
all escapable sequences to being *required* to be escaped.
Closes#7945
Progress on #7981
This doesn't completely close the issue because `struct A;` is still allowed, and it's a much larger change to disallow that. I'm also not entirely sure that we want to disallow that. Regardless, punting that discussion to the issue instead.
This fixes private statics and functions from being usable cross-crates, along
with some bad privacy error messages. This is a reopening of #8365 with all the
privacy checks in privacy.rs instead of resolve.rs (where they should be
anyway).
These maps of exported items will hopefully get used for generating
documentation by rustdoc
Closes#8592
This large commit implements and `html` output option for rustdoc_ng. The
executable has been altered to be invoked as "rustdoc_ng html <crate>" and
it will dump everything into the local "doc" directory. JSON can still be
generated by changing 'html' to 'json'.
This also fixes a number of bugs in rustdoc_ng relating to comment stripping,
along with some other various issues that I found along the way.
The `make doc` command has been altered to generate the new documentation into
the `doc/ng/$(CRATE)` directories.
... instead of giving their numeric codepoint, following the lead of
fdaae34. So the error message for, say, '\_' mentions _ instead of 95,
and '\●' now mentions \u25cf.
Previously, the lexer calling `rdr.fatal(...)` would report the span of
the last complete token, instead of a span within the erroneous token
(besides one span fixed in 1ac90bb).
This commit adds a wrapper around `rdr.fatal(...)` that sets the span
explicilty, so that all fatal errors in `libsyntax/parse/lexer.rs` now
report the offending code more precisely. A number of tests try to
verify that, though the `compile-fail` testing setup can only check that
the spans are on the right lines, and the "unterminated string/block
comment" errors can't have the line marked at all, so that's incomplete.
Closes#9149.
This commit adds support for `\0` escapes in character and string literals.
Since `\0` is equivalent to `\x00`, this is a direct translation to the latter
escape sequence. Future builds will be able to compile using `\0` directly.
Also updated the grammar specification and added a test for NUL characters.
This doesn't close any bugs as the goal is to convert the parameter to by-value, but this is a step towards being able to make guarantees about `&T` pointers (where T is Freeze) to LLVM.
Remove these in favor of the two traits themselves and the wrapper
function std::from_str::from_str.
Add the function std::num::from_str_radix in the corresponding role for
the FromStrRadix trait.
Work a bit towards #9157 "Remove Either". These instances don't need to use Either and are better expressed in other ways (removing allocations and simplifying types).
This way syntax extensions can generate unsafe blocks without worrying about
them generating unnecessary unsafe warnings. Perhaps a special keyword could be
added to be used in macros, but I don't think that's the best solution.
has a unique id. Fixes numerous bugs in macro expansion and deriving. Add two
representative tests.
Fixes#7971Fixes#6304Fixes#8367Fixes#8754Fixes#8852Fixes#2543Fixes#7654
Also redefine all of the standard logging macros to use more rust code instead
of custom LLVM translation code. This makes them a bit easier to understand, but
also more flexibile for future types of logging.
Additionally, this commit removes the LogType language item in preparation for
changing how logging is performed.
this is necessary so that the new idents are connected to the original strings.
this is important both for error messages, and so that top-level refs get connected
to the right things.
Also redefine all of the standard logging macros to use more rust code instead
of custom LLVM translation code. This makes them a bit easier to understand, but
also more flexibile for future types of logging.
Additionally, this commit removes the LogType language item in preparation for
changing how logging is performed.
This removes the stacking of type parameters that occurs when invoking
trait methods, and fixes all places in the standard library that were
relying on it. It is somewhat awkward in places; I think we'll probably
want something like the `Foo::<for T>::new()` syntax.
The span was fixed at some point to point to the correct character, but
the error message is still bad. Update it to emit the actual character
in question (potentially escaped).
Fixes#3747.
When parsing a trait function, the function must end with either `;` or
`{` (signifying a default implementation). The error message incorrectly
stated that it must be `;` or `}`.
Fixes#6610.
The span was fixed at some point to point to the correct character, but
the error message is still bad. Update it to emit the actual character
in question (potentially escaped).
Fixes#3747.
When parsing a trait function, the function must end with either `;` or
`{` (signifying a default implementation). The error message incorrectly
stated that it must be `;` or `}`.
Fixes#6610.
Fix#3192. r? anyone
There are 4 different new tests, to check some different scenarios for
what the parse context is at the time of recovery, becasue our
compile-fail infrastructure does not appear to handle verifying
error-recovery situations.
Differentiate between unit-like struct definition item and unit-like
struct construction in the error message.
----
More generally, outlines a more generic strategy for parse error
recovery: By committing to an expression/statement at set points in
the parser, we can then do some look-ahead to catch common mistakes
and skip over them.
One detail about this strategy is that you want to avoid emitting the
"helpful" message unless the input is reasonably close to the case of
interest. (E.g. do not warn about a potential unit struct for an
input of the form `let hmm = do foo { } { };`)
To accomplish this, I added (partial) last_token tracking; used for
`commit_stmt` support.
The check_for_erroneous_unit_struct_expecting fn returns bool to
signal whether it "made progress"; currently unused; this is meant for
use to compose several such recovery checks together in a loop.
`enum Token` was 192 bytes (64-bit), as pointed out by pnkfelix; the only
bloating variant being `INTERPOLATED(nonterminal)`.
Updating `enum nonterminal` to use ~ where variants included big types,
shrunk size_of(Token) to 32 bytes (64-bit).
I am unsure if the `nt_ident` variant should have an indirection, with
ast::ident being only 16 bytes (64-bit), but without this, enum Token
would be 40 bytes.
A dumb benchmark says that compilation time is unchanged, while peak
memory usage for compiling std.rs is down 3%
Before::
$ time ./x86_64-unknown-linux-gnu/stage1/bin/rustc --cfg stage1 src/libstd/std.rs
19.00user 0.39system 0:19.41elapsed 99%CPU (0avgtext+0avgdata 627820maxresident)k
0inputs+28896outputs (0major+228665minor)pagefaults 0swaps
$ time ./x86_64-unknown-linux-gnu/stage1/bin/rustc -O --cfg stage1 src/libstd/std.rs
31.64user 0.34system 0:32.02elapsed 99%CPU (0avgtext+0avgdata 629876maxresident)k
0inputs+22432outputs (0major+229411minor)pagefaults 0swaps
After::
$ time ./x86_64-unknown-linux-gnu/stage1/bin/rustc --cfg stage1 src/libstd/std.rs
19.07user 0.45system 0:19.55elapsed 99%CPU (0avgtext+0avgdata 609384maxresident)k
0inputs+28896outputs (0major+221997minor)pagefaults 0swaps
$ time ./x86_64-unknown-linux-gnu/stage1/bin/rustc -O --cfg stage1 src/libstd/std.rs
31.90user 0.34system 0:32.28elapsed 99%CPU (0avgtext+0avgdata 612080maxresident)k
0inputs+22432outputs (0major+223726minor)pagefaults 0swaps
r? @graydon Also, notably, make rustpkgtest depend on the rustpkg executable (otherwise, tests that shell out to rustpgk might run when rustpkg doesn't exist).
This commit allows you to write:
extern mod x = "a/b/c";
which means rustc will search in the RUST_PATH for a package with
ID a/b/c, and bind it to the name `x` if it's found.
Incidentally, move get_relative_to from back::rpath into std::path
There are 4 different new tests, to check some different scenarios for
what the parse context is at the time of recovery, becasue our
compile-fail infrastructure does not appear to handle verifying
error-recovery situations.
Differentiate between unit-like struct definition item and unit-like
struct construction in the error message.
----
More generally, outlines a more generic strategy for parse error
recovery: By committing to an expression/statement at set points in
the parser, we can then do some look-ahead to catch common mistakes
and skip over them.
One detail about this strategy is that you want to avoid emitting the
"helpful" message unless the input is reasonably close to the case of
interest. (E.g. do not warn about a potential unit struct for an
input of the form `let hmm = do foo { } { };`)
To accomplish this, I added (partial) last_token tracking; used for
`commit_stmt` support.
The check_for_erroneous_unit_struct_expecting fn returns bool to
signal whether it "made progress"; currently unused; this is meant for
use to compose several such recovery checks together in a loop.
- Made naming schemes consistent between Option, Result and Either
- Changed Options Add implementation to work like the maybe monad (return None if any of the inputs is None)
- Removed duplicate Option::get and renamed all related functions to use the term `unwrap` instead
`crate => Crate`
`local => Local`
`blk => Block`
`crate_num => CrateNum`
`crate_cfg => CrateConfig`
Also, Crate and Local are not wrapped in spanned<T> anymore.
This does a number of things, but especially dramatically reduce the
number of allocations performed for operations involving attributes/
meta items:
- Converts ast::meta_item & ast::attribute and other associated enums
to CamelCase.
- Converts several standalone functions in syntax::attr into methods,
defined on two traits AttrMetaMethods & AttributeMethods. The former
is common to both MetaItem and Attribute since the latter is a thin
wrapper around the former.
- Deletes functions that are unnecessary due to iterators.
- Converts other standalone functions to use iterators and the generic
AttrMetaMethods rather than allocating a lot of new vectors (e.g. the
old code would have to allocate a new vector to use functions that
operated on &[meta_item] on &[attribute].)
- Moves the core algorithm of the #[cfg] matching to syntax::attr,
similar to find_inline_attr and find_linkage_metas.
This doesn't have much of an effect on the speed of #[cfg] stripping,
despite hugely reducing the number of allocations performed; presumably
most of the time is spent in the ast folder rather than doing attribute
checks.
Also fixes the Eq instance of MetaItem_ to correctly ignore spans, so
that `rustc --cfg 'foo(bar)'` now works.
This does a number of things, but especially dramatically reduce the
number of allocations performed for operations involving attributes/
meta items:
- Converts ast::meta_item & ast::attribute and other associated enums
to CamelCase.
- Converts several standalone functions in syntax::attr into methods,
defined on two traits AttrMetaMethods & AttributeMethods. The former
is common to both MetaItem and Attribute since the latter is a thin
wrapper around the former.
- Deletes functions that are unnecessary due to iterators.
- Converts other standalone functions to use iterators and the generic
AttrMetaMethods rather than allocating a lot of new vectors (e.g. the
old code would have to allocate a new vector to use functions that
operated on &[meta_item] on &[attribute].)
- Moves the core algorithm of the #[cfg] matching to syntax::attr,
similar to find_inline_attr and find_linkage_metas.
This doesn't have much of an effect on the speed of #[cfg] stripping,
despite hugely reducing the number of allocations performed; presumably
most of the time is spent in the ast folder rather than doing attribute
checks.
Also fixes the Eq instance of MetaItem_ to correctly ignore spaces, so
that `rustc --cfg 'foo(bar)'` now works.
