Our implementation of ebml has diverged from the standard in order
to better serve the needs of the compiler, so it doesn't make much
sense to call what we have ebml anyore. Furthermore, our implementation
is pretty crufty, and should eventually be rewritten into a format
that better suits the needs of the compiler. This patch factors out
serialize::ebml into librbml, otherwise known as the Really Bad
Markup Language. This is a stopgap library that shouldn't be used
by end users, and will eventually be replaced by something better.
[breaking-change]
- unicode tests live in coretest crate
- libcollections str tests need UnicodeChar trait.
- libregex perlw tests were checking a char in the Alphabetic category,
\x2161. Confirmed perl 5.18 considers this a \w character. Changed to
\x2961, which is not \w as the test expects.
- created new crate, libunicode, below libstd
- split Char trait into Char (libcore) and UnicodeChar (libunicode)
- Unicode-aware functions now live in libunicode
- is_alphabetic, is_XID_start, is_XID_continue, is_lowercase,
is_uppercase, is_whitespace, is_alphanumeric, is_control,
is_digit, to_uppercase, to_lowercase
- added width method in UnicodeChar trait
- determines printed width of character in columns, or None if it is
a non-NULL control character
- takes a boolean argument indicating whether the present context is
CJK or not (characters with 'A'mbiguous widths are double-wide in
CJK contexts, single-wide otherwise)
- split StrSlice into StrSlice (libcore) and UnicodeStrSlice
(libunicode)
- functionality formerly in StrSlice that relied upon Unicode
functionality from Char is now in UnicodeStrSlice
- words, is_whitespace, is_alphanumeric, trim, trim_left, trim_right
- also moved Words type alias into libunicode because words method is
in UnicodeStrSlice
- unified Unicode tables from libcollections, libcore, and libregex into
libunicode
- updated unicode.py in src/etc to generate aforementioned tables
- generated new tables based on latest Unicode data
- added UnicodeChar and UnicodeStrSlice traits to prelude
- libunicode is now the collection point for the std::char module,
combining the libunicode functionality with the Char functionality
from libcore
- thus, moved doc comment for char from core::char to unicode::char
- libcollections remains the collection point for std::str
The Unicode-aware functions that previously lived in the Char and
StrSlice traits are no longer available to programs that only use
libcore. To regain use of these methods, include the libunicode crate
and use the UnicodeChar and/or UnicodeStrSlice traits:
extern crate unicode;
use unicode::UnicodeChar;
use unicode::UnicodeStrSlice;
use unicode::Words; // if you want to use the words() method
NOTE: this does *not* impact programs that use libstd, since UnicodeChar
and UnicodeStrSlice have been added to the prelude.
closes#15224
[breaking-change]
This commit is the final step in the libstd facade, #13851. The purpose of this
commit is to move libsync underneath the standard library, behind the facade.
This will allow core primitives like channels, queues, and atomics to all live
in the same location.
There were a few notable changes and a few breaking changes as part of this
movement:
* The `Vec` and `String` types are reexported at the top level of libcollections
* The `unreachable!()` macro was copied to libcore
* The `std::rt::thread` module was moved to librustrt, but it is still
reexported at the same location.
* The `std::comm` module was moved to libsync
* The `sync::comm` module was moved under `sync::comm`, and renamed to `duplex`.
It is now a private module with types/functions being reexported under
`sync::comm`. This is a breaking change for any existing users of duplex
streams.
* All concurrent queues/deques were moved directly under libsync. They are also
all marked with #![experimental] for now if they are public.
* The `task_pool` and `future` modules no longer live in libsync, but rather
live under `std::sync`. They will forever live at this location, but they may
move to libsync if the `std::task` module moves as well.
[breaking-change]
As part of the libstd facade efforts, this commit extracts the runtime interface
out of the standard library into a standalone crate, librustrt. This crate will
provide the following services:
* Definition of the rtio interface
* Definition of the Runtime interface
* Implementation of the Task structure
* Implementation of task-local-data
* Implementation of task failure via unwinding via libunwind
* Implementation of runtime initialization and shutdown
* Implementation of thread-local-storage for the local rust Task
Notably, this crate avoids the following services:
* Thread creation and destruction. The crate does not require the knowledge of
an OS threading system, and as a result it seemed best to leave out the
`rt::thread` module from librustrt. The librustrt module does depend on
mutexes, however.
* Implementation of backtraces. There is no inherent requirement for the runtime
to be able to generate backtraces. As will be discussed later, this
functionality continues to live in libstd rather than librustrt.
As usual, a number of architectural changes were required to make this crate
possible. Users of "stable" functionality will not be impacted by this change,
but users of the `std::rt` module will likely note the changes. A list of
architectural changes made is:
* The stdout/stderr handles no longer live directly inside of the `Task`
structure. This is a consequence of librustrt not knowing about `std::io`.
These two handles are now stored inside of task-local-data.
The handles were originally stored inside of the `Task` for perf reasons, and
TLD is not currently as fast as it could be. For comparison, 100k prints goes
from 59ms to 68ms (a 15% slowdown). This appeared to me to be an acceptable
perf loss for the successful extraction of a librustrt crate.
* The `rtio` module was forced to duplicate more functionality of `std::io`. As
the module no longer depends on `std::io`, `rtio` now defines structures such
as socket addresses, addrinfo fiddly bits, etc. The primary change made was
that `rtio` now defines its own `IoError` type. This type is distinct from
`std::io::IoError` in that it does not have an enum for what error occurred,
but rather a platform-specific error code.
The native and green libraries will be updated in later commits for this
change, and the bulk of this effort was put behind updating the two libraries
for this change (with `rtio`).
* Printing a message on task failure (along with the backtrace) continues to
live in libstd, not in librustrt. This is a consequence of the above decision
to move the stdout/stderr handles to TLD rather than inside the `Task` itself.
The unwinding API now supports registration of global callback functions which
will be invoked when a task fails, allowing for libstd to register a function
to print a message and a backtrace.
The API for registering a callback is experimental and unsafe, as the
ramifications of running code on unwinding is pretty hairy.
* The `std::unstable::mutex` module has moved to `std::rt::mutex`.
* The `std::unstable::sync` module has been moved to `std::rt::exclusive` and
the type has been rewritten to not internally have an Arc and to have an RAII
guard structure when locking. Old code should stop using `Exclusive` in favor
of the primitives in `libsync`, but if necessary, old code should port to
`Arc<Exclusive<T>>`.
* The local heap has been stripped down to have fewer debugging options. None of
these were tested, and none of these have been used in a very long time.
[breaking-change]
As with the previous commit with `librand`, this commit shuffles around some
`collections` code. The new state of the world is similar to that of librand:
* The libcollections crate now only depends on libcore and liballoc.
* The standard library has a new module, `std::collections`. All functionality
of libcollections is reexported through this module.
I would like to stress that this change is purely cosmetic. There are very few
alterations to these primitives.
There are a number of notable points about the new organization:
* std::{str, slice, string, vec} all moved to libcollections. There is no reason
that these primitives shouldn't be necessarily usable in a freestanding
context that has allocation. These are all reexported in their usual places in
the standard library.
* The `hashmap`, and transitively the `lru_cache`, modules no longer reside in
`libcollections`, but rather in libstd. The reason for this is because the
`HashMap::new` contructor requires access to the OSRng for initially seeding
the hash map. Beyond this requirement, there is no reason that the hashmap
could not move to libcollections.
I do, however, have a plan to move the hash map to the collections module. The
`HashMap::new` function could be altered to require that the `H` hasher
parameter ascribe to the `Default` trait, allowing the entire `hashmap` module
to live in libcollections. The key idea would be that the default hasher would
be different in libstd. Something along the lines of:
// src/libstd/collections/mod.rs
pub type HashMap<K, V, H = RandomizedSipHasher> =
core_collections::HashMap<K, V, H>;
This is not possible today because you cannot invoke static methods through
type aliases. If we modified the compiler, however, to allow invocation of
static methods through type aliases, then this type definition would
essentially be switching the default hasher from `SipHasher` in libcollections
to a libstd-defined `RandomizedSipHasher` type. This type's `Default`
implementation would randomly seed the `SipHasher` instance, and otherwise
perform the same as `SipHasher`.
This future state doesn't seem incredibly far off, but until that time comes,
the hashmap module will live in libstd to not compromise on functionality.
* In preparation for the hashmap moving to libcollections, the `hash` module has
moved from libstd to libcollections. A previously snapshotted commit enables a
distinct `Writer` trait to live in the `hash` module which `Hash`
implementations are now parameterized over.
Due to using a custom trait, the `SipHasher` implementation has lost its
specialized methods for writing integers. These can be re-added
backwards-compatibly in the future via default methods if necessary, but the
FNV hashing should satisfy much of the need for speedier hashing.
A list of breaking changes:
* HashMap::{get, get_mut} no longer fails with the key formatted into the error
message with `{:?}`, instead, a generic message is printed. With backtraces,
it should still be not-too-hard to track down errors.
* The HashMap, HashSet, and LruCache types are now available through
std::collections instead of the collections crate.
* Manual implementations of hash should be parameterized over `hash::Writer`
instead of just `Writer`.
[breaking-change]
This commit shuffles around some of the `rand` code, along with some
reorganization. The new state of the world is as follows:
* The librand crate now only depends on libcore. This interface is experimental.
* The standard library has a new module, `std::rand`. This interface will
eventually become stable.
Unfortunately, this entailed more of a breaking change than just shuffling some
names around. The following breaking changes were made to the rand library:
* Rng::gen_vec() was removed. This has been replaced with Rng::gen_iter() which
will return an infinite stream of random values. Previous behavior can be
regained with `rng.gen_iter().take(n).collect()`
* Rng::gen_ascii_str() was removed. This has been replaced with
Rng::gen_ascii_chars() which will return an infinite stream of random ascii
characters. Similarly to gen_iter(), previous behavior can be emulated with
`rng.gen_ascii_chars().take(n).collect()`
* {IsaacRng, Isaac64Rng, XorShiftRng}::new() have all been removed. These all
relied on being able to use an OSRng for seeding, but this is no longer
available in librand (where these types are defined). To retain the same
functionality, these types now implement the `Rand` trait so they can be
generated with a random seed from another random number generator. This allows
the stdlib to use an OSRng to create seeded instances of these RNGs.
* Rand implementations for `Box<T>` and `@T` were removed. These seemed to be
pretty rare in the codebase, and it allows for librand to not depend on
liballoc. Additionally, other pointer types like Rc<T> and Arc<T> were not
supported. If this is undesirable, librand can depend on liballoc and regain
these implementations.
* The WeightedChoice structure is no longer built with a `Vec<Weighted<T>>`,
but rather a `&mut [Weighted<T>]`. This means that the WeightedChoice
structure now has a lifetime associated with it.
* The `sample` method on `Rng` has been moved to a top-level function in the
`rand` module due to its dependence on `Vec`.
cc #13851
[breaking-change]
This commit moves reflection (as well as the {:?} format modifier) to a new
libdebug crate, all of which is marked experimental.
This is a breaking change because it now requires the debug crate to be
explicitly linked if the :? format qualifier is used. This means that any code
using this feature will have to add `extern crate debug;` to the top of the
crate. Any code relying on reflection will also need to do this.
Closes#12019
[breaking-change]
This commit is part of the libstd facade RFC, issue #13851. This creates a new
library, liballoc, which is intended to be the core allocation library for all
of Rust. It is pinned on the basic assumption that an allocation failure is an
abort or failure.
This module has inherited the heap/libc_heap modules from std::rt, the owned/rc
modules from std, and the arc module from libsync. These three pointers are
currently the three most core pointer implementations in Rust.
The UnsafeArc type in std::sync should be considered deprecated and replaced by
Arc<Unsafe<T>>. This commit does not currently migrate to this type, but future
commits will continue this refactoring.
Use sync:1️⃣:Once to fetch the mach_timebase_info only once when
running precise_time_ns(). This helps because mach_timebase_info() is
surprisingly inefficient. Also fix the order of operations when applying
the timebase to the mach absolute time value.
This improves the time on my machine from
```
test tests::bench_precise_time_ns ... bench: 157 ns/iter (+/- 4)
```
to
```
test tests::bench_precise_time_ns ... bench: 38 ns/iter (+/- 3)
```
and it will get even faster once #14174 lands.
Passing `--pretty flowgraph=<NODEID>` makes rustc print a control flow graph.
In pratice, you will also need to pass the additional option:
`-o <FILE>` to emit output to a `.dot` file for graphviz.
(You can only print the flow-graph for a particular block in the AST.)
----
An interesting implementation detail is the way the code puts both the
node index (`cfg::CFGIndex`) and a reference to the payload
(`cfg::CFGNode`) into the single `Node` type that is used for
labelling and walking the graph. I had once mistakenly thought that I
only wanted the `cfg::CFGNode`, but for labelling, you really want the
cfg index too, rather than e.g. trying to use the `ast::NodeId` as the
label (which breaks down e.g. due to `ast::DUMMY_NODE_ID`).
----
As a drive-by fix, I had to fix `rustc::middle::cfg::construct`
interface to reflect changes that have happened on the master branch
while I was getting this integrated into the compiler. (The next
commit actually adds tests of the `--pretty flowgraph` functionality,
so that should ensure that the `rustc::middle::cfg` code does not go
stale again.)
The core library in theory has 0 dependencies, but in practice it has some in
order for it to be efficient. These dependencies are in the form of the basic
memory operations provided by libc traditionally, such as memset, memcmp, etc.
These functions are trivial to implement and themselves have 0 dependencies.
This commit adds a new crate, librlibc, which will serve the purpose of
providing these dependencies. The crate is never linked to by default, but is
available to be linked to by downstream consumers. Normally these functions are
provided by the system libc, but in other freestanding contexts a libc may not
be available. In these cases, librlibc will suffice for enabling execution with
libcore.
cc #10116
This adds a `std::rt::heap` module with a nice allocator API. It's a
step towards fixing #13094 and is a starting point for working on a
generic allocator trait.
The revision used for the jemalloc submodule is the stable 3.6.0 release.
Closes#11807
This code does not belong in libstd, and rather belongs in a dedicated crate. In
the future, the syntax::ext::format module should move to the fmt_macros crate
(hence the name of the crate), but for now the fmt_macros crate will only
contain the format string parser.
The entire fmt_macros crate is marked #[experimental] because it is not meant
for general consumption, only the format!() interface is officially supported,
not the internals.
This is a breaking change for anyone using the internals of std::fmt::parse.
Some of the flags have moved to std::fmt::rt, while the actual parsing support
has all moved to the fmt_macros library.
[breaking-change]
This primary fix brought on by this upgrade is the proper matching of the ```
and ~~~ doc blocks. This also moves hoedown to a git submodule rather than a
bundled repository.
Additionally, hoedown is stricter about code blocks, so this ended up fixing a
lot of invalid code blocks (ending with " ```" instead of "```", or ending with
"~~~~" instead of "~~~").
Closes#12776
The previous dependency calculation was based on an arbitrary set of asterisks
at an arbitrary depth, but using the recursive version should be much more
robust in figuring out what's dependent.
Closes#13118
This commit switches over the backtrace infrastructure from piggy-backing off
the RUST_LOG environment variable to using the RUST_BACKTRACE environment
variable (logging is now disabled in libstd).
This commit moves all logging out of the standard library into an external
crate. This crate is the new crate which is responsible for all logging macros
and logging implementation. A few reasons for this change are:
* The crate map has always been a bit of a code smell among rust programs. It
has difficulty being loaded on almost all platforms, and it's used almost
exclusively for logging and only logging. Removing the crate map is one of the
end goals of this movement.
* The compiler has a fair bit of special support for logging. It has the
__log_level() expression as well as generating a global word per module
specifying the log level. This is unfairly favoring the built-in logging
system, and is much better done purely in libraries instead of the compiler
itself.
* Initialization of logging is much easier to do if there is no reliance on a
magical crate map being available to set module log levels.
* If the logging library can be written outside of the standard library, there's
no reason that it shouldn't be. It's likely that we're not going to build the
highest quality logging library of all time, so third-party libraries should
be able to provide just as high-quality logging systems as the default one
provided in the rust distribution.
With a migration such as this, the change does not come for free. There are some
subtle changes in the behavior of liblog vs the previous logging macros:
* The core change of this migration is that there is no longer a physical
log-level per module. This concept is still emulated (it is quite useful), but
there is now only a global log level, not a local one. This global log level
is a reflection of the maximum of all log levels specified. The previously
generated logging code looked like:
if specified_level <= __module_log_level() {
println!(...)
}
The newly generated code looks like:
if specified_level <= ::log::LOG_LEVEL {
if ::log::module_enabled(module_path!()) {
println!(...)
}
}
Notably, the first layer of checking is still intended to be "super fast" in
that it's just a load of a global word and a compare. The second layer of
checking is executed to determine if the current module does indeed have
logging turned on.
This means that if any module has a debug log level turned on, all modules
with debug log levels get a little bit slower (they all do more expensive
dynamic checks to determine if they're turned on or not).
Semantically, this migration brings no change in this respect, but
runtime-wise, this will have a perf impact on some code.
* A `RUST_LOG=::help` directive will no longer print out a list of all modules
that can be logged. This is because the crate map will no longer specify the
log levels of all modules, so the list of modules is not known. Additionally,
warnings can no longer be provided if a malformed logging directive was
supplied.
The new "hello world" for logging looks like:
#[phase(syntax, link)]
extern crate log;
fn main() {
debug!("Hello, world!");
}
This commit shreds all remnants of libextra from the compiler and standard
distribution. Two modules, c_vec/tempfile, were moved into libstd after some
cleanup, and the other modules were moved to separate crates as seen fit.
Closes#8784Closes#12413Closes#12576
Whenever a failure happens, if a program is run with
`RUST_LOG=std::rt::backtrace` a backtrace will be printed to the task's stderr
handle. Stack traces are uncondtionally printed on double-failure and
rtabort!().
This ended up having a nontrivial implementation, and here's some highlights of
it:
* We're bundling libbacktrace for everything but OSX and Windows
* We use libgcc_s and its libunwind apis to get a backtrace of instruction
pointers
* On OSX we use dladdr() to go from an instruction pointer to a symbol
* On unix that isn't OSX, we use libbacktrace to get symbols
* Windows, as usual, has an entirely separate implementation
Lots more fun details and comments can be found in the source itself.
Closes#10128
This functionality is not super-core and so doesn't need to be included
in std. It's possible that std may need rand (it does a little bit now,
for io::test) in which case the functionality required could be moved to
a secret hidden module and reexposed by librand.
Unfortunately, using #[deprecated] here is hard: there's too much to
mock to make it feasible, since we have to ensure that programs still
typecheck to reach the linting phase.
This converts it to be very similar to crates.mk, with a single list of
the documentation items creating all the necessary bits and pieces.
Changes include:
- rustdoc is used to render HTML & test standalone docs
- documentation building now obeys NO_REBUILD=1
- testing standalone docs now obeys NO_REBUILD=1
- L10N is slightly less broken (in particular, it shares dependencies
and code with the rest of the code)
- PDFs can be built for all documentation items, not just tutorial and
manual
- removes the obsolete & unused extract-tests.py script
- adjust the CSS for standalone docs to use the rustdoc syntax
highlighting
The compiler itself doesn't necessarily need any features of green threading
such as spawning tasks and lots of I/O, so libnative is slightly more
appropriate for rustc to use itself.
This should also help the rusti bot which is currently incompatible with libuv.
This trades an O(n) allocation + memcpy for a O(1) proc allocation (for
the destructor). Most users only need &[u8] anyway (all of the users in
the main repo), and so this offers large gains.
These two containers are indeed collections, so their place is in
libcollections, not in libstd. There will always be a hash map as part of the
standard distribution of Rust, but by moving it out of the standard library it
makes libstd that much more portable to more platforms and environments.
This conveniently also removes the stuttering of 'std::hashmap::HashMap',
although 'collections::HashMap' is only one character shorter.
Two unfortunate allocations were wrapping a proc() in a proc() with
GreenTask::build_start_wrapper, and then boxing this proc in a ~proc() inside of
Context::new(). Both of these allocations were a direct result from two
conditions:
1. The Context::new() function has a nice api of taking a procedure argument to
start up a new context with. This inherently required an allocation by
build_start_wrapper because extra code needed to be run around the edges of a
user-provided proc() for a new task.
2. The initial bootstrap code only understood how to pass one argument to the
next function. By modifying the assembly and entry points to understand more
than one argument, more information is passed through in registers instead of
allocating a pointer-sized context.
This is sadly where I end up throwing mips under a bus because I have no idea
what's going on in the mips context switching code and don't know how to modify
it.
Closes#7767
cc #11389
