Previously, symbols with rust escape sequences (denoted with dollar signs)
weren't demangled if the escape sequence showed up in the middle. This alters
the printing loop to look through the entire string for dollar characters.
On windows, correctly check for errors when spawning threads, and on both
windows and unix handle the error more gracefully rather than printing an opaque
assertion failure.
Closes#13589
This removes all resizability support for ~[T] vectors in preparation of DST.
The only growable vector remaining is Vec<T>. In summary, the following methods
from ~[T] and various functions were removed. Each method/function has an
equivalent on the Vec type in std::vec unless otherwise stated.
* slice::OwnedCloneableVector
* slice::OwnedEqVector
* slice::append
* slice::append_one
* slice::build (no replacement)
* slice::bytes::push_bytes
* slice::from_elem
* slice::from_fn
* slice::with_capacity
* ~[T].capacity()
* ~[T].clear()
* ~[T].dedup()
* ~[T].extend()
* ~[T].grow()
* ~[T].grow_fn()
* ~[T].grow_set()
* ~[T].insert()
* ~[T].pop()
* ~[T].push()
* ~[T].push_all()
* ~[T].push_all_move()
* ~[T].remove()
* ~[T].reserve()
* ~[T].reserve_additional()
* ~[T].reserve_exect()
* ~[T].retain()
* ~[T].set_len()
* ~[T].shift()
* ~[T].shrink_to_fit()
* ~[T].swap_remove()
* ~[T].truncate()
* ~[T].unshift()
* ~str.clear()
* ~str.set_len()
* ~str.truncate()
Note that no other API changes were made. Existing apis that took or returned
~[T] continue to do so.
[breaking-change]
There are currently a number of return values from the std::comm methods, not
all of which are necessarily completely expressive:
* `Sender::try_send(t: T) -> bool`
This method currently doesn't transmit back the data `t` if the send fails
due to the other end having disconnected. Additionally, this shares the name
of the synchronous try_send method, but it differs in semantics in that it
only has one failure case, not two (the buffer can never be full).
* `SyncSender::try_send(t: T) -> TrySendResult<T>`
This method accurately conveys all possible information, but it uses a
custom type to the std::comm module with no convenience methods on it.
Additionally, if you want to inspect the result you're forced to import
something from `std::comm`.
* `SyncSender::send_opt(t: T) -> Option<T>`
This method uses Some(T) as an "error value" and None as a "success value",
but almost all other uses of Option<T> have Some/None the other way
* `Receiver::try_recv(t: T) -> TryRecvResult<T>`
Similarly to the synchronous try_send, this custom return type is lacking in
terms of usability (no convenience methods).
With this number of drawbacks in mind, I believed it was time to re-work the
return types of these methods. The new API for the comm module is:
Sender::send(t: T) -> ()
Sender::send_opt(t: T) -> Result<(), T>
SyncSender::send(t: T) -> ()
SyncSender::send_opt(t: T) -> Result<(), T>
SyncSender::try_send(t: T) -> Result<(), TrySendError<T>>
Receiver::recv() -> T
Receiver::recv_opt() -> Result<T, ()>
Receiver::try_recv() -> Result<T, TryRecvError>
The notable changes made are:
* Sender::try_send => Sender::send_opt. This renaming brings the semantics in
line with the SyncSender::send_opt method. An asychronous send only has one
failure case, unlike the synchronous try_send method which has two failure
cases (full/disconnected).
* Sender::send_opt returns the data back to the caller if the send is guaranteed
to fail. This method previously returned `bool`, but then it was unable to
retrieve the data if the data was guaranteed to fail to send. There is still a
race such that when `Ok(())` is returned the data could still fail to be
received, but that's inherent to an asynchronous channel.
* Result is now the basis of all return values. This not only adds lots of
convenience methods to all return values for free, but it also means that you
can inspect the return values with no extra imports (Ok/Err are in the
prelude). Additionally, it's now self documenting when something failed or not
because the return value has "Err" in the name.
Things I'm a little uneasy about:
* The methods send_opt and recv_opt are not returning options, but rather
results. I felt more strongly that Option was the wrong return type than the
_opt prefix was wrong, and I coudn't think of a much better name for these
methods. One possible way to think about them is to read the _opt suffix as
"optionally".
* Result<T, ()> is often better expressed as Option<T>. This is only applicable
to the recv_opt() method, but I thought it would be more consistent for
everything to return Result rather than one method returning an Option.
Despite my two reasons to feel uneasy, I feel much better about the consistency
in return values at this point, and I think the only real open question is if
there's a better suffix for {send,recv}_opt.
Closes#11527
There are currently a number of return values from the std::comm methods, not
all of which are necessarily completely expressive:
Sender::try_send(t: T) -> bool
This method currently doesn't transmit back the data `t` if the send fails
due to the other end having disconnected. Additionally, this shares the name
of the synchronous try_send method, but it differs in semantics in that it
only has one failure case, not two (the buffer can never be full).
SyncSender::try_send(t: T) -> TrySendResult<T>
This method accurately conveys all possible information, but it uses a
custom type to the std::comm module with no convenience methods on it.
Additionally, if you want to inspect the result you're forced to import
something from `std::comm`.
SyncSender::send_opt(t: T) -> Option<T>
This method uses Some(T) as an "error value" and None as a "success value",
but almost all other uses of Option<T> have Some/None the other way
Receiver::try_recv(t: T) -> TryRecvResult<T>
Similarly to the synchronous try_send, this custom return type is lacking in
terms of usability (no convenience methods).
With this number of drawbacks in mind, I believed it was time to re-work the
return types of these methods. The new API for the comm module is:
Sender::send(t: T) -> ()
Sender::send_opt(t: T) -> Result<(), T>
SyncSender::send(t: T) -> ()
SyncSender::send_opt(t: T) -> Result<(), T>
SyncSender::try_send(t: T) -> Result<(), TrySendError<T>>
Receiver::recv() -> T
Receiver::recv_opt() -> Result<T, ()>
Receiver::try_recv() -> Result<T, TryRecvError>
The notable changes made are:
* Sender::try_send => Sender::send_opt. This renaming brings the semantics in
line with the SyncSender::send_opt method. An asychronous send only has one
failure case, unlike the synchronous try_send method which has two failure
cases (full/disconnected).
* Sender::send_opt returns the data back to the caller if the send is guaranteed
to fail. This method previously returned `bool`, but then it was unable to
retrieve the data if the data was guaranteed to fail to send. There is still a
race such that when `Ok(())` is returned the data could still fail to be
received, but that's inherent to an asynchronous channel.
* Result is now the basis of all return values. This not only adds lots of
convenience methods to all return values for free, but it also means that you
can inspect the return values with no extra imports (Ok/Err are in the
prelude). Additionally, it's now self documenting when something failed or not
because the return value has "Err" in the name.
Things I'm a little uneasy about:
* The methods send_opt and recv_opt are not returning options, but rather
results. I felt more strongly that Option was the wrong return type than the
_opt prefix was wrong, and I coudn't think of a much better name for these
methods. One possible way to think about them is to read the _opt suffix as
"optionally".
* Result<T, ()> is often better expressed as Option<T>. This is only applicable
to the recv_opt() method, but I thought it would be more consistent for
everything to return Result rather than one method returning an Option.
Despite my two reasons to feel uneasy, I feel much better about the consistency
in return values at this point, and I think the only real open question is if
there's a better suffix for {send,recv}_opt.
Closes#11527
Few places where previous version of tidy script cannot find XXX:
* inside one-line comment preceding by a few spaces;
* inside multiline comments (now it finds it if multiline comment starts
on the same line with XXX).
Change occurences of XXX found by new tidy script.
move errno -> IoError converter into std, bubble up OSRng errors
Also adds a general errno -> `~str` converter to `std::os`, and makes the failure messages for the things using `OSRng` (e.g. (transitively) the task-local RNG, meaning hashmap initialisation failures aren't such a black box).
The various ...Rng::new() methods can hit IO errors from the OSRng they use,
and it seems sensible to expose them at a higher level. Unfortunately, writing
e.g. `StdRng::new().unwrap()` gives a much poorer error message than if it
failed internally, but this is a problem with all `IoResult`s.
This commit deals with the fallout of the previous change by making tuples
structs have public fields where necessary (now that the fields are private by
default).
This commit contains an implementation of synchronous, bounded channels for
Rust. This is an implementation of the proposal made last January [1]. These
channels are built on mutexes, and currently focus on a working implementation
rather than speed. Receivers for sync channels have select() implemented for
them, but there is currently no implementation of select() for sync senders.
Rust will continue to provide both synchronous and asynchronous channels as part
of the standard distribution, there is no intent to remove asynchronous
channels. This flavor of channels is meant to provide an alternative to
asynchronous channels because like green tasks, asynchronous channels are not
appropriate for all situations.
[1] - https://mail.mozilla.org/pipermail/rust-dev/2014-January/007924.html
* Remove clone-ability from all primitives. All shared state will now come
from the usage of the primitives being shared, not the primitives being
inherently shareable. This allows for fewer allocations for stack-allocated
primitives.
* Add `Mutex<T>` and `RWLock<T>` which are stack-allocated primitives for purely
wrapping a piece of data
* Remove `RWArc<T>` in favor of `Arc<RWLock<T>>`
* Remove `MutexArc<T>` in favor of `Arc<Mutex<T>>`
* Shuffle around where things are located
* The `arc` module now only contains `Arc`
* A new `lock` module contains `Mutex`, `RWLock`, and `Barrier`
* A new `raw` module contains the primitive implementations of `Semaphore`,
`Mutex`, and `RWLock`
* The Deref/DerefMut trait was implemented where appropriate
* `CowArc` was removed, the functionality is now part of `Arc` and is tagged
with `#[experimental]`.
* The crate now has #[deny(missing_doc)]
* `Arc` now supports weak pointers
This is not a large-scale rewrite of the functionality contained within the
`sync` crate, but rather a shuffling of who does what an a thinner hierarchy of
ownership to allow for better composability.
The proper usage of shared types is now sharing through `&self` rather than
`&mut self` because the mutable version will provide stronger guarantees (no
aliasing on *any* thread).
This commit switches over the backtrace infrastructure from piggy-backing off
the RUST_LOG environment variable to using the RUST_BACKTRACE environment
variable (logging is now disabled in libstd).
This commit removes all internal support for the previously used __log_level()
expression. The logging subsystem was previously modified to not rely on this
magical expression. This also removes the only other function to use the
module_data map in trans, decl_gc_metadata. It appears that this is an ancient
function from a GC only used long ago.
This does not remove the crate map entirely, as libgreen still uses it to hook
in to the event loop provided by libgreen.
This commit moves all logging out of the standard library into an external
crate. This crate is the new crate which is responsible for all logging macros
and logging implementation. A few reasons for this change are:
* The crate map has always been a bit of a code smell among rust programs. It
has difficulty being loaded on almost all platforms, and it's used almost
exclusively for logging and only logging. Removing the crate map is one of the
end goals of this movement.
* The compiler has a fair bit of special support for logging. It has the
__log_level() expression as well as generating a global word per module
specifying the log level. This is unfairly favoring the built-in logging
system, and is much better done purely in libraries instead of the compiler
itself.
* Initialization of logging is much easier to do if there is no reliance on a
magical crate map being available to set module log levels.
* If the logging library can be written outside of the standard library, there's
no reason that it shouldn't be. It's likely that we're not going to build the
highest quality logging library of all time, so third-party libraries should
be able to provide just as high-quality logging systems as the default one
provided in the rust distribution.
With a migration such as this, the change does not come for free. There are some
subtle changes in the behavior of liblog vs the previous logging macros:
* The core change of this migration is that there is no longer a physical
log-level per module. This concept is still emulated (it is quite useful), but
there is now only a global log level, not a local one. This global log level
is a reflection of the maximum of all log levels specified. The previously
generated logging code looked like:
if specified_level <= __module_log_level() {
println!(...)
}
The newly generated code looks like:
if specified_level <= ::log::LOG_LEVEL {
if ::log::module_enabled(module_path!()) {
println!(...)
}
}
Notably, the first layer of checking is still intended to be "super fast" in
that it's just a load of a global word and a compare. The second layer of
checking is executed to determine if the current module does indeed have
logging turned on.
This means that if any module has a debug log level turned on, all modules
with debug log levels get a little bit slower (they all do more expensive
dynamic checks to determine if they're turned on or not).
Semantically, this migration brings no change in this respect, but
runtime-wise, this will have a perf impact on some code.
* A `RUST_LOG=::help` directive will no longer print out a list of all modules
that can be logged. This is because the crate map will no longer specify the
log levels of all modules, so the list of modules is not known. Additionally,
warnings can no longer be provided if a malformed logging directive was
supplied.
The new "hello world" for logging looks like:
#[phase(syntax, link)]
extern crate log;
fn main() {
debug!("Hello, world!");
}
I ignored AtomicU64 methods on MIPS target
because libgcc doesn't implement MIPS32 64-bit atomic operations.
Otherwise it would cause link failure.
By the way, the patched LLVM doesn't have MIPS split stack anymore.
Should I file an issue about that?
On android, libgcc is missing the _Unwind_GetIP symbol because it's defined as a
macro. This is the same case for arm linux, so this commit adds the necessary
cfgs in place to use the "expanded macro" in rust for arm linux.
Add some more infrastructure support for demangling `$`-sequences, as well as fixing demangling of closure symbol names if there's more than one closure in a function.
This is something that is plausibly useful, and is provided by libuv. This is
not currently surfaced as part of the `TcpStream` type, but it may possibly
appear in the future. For now only the raw functionality is provided through the
Rtio objects.
The rust compiler not only outputs symbols in the form that C++ does, but it
also mangle symbols like '&' and '~' to special compiler-defined escape
sequences. For convenience, these symbols are demangled when printing
backtraces.
This is something that is plausibly useful, and is provided by libuv. This is
not currently surfaced as part of the `TcpStream` type, but it may possibly
appear in the future. For now only the raw functionality is provided through the
Rtio objects.
Whenever a failure happens, if a program is run with
`RUST_LOG=std::rt::backtrace` a backtrace will be printed to the task's stderr
handle. Stack traces are uncondtionally printed on double-failure and
rtabort!().
This ended up having a nontrivial implementation, and here's some highlights of
it:
* We're bundling libbacktrace for everything but OSX and Windows
* We use libgcc_s and its libunwind apis to get a backtrace of instruction
pointers
* On OSX we use dladdr() to go from an instruction pointer to a symbol
* On unix that isn't OSX, we use libbacktrace to get symbols
* Windows, as usual, has an entirely separate implementation
Lots more fun details and comments can be found in the source itself.
Closes#10128
It is often convenient to have forms of weak linkage or other various types of
linkage. Sadly, just using these flavors of linkage are not compatible with
Rust's typesystem and how it considers some pointers to be non-null.
As a compromise, this commit adds support for weak linkage to external symbols,
but it requires that this is only placed on extern statics of type `*T`.
Codegen-wise, we get translations like:
// rust code
extern {
#[linkage = "extern_weak"]
static foo: *i32;
}
// generated IR
@foo = extern_weak global i32
@_some_internal_symbol = internal global *i32 @foo
All references to the rust value of `foo` then reference `_some_internal_symbol`
instead of the symbol `_foo` itself. This allows us to guarantee that the
address of `foo` will never be null while the value may sometimes be null.
An example was implemented in `std::rt::thread` to determine if
`__pthread_get_minstack()` is available at runtime, and a test is checked in to
use it for a static value as well. Function pointers a little odd because you
still need to transmute the pointer value to a function pointer, but it's
thankfully better than not having this capability at all.
When using tasks in Rust, the expectation is that the runtime does not exit
before all tasks have exited. This is enforced in libgreen through the
`SchedPool` type, and it is enforced in libnative through a `bookkeeping` module
and a global count/mutex pair. Unfortunately, this means that a process which
originates with libgreen will not wait for spawned native tasks.
In order to fix this problem, the bookkeeping module was moved from libnative to
libstd so the runtime itself can wait for native tasks to exit. Green tasks do
not manage themselves through this bookkeeping module, but native tasks will
continue to manage themselves through this module.
Closes#12684
I've been playing around with code size when linking to libstd recently, and these were some findings I found that really helped code size. I started out by eliminating all I/O implementations from libnative and instead just return an unimplemented error.
In doing so, a `fn main() {}` executable was ~378K before this patch, and about 170K after the patch. These size wins are all pretty minor, but they all seemed pretty reasonable to me. With native I/O not stubbed out, this takes the size of an LTO executable from 675K to 400K.
Most of these are unnecessary because we're only looking at static strings. This
also moves to Vec in a few places instead of ~[T].
This didn't end up getting much of a code size win (update_log_settings is the
third largest function in the executables I'm looking at), but this seems like a
generally nice improvement regardless.
There's a lot of these types in the compiler libraries, and a few of the
older or private stdlib ones. Some types are obviously meant to be
public, others not so much.
The printing of the error message on stack overflow had two sometimes false
assumptions previously. The first is that a local task was always available (it
called Local::take) and the second is that it used `println!` instead of
manually writing.
The first assumption isn't necessarily true because while stack overflow will
likely only be detected in situations that a local task is available, it's not
guaranteed to always be in TLS. For example, during a `println!` call a task
may be blocking, causing it to be unavailable. By using Local::try_take(), we
can be resilient against these occurrences.
The second assumption could lead to odd behavior because the stdout logger can
be overwritten to run arbitrary code. Currently this should be possible, but the
utility is much diminished because a stack overflow translates to an abort()
instead of a failure.
The printing of the error message on stack overflow had two sometimes false
assumptions previously. The first is that a local task was always available (it
called Local::take) and the second is that it used println! instead of
manually writing.
The first assumption isn't necessarily true because while stack overflow will
likely only be detected in situations that a local task is available, it's not
guaranteed to always be in TLS. For example, during a println! call a task
may be blocking, causing it to be unavailable. By using Local::try_take(), we
can be resilient against these occurrences.
The second assumption could lead to odd behavior because the stdout logger can
be overwritten to run arbitrary code. Currently this should be possible, but the
utility is much diminished because a stack overflow translates to an abort()
instead of a failure.
Apparently weak linkage and dlopen aren't quite working out for applications
like servo on android. There appears to be a bug or two in how android loads
dynamic libraries and for some reason libservo.so isn't being found.
As a temporary solution, add an extern "C" function to libstd which can be
called if you have a handle to the crate map manually. When crawling the crate
map, we then check this manual symbol before falling back to the old solutions.
cc #11731
The std::run module is a relic from a standard library long since past, and
there's not much use to having two modules to execute processes with where one
is slightly more convenient. This commit merges the two modules, moving lots of
functionality from std::run into std::io::process and then deleting
std::run.
New things you can find in std::io::process are:
* Process::new() now only takes prog/args
* Process::configure() takes a ProcessConfig
* Process::status() is the same as run::process_status
* Process::output() is the same as run::process_output
* I/O for spawned tasks is now defaulted to captured in pipes instead of ignored
* Process::kill() was added (plus an associated green/native implementation)
* Process::wait_with_output() is the same as the old finish_with_output()
* destroy() is now signal_exit()
* force_destroy() is now signal_kill()
Closes#2625Closes#10016
With the stability attributes we can put public-but unstable modules next to others, so this moves `intrinsics` and `raw` out of the `unstable` module (and marks both as `#[experimental]`).
These two containers are indeed collections, so their place is in
libcollections, not in libstd. There will always be a hash map as part of the
standard distribution of Rust, but by moving it out of the standard library it
makes libstd that much more portable to more platforms and environments.
This conveniently also removes the stuttering of 'std::hashmap::HashMap',
although 'collections::HashMap' is only one character shorter.
- adds a `LockGuard` type returned by `.lock` and `.trylock` that unlocks the mutex in the destructor
- renames `mutex::Mutex` to `StaticNativeMutex`
- adds a `NativeMutex` type with a destructor
- removes `LittleLock`
- adds `#[must_use]` to `sync::mutex::Guard` to remind people to use it
Change `os::args()` and `os::env()` to use `str::from_utf8_lossy()`.
Add new functions `os::args_as_bytes()` and `os::env_as_bytes()` to retrieve the args/env as byte vectors instead.
The existing methods were left returning strings because I expect that the common use-case is to want string handling.
Fixes#7188.
os::args() was using str::raw::from_c_str(), which would assert if the
C-string wasn't valid UTF-8. Switch to using from_utf8_lossy() instead,
and add a separate function os::args_as_bytes() that returns the ~[u8]
byte-vectors instead.
This will hopefully bring us closer to #11937. We're still using gcc's idea of
"startup files", but this should prevent us from leaking in dependencies that we
don't quite want (libgcc for example once compiler-rt is what we use).
Any single-threaded task benchmark will spend a good chunk of time in `kqueue()` on osx and `epoll()` on linux, and the reason for this is that each time a task is terminated it will hit the syscall. When a task terminates, it context switches back to the scheduler thread, and the scheduler thread falls out of `run_sched_once` whenever it figures out that it did some work.
If we know that `epoll()` will return nothing, then we can continue to do work locally (only while there's work to be done). We must fall back to `epoll()` whenever there's active I/O in order to check whether it's ready or not, but without that (which is largely the case in benchmarks), we can prevent the costly syscall and can get a nice speedup.
I've separated the commits into preparation for this change and then the change itself, the last commit message has more details.
Instead, use an enum to allow running both a procedure and sending the task
result over a channel. I expect the common case to be sending on a channel (e.g.
task::try), so don't require an extra allocation in the common case.
cc #11389
The green scheduler can optimize its runtime based on this by deciding to not go
to sleep in epoll() if there is no active I/O and there is a task to be stolen.
This is implemented for librustuv by keeping a count of the number of tasks
which are currently homed. If a task is homed, and then performs a blocking I/O
operation, the count will be nonzero while the task is blocked. The homing count
is intentionally 0 when there are I/O handles, but no handles currently blocked.
The reason for this is that epoll() would only be used to wake up the scheduler
anyway.
The crux of this change was to have a `HomingMissile` contain a mutable borrowed
reference back to the `HomeHandle`. The rest of the change was just dealing with
this fallout. This reference is used to decrement the homed handle count in a
HomingMissile's destructor.
Also note that the count maintained is not atomic because all of its
increments/decrements/reads are all on the same I/O thread.
This, the Nth rewrite of channels, is not a rewrite of the core logic behind
channels, but rather their API usage. In the past, we had the distinction
between oneshot, stream, and shared channels, but the most recent rewrite
dropped oneshots in favor of streams and shared channels.
This distinction of stream vs shared has shown that it's not quite what we'd
like either, and this moves the `std::comm` module in the direction of "one
channel to rule them all". There now remains only one Chan and one Port.
This new channel is actually a hybrid oneshot/stream/shared channel under the
hood in order to optimize for the use cases in question. Additionally, this also
reduces the cognitive burden of having to choose between a Chan or a SharedChan
in an API.
My simple benchmarks show no reduction in efficiency over the existing channels
today, and a 3x improvement in the oneshot case. I sadly don't have a
pre-last-rewrite compiler to test out the old old oneshots, but I would imagine
that the performance is comparable, but slightly slower (due to atomic reference
counting).
This commit also brings the bonus bugfix to channels that the pending queue of
messages are all dropped when a Port disappears rather then when both the Port
and the Chan disappear.
Declare a `type SendStr = MaybeOwned<'static>` to ease readibility of
types that needed the old SendStr behavior.
Implement all the traits for MaybeOwned that SendStr used to implement.
This also drops support for the managed pointer POISON_ON_FREE feature
as it's not worth adding back the support for it. After a snapshot, the
leftovers can be removed.
This has been a long time coming. Conditions in rust were initially envisioned
as being a good alternative to error code return pattern. The idea is that all
errors are fatal-by-default, and you can opt-in to handling the error by
registering an error handler.
While sounding nice, conditions ended up having some unforseen shortcomings:
* Actually handling an error has some very awkward syntax:
let mut result = None;
let mut answer = None;
io::io_error::cond.trap(|e| { result = Some(e) }).inside(|| {
answer = Some(some_io_operation());
});
match result {
Some(err) => { /* hit an I/O error */ }
None => {
let answer = answer.unwrap();
/* deal with the result of I/O */
}
}
This pattern can certainly use functions like io::result, but at its core
actually handling conditions is fairly difficult
* The "zero value" of a function is often confusing. One of the main ideas
behind using conditions was to change the signature of I/O functions. Instead
of read_be_u32() returning a result, it returned a u32. Errors were notified
via a condition, and if you caught the condition you understood that the "zero
value" returned is actually a garbage value. These zero values are often
difficult to understand, however.
One case of this is the read_bytes() function. The function takes an integer
length of the amount of bytes to read, and returns an array of that size. The
array may actually be shorter, however, if an error occurred.
Another case is fs::stat(). The theoretical "zero value" is a blank stat
struct, but it's a little awkward to create and return a zero'd out stat
struct on a call to stat().
In general, the return value of functions that can raise error are much more
natural when using a Result as opposed to an always-usable zero-value.
* Conditions impose a necessary runtime requirement on *all* I/O. In theory I/O
is as simple as calling read() and write(), but using conditions imposed the
restriction that a rust local task was required if you wanted to catch errors
with I/O. While certainly an surmountable difficulty, this was always a bit of
a thorn in the side of conditions.
* Functions raising conditions are not always clear that they are raising
conditions. This suffers a similar problem to exceptions where you don't
actually know whether a function raises a condition or not. The documentation
likely explains, but if someone retroactively adds a condition to a function
there's nothing forcing upstream users to acknowledge a new point of task
failure.
* Libaries using I/O are not guaranteed to correctly raise on conditions when an
error occurs. In developing various I/O libraries, it's much easier to just
return `None` from a read rather than raising an error. The silent contract of
"don't raise on EOF" was a little difficult to understand and threw a wrench
into the answer of the question "when do I raise a condition?"
Many of these difficulties can be overcome through documentation, examples, and
general practice. In the end, all of these difficulties added together ended up
being too overwhelming and improving various aspects didn't end up helping that
much.
A result-based I/O error handling strategy also has shortcomings, but the
cognitive burden is much smaller. The tooling necessary to make this strategy as
usable as conditions were is much smaller than the tooling necessary for
conditions.
Perhaps conditions may manifest themselves as a future entity, but for now
we're going to remove them from the standard library.
Closes#9795Closes#8968
This is part of the overall strategy I would like to take when approaching
issue #11165. The only two I/O objects that reasonably want to be "split" are
the network stream objects. Everything else can be "split" by just creating
another version.
The initial idea I had was the literally split the object into a reader and a
writer half, but that would just introduce lots of clutter with extra interfaces
that were a little unnnecssary, or it would return a ~Reader and a ~Writer which
means you couldn't access things like the remote peer name or local socket name.
The solution I found to be nicer was to just clone the stream itself. The clone
is just a clone of the handle, nothing fancy going on at the kernel level.
Conceptually I found this very easy to wrap my head around (everything else
supports clone()), and it solved the "split" problem at the same time.
The cloning support is pretty specific per platform/lib combination:
* native/win32 - uses some specific WSA apis to clone the SOCKET handle
* native/unix - uses dup() to get another file descriptor
* green/all - This is where things get interesting. When we support full clones
of a handle, this implies that we're allowing simultaneous writes
and reads to happen. It turns out that libuv doesn't support two
simultaneous reads or writes of the same object. It does support
*one* read and *one* write at the same time, however. Some extra
infrastructure was added to just block concurrent writers/readers
until the previous read/write operation was completed.
I've added tests to the tcp/unix modules to make sure that this functionality is
supported everywhere.
* All I/O now returns IoResult<T> = Result<T, IoError>
* All formatting traits now return fmt::Result = IoResult<()>
* The if_ok!() macro was added to libstd
EINVAL means that the requested stack size is either not a multiple
of the system page size or that it's smaller than PTHREAD_STACK_MIN.
Figure out what the case is, fix it up and retry. If it still fails,
give up, like before.
Suggestions for future improvements:
* don't fail!() but instead signal a condition, or
* silently ignore the error and use a default sized stack.
Fixes#11694.
The first two commits put the framework in place, the third one contains the meat.
glibc >= 2.15 has a __pthread_get_minstack() function that returns
PTHREAD_STACK_MIN plus however many bytes are needed for thread-local
storage. Use it when it's available because just PTHREAD_STACK_MIN is
not enough in applications that have big thread-local storage
requirements.
Fixes#6233.
Enforce that the stack size is > RED_ZONE + PTHREAD_STACK_MIN. If the
call to pthread_attr_setstacksize() subsequently fails with EINVAL, it
means that the platform requires the stack size to be a multiple of the
page size. In that case, round up to the nearest page and retry.
Fixes#11694.
This ends up saving a single `call` instruction in the optimised code,
but saves a few hundred lines of non-optimised IR for `fn main() {
fail!("foo {}", "bar"); }` (comparing against the minimal generic
baseline from the parent commit).
This splits the vast majority of the code path taken by
`fail!()` (`begin_unwind`) into a separate non-generic inline(never)
function, so that uses of `fail!()` only monomorphise a small amount of
code, reducing code bloat and making very small crates compile faster.
The following are renamed:
* `min_value` => `MIN`
* `max_value` => `MAX`
* `bits` => `BITS`
* `bytes` => `BYTES`
All tests pass, except for `run-pass/phase-syntax-link-does-resolve.rs`. I doubt that failure is related, though.
Fixes#10010.
This is just an initial implementation and does not yet fully replace `~[T]`. A generic initialization syntax for containers is missing, and the slice functionality needs to be reworked to make auto-slicing unnecessary.
Traits for supporting indexing properly are also required. This also needs to be fixed to make ring buffers as easy to use as vectors.
The tests and documentation for `~[T]` can be ported over to this type when it is removed. I don't really expect DST to happen for vectors as having both `~[T]` and `Vec<T>` is overcomplicated and changing the slice representation to 3 words is not at all appealing. Unlike with traits, it's possible (and easy) to implement `RcSlice<T>` and `GcSlice<T>` without compiler help.
Native timers are a much hairier thing to deal with than green timers due to the
interface that we would like to expose (both a blocking sleep() and a
channel-based interface). I ended up implementing timers in three different ways
for the various platforms that we supports.
In all three of the implementations, there is a worker thread which does send()s
on channels for timers. This worker thread is initialized once and then
communicated to in a platform-specific manner, but there's always a shared
channel available for sending messages to the worker thread.
* Windows - I decided to use windows kernel timer objects via
CreateWaitableTimer and SetWaitableTimer in order to provide sleeping
capabilities. The worker thread blocks via WaitForMultipleObjects where one of
the objects is an event that is used to wake up the helper thread (which then
drains the incoming message channel for requests).
* Linux/(Android?) - These have the ideal interface for implementing timers,
timerfd_create. Each timer corresponds to a timerfd, and the helper thread
uses epoll to wait for all active timers and then send() for the next one that
wakes up. The tricky part in this implementation is updating a timerfd, but
see the implementation for the fun details
* OSX/FreeBSD - These obviously don't have the windows APIs, and sadly don't
have the timerfd api available to them, so I have thrown together a solution
which uses select() plus a timeout in order to ad-hoc-ly implement a timer
solution for threads. The implementation is backed by a sorted array of timers
which need to fire. As I said, this is an ad-hoc solution which is certainly
not accurate timing-wise. I have done this implementation due to the lack of
other primitives to provide an implementation, and I've done it the best that
I could, but I'm sure that there's room for improvement.
I'm pretty happy with how these implementations turned out. In theory we could
drop the timerfd implementation and have linux use the select() + timeout
implementation, but it's so inaccurate that I would much rather continue to use
timerfd rather than my ad-hoc select() implementation.
The only change that I would make to the API in general is to have a generic
sleep() method on an IoFactory which doesn't require allocating a Timer object.
For everything but windows it's super-cheap to request a blocking sleep for a
set amount of time, and it's probably worth it to provide a sleep() which
doesn't do something like allocate a file descriptor on linux.
This routine is currently only used to clean up the timer helper thread in the
libnative implementation, but there are possibly other uses for this.
The documentation is clear that the procedures are *not* run with any task
context and hence have very little available to them. I also opted to disallow
at_exit inside of at_exit and just abort the process at that point.
The `malloc` family of functions may return a null pointer for a
zero-size allocation, which should not be interpreted as an
out-of-memory error.
If the implementation does not return a null pointer, then handling
this will result in memory savings for zero-size types.
This also switches some code to `malloc_raw` in order to maintain a
centralized point for handling out-of-memory in `rt::global_heap`.
Closes#11634
The `malloc` family of functions may return a null pointer for a
zero-size allocation, which should not be interpreted as an
out-of-memory error.
If the implementation does not return a null pointer, then handling
this will result in memory savings for zero-size types.
This also switches some code to `malloc_raw` in order to maintain a
centralized point for handling out-of-memory in `rt::global_heap`.
Closes#11634
Major changes:
- Define temporary scopes in a syntax-based way that basically defaults
to the innermost statement or conditional block, except for in
a `let` initializer, where we default to the innermost block. Rules
are documented in the code, but not in the manual (yet).
See new test run-pass/cleanup-value-scopes.rs for examples.
- Refactors Datum to better define cleanup roles.
- Refactor cleanup scopes to not be tied to basic blocks, permitting
us to have a very large number of scopes (one per AST node).
- Introduce nascent documentation in trans/doc.rs covering datums and
cleanup in a more comprehensive way.
r? @pcwalton
Major changes:
- Define temporary scopes in a syntax-based way that basically defaults
to the innermost statement or conditional block, except for in
a `let` initializer, where we default to the innermost block. Rules
are documented in the code, but not in the manual (yet).
See new test run-pass/cleanup-value-scopes.rs for examples.
- Refactors Datum to better define cleanup roles.
- Refactor cleanup scopes to not be tied to basic blocks, permitting
us to have a very large number of scopes (one per AST node).
- Introduce nascent documentation in trans/doc.rs covering datums and
cleanup in a more comprehensive way.
The failure functions are generic, meaning they're candidates for getting
inlined across crates. This has been happening, leading to monstrosities like
that found in #11549. I have verified that the codegen is *much* better now that
we're not inlining the failure path (the slow path).
The failure functions are generic, meaning they're candidates for getting
inlined across crates. This has been happening, leading to monstrosities like
that found in #11549. I have verified that the codegen is *much* better now that
we're not inlining the failure path (the slow path).
This will allow capturing of common things like logging messages, stdout prints
(using stdio println), and failure messages (printed to stderr). Any new prints
added to libstd should be funneled through these task handles to allow capture
as well.
Additionally, this commit redirects logging back through a `Logger` trait so the
log level can be usefully consumed by an arbitrary logger.
This commit also introduces methods to set the task-local stdout handles:
* std::io::stdio::set_stdout
* std::io::stdio::set_stderr
* std::io::logging::set_logger
These methods all return the previous logger just in case it needs to be used
for inspection.
I plan on using this infrastructure for extra::test soon, but we don't quite
have the primitives that I'd like to use for it, so it doesn't migrate
extra::test at this time.
Closes#6369
This will allow capturing of common things like logging messages, stdout prints
(using stdio println), and failure messages (printed to stderr). Any new prints
added to libstd should be funneled through these task handles to allow capture
as well.
Additionally, this commit redirects logging back through a `Logger` trait so the
log level can be usefully consumed by an arbitrary logger.
This commit also introduces methods to set the task-local stdout handles:
* std::io::stdio::set_stdout
* std::io::stdio::set_stderr
* std::io::logging::set_logger
These methods all return the previous logger just in case it needs to be used
for inspection.
I plan on using this infrastructure for extra::test soon, but we don't quite
have the primitives that I'd like to use for it, so it doesn't migrate
extra::test at this time.
Closes#6369
This reverts commit f1b5f59287.
Using a private function of a library is a bad idea: several people (on
Linux) were meeting with linking errors because of it (different/older
versions of glibc).
This removes the feature where newtype structs can be dereferenced like pointers, and likewise where certain enums can be dereferenced (which I imagine nobody realized still existed). This ad-hoc behavior is to be replaced by a more general overloadable dereference trait in the future.
I've been nursing this patch for two months and think it's about rebased up to master.
@nikomatsakis this makes a bunch of your type checking code noticeably uglier.
If there is a lot of data in thread-local storage some implementations
of pthreads (e.g. glibc) fail if you don't request a stack large enough
-- by adjusting for the minimum size we guarantee that our stacks are
always large enough. Issue #6233.
Previously this was an `rtabort!`, indicating a runtime bug. Promote
this to a more intentional abort and print a (slightly) more
informative error message.
Can't test this sense our test suite can't handle an abort exit.
I consider this to close#910, and that we should open another issue about implementing less conservative semantics here.
If there is a lot of data in thread-local storage some implementations
of pthreads (e.g. glibc) fail if you don't request a stack large enough
-- by adjusting for the minimum size we guarantee that our stacks are
always large enough. Issue #6233.
Previously this was an rtabort!, indicating a runtime bug. Promote
this to a more intentional abort and print a (slightly) more
informative error message.
Can't test this sense our test suite can't handle an abort exit.
For libgreen, bookeeping should not be global but rather on a per-pool basis.
Inside libnative, it's known that there must be a global counter with a
mutex/cvar.
The benefit of taking this strategy is to remove this functionality from libstd
to allow fine-grained control of it through libnative/libgreen. Notably, helper
threads in libnative can manually decrement the global count so they don't count
towards the global count of threads. Also, the shutdown process of *all* sched
pools is now dependent on the number of tasks in the pool being 0 rather than
this only being a hardcoded solution for the initial sched pool in libgreen.
This involved adding a Local::try_take() method on the Local trait in order for
the channel wakeup to work inside of libgreen. The channel send was happening
from a SchedTask when there is no Task available in TLS, and now this is
possible to work (remote wakeups are always possible, just a little slower).
For libgreen, bookeeping should not be global but rather on a per-pool basis.
Inside libnative, it's known that there must be a global counter with a
mutex/cvar.
The benefit of taking this strategy is to remove this functionality from libstd
to allow fine-grained control of it through libnative/libgreen. Notably, helper
threads in libnative can manually decrement the global count so they don't count
towards the global count of threads. Also, the shutdown process of *all* sched
pools is now dependent on the number of tasks in the pool being 0 rather than
this only being a hardcoded solution for the initial sched pool in libgreen.
This involved adding a Local::try_take() method on the Local trait in order for
the channel wakeup to work inside of libgreen. The channel send was happening
from a SchedTask when there is no Task available in TLS, and now this is
possible to work (remote wakeups are always possible, just a little slower).
This commit uniforms the short title of modules provided by libstd,
in order to make their roles more explicit when glancing at the index.
Signed-off-by: Luca Bruno <lucab@debian.org>
* vec::raw::to_ptr is gone
* Pausible => Pausable
* Removing @
* Calling the main task "<main>"
* Removing unused imports
* Removing unused mut
* Bringing some libextra tests up to date
* Allowing compiletest to work at stage0
* Fixing the bootstrap-from-c rmake tests
* assert => rtassert in a few cases
* printing to stderr instead of stdout in fail!()
There was a race in the code previously where schedulers would *immediately*
shut down after spawning the main task (because the global task count would
still be 0). This fixes the logic by blocking the sched pool task in receving on
a port instead of spawning a task into the pool to receive on a port.
The modifications necessary were to have a "simple task" running by the time the
code is executing, but this is a simple enough thing to implement and I forsee
this being necessary to have implemented in the future anyway.
Note that this removes a number of run-pass tests which are exercising behavior
of the old runtime. This functionality no longer exists and is thoroughly tested
inside of libgreen and libnative. There isn't really the notion of "starting the
runtime" any more. The major notion now is "bootstrapping the initial task".
This allows creation of different sched pools with different io factories.
Namely, this will be used to test the basic I/O loop in the green crate. This
can also be used to override the global default.
The scheduler pool now has a much more simplified interface. There is now a
clear distinction between creating the pool and then interacting the pool. When
a pool is created, all schedulers are not active, and only later if a spawn is
done does activity occur.
There are four operations that you can do on a pool:
1. Create a new pool. The only argument to this function is the configuration
for the scheduler pool. Currently the only configuration parameter is the
number of threads to initially spawn.
2. Spawn a task into this pool. This takes a procedure and task configuration
options and spawns a new task into the pool of schedulers.
3. Spawn a new scheduler into the pool. This will return a handle on which to
communicate with the scheduler in order to do something like a pinned task.
4. Shut down the scheduler pool. This will consume the scheduler pool, request
all of the schedulers to shut down, and then wait on all the scheduler
threads. Currently this will block the invoking OS thread, but I plan on
making 'Thread::join' not a thread-blocking call.
These operations can be used to encode all current usage of M:N schedulers, as
well as providing a simple interface through which a pool can be modified. There
is currently no way to remove a scheduler from a pool of scheduler, as there's
no way to guarantee that a scheduler has exited. This may be added in the
future, however (as necessary).
In the compiled version of local_ptr (that with #[thread_local]), the take()
funciton didn't zero-out the previous pointer, allowing for multiple takes (with
fewer runtime assertions being tripped).
This extracts everything related to green scheduling from libstd and introduces
a new libgreen crate. This mostly involves deleting most of std::rt and moving
it to libgreen.
Along with the movement of code, this commit rearchitects many functions in the
scheduler in order to adapt to the fact that Local::take now *only* works on a
Task, not a scheduler. This mostly just involved threading the current green
task through in a few locations, but there were one or two spots where things
got hairy.
There are a few repercussions of this commit:
* tube/rc have been removed (the runtime implementation of rc)
* There is no longer a "single threaded" spawning mode for tasks. This is now
encompassed by 1:1 scheduling + communication. Convenience methods have been
introduced that are specific to libgreen to assist in the spawning of pools of
schedulers.
Printing is an incredibly useful debugging utility, and it's not much help if
your debugging prints just trigger an obscure abort when you need them most. In
order to handle this case, forcibly fall back to a libc::write implementation of
printing whenever a local task is not available.
Note that this is *not* a 1:1 fallback. All 1:1 rust tasks will still have a
local Task that it can go through (and stdio will be created through the local
IO factory), this is only a fallback for "no context" rust code (such as that
setting up the context).
It is not the case that all programs will always be able to acquire an instance
of the LocalIo borrow, so this commit exposes this limitation by returning
Option<LocalIo> from LocalIo::borrow().
At the same time, a helper method LocalIo::maybe_raise() has been added in order
to encapsulate the functionality of raising on io_error if there is on local I/O
available.
For now, this moves the following modules to std::sync
* UnsafeArc (also removed unwrap method)
* mpsc_queue
* spsc_queue
* atomics
* mpmc_bounded_queue
* deque
We may want to remove some of the queues, but for now this moves things out of
std::rt into std::sync
This module contains many M:N specific concepts. This will no longer be
available with libgreen, and most functions aren't really that necessary today
anyway. New testing primitives will be introduced as they become available for
1:1 and M:N.
A new io::test module is introduced with the new ip4/ip6 address helpers to
continue usage in io tests.
This trait is used to abstract the differences between 1:1 and M:N scheduling
and is the sole dispatch point for the differences between these two scheduling
modes.
This, and the following series of commits, is not intended to compile. Only
after the entire transition is complete are programs expected to compile.
For `str.as_mut_buf`, un-closure-ification is achieved by outright removal (see commit message). The others are replaced by `.as_ptr`, `.as_mut_ptr` and `.len`
This code in resolve accidentally forced all types with an impl to become
public. This fixes it by default inheriting the privacy of what was previously
there and then becoming `true` if nothing else exits.
Closes#10545
* Streams are now ~3x faster than before (fewer allocations and more optimized)
* Based on a single-producer single-consumer lock-free queue that doesn't
always have to allocate on every send.
* Blocking via mutexes/cond vars outside the runtime
* Streams work in/out of the runtime seamlessly
* Select now works in/out of the runtime seamlessly
* Streams will now fail!() on send() if the other end has hung up
* try_send() will not fail
* PortOne/ChanOne removed
* SharedPort removed
* MegaPipe removed
* Generic select removed (only one kind of port now)
* API redesign
* try_recv == never block
* recv_opt == block, don't fail
* iter() == Iterator<T> for Port<T>
* removed peek
* Type::new
* Removed rt::comm
This replaces the link meta attributes with a pkgid attribute and uses a hash
of this as the crate hash. This makes the crate hash computable by things
other than the Rust compiler. It also switches the hash function ot SHA1 since
that is much more likely to be available in shell, Python, etc than SipHash.
Fixes#10188, #8523.
