Instead, use an enum to allow running both a procedure and sending the task
result over a channel. I expect the common case to be sending on a channel (e.g.
task::try), so don't require an extra allocation in the common case.
cc #11389
This, the Nth rewrite of channels, is not a rewrite of the core logic behind
channels, but rather their API usage. In the past, we had the distinction
between oneshot, stream, and shared channels, but the most recent rewrite
dropped oneshots in favor of streams and shared channels.
This distinction of stream vs shared has shown that it's not quite what we'd
like either, and this moves the `std::comm` module in the direction of "one
channel to rule them all". There now remains only one Chan and one Port.
This new channel is actually a hybrid oneshot/stream/shared channel under the
hood in order to optimize for the use cases in question. Additionally, this also
reduces the cognitive burden of having to choose between a Chan or a SharedChan
in an API.
My simple benchmarks show no reduction in efficiency over the existing channels
today, and a 3x improvement in the oneshot case. I sadly don't have a
pre-last-rewrite compiler to test out the old old oneshots, but I would imagine
that the performance is comparable, but slightly slower (due to atomic reference
counting).
This commit also brings the bonus bugfix to channels that the pending queue of
messages are all dropped when a Port disappears rather then when both the Port
and the Chan disappear.
Declare a `type SendStr = MaybeOwned<'static>` to ease readibility of
types that needed the old SendStr behavior.
Implement all the traits for MaybeOwned that SendStr used to implement.
This is part of the overall strategy I would like to take when approaching
issue #11165. The only two I/O objects that reasonably want to be "split" are
the network stream objects. Everything else can be "split" by just creating
another version.
The initial idea I had was the literally split the object into a reader and a
writer half, but that would just introduce lots of clutter with extra interfaces
that were a little unnnecssary, or it would return a ~Reader and a ~Writer which
means you couldn't access things like the remote peer name or local socket name.
The solution I found to be nicer was to just clone the stream itself. The clone
is just a clone of the handle, nothing fancy going on at the kernel level.
Conceptually I found this very easy to wrap my head around (everything else
supports clone()), and it solved the "split" problem at the same time.
The cloning support is pretty specific per platform/lib combination:
* native/win32 - uses some specific WSA apis to clone the SOCKET handle
* native/unix - uses dup() to get another file descriptor
* green/all - This is where things get interesting. When we support full clones
of a handle, this implies that we're allowing simultaneous writes
and reads to happen. It turns out that libuv doesn't support two
simultaneous reads or writes of the same object. It does support
*one* read and *one* write at the same time, however. Some extra
infrastructure was added to just block concurrent writers/readers
until the previous read/write operation was completed.
I've added tests to the tcp/unix modules to make sure that this functionality is
supported everywhere.
Make the definition of epoll_event use natural alignment on all
architectures except x86_64.
Before this commit, the struct was always 12 bytes big, which works okay
on x86 and x86_64 but not on ARM and MIPS, where it should be 16 bytes
big with the `data` field aligned on an 8 byte boundary.
`Times::times` was always a second-class loop because it did not support the `break` and `continue` operations. Its playful appeal was then lost after `do` was disabled for closures. It's time to let this one go.
The general consensus is that we want to move away from conditions for I/O, and I propose a two-step plan for doing so:
1. Warn about unused `Result` types. When all of I/O returns `Result`, it will require you inspect the return value for an error *only if* you have a result you want to look at. By default, for things like `write` returning `Result<(), Error>`, these will all go silently ignored. This lint will prevent blind ignorance of these return values, letting you know that there's something you should do about them.
2. Implement a `try!` macro:
```
macro_rules! try( ($e:expr) => (match $e { Ok(e) => e, Err(e) => return Err(e) }) )
```
With these two tools combined, I feel that we get almost all the benefits of conditions. The first step (the lint) is a sanity check that you're not ignoring return values at callsites. The second step is to provide a convenience method of returning early out of a sequence of computations. After thinking about this for awhile, I don't think that we need the so-called "do-notation" in the compiler itself because I think it's just *too* specialized. Additionally, the `try!` macro is super lightweight, easy to understand, and works almost everywhere. As soon as you want to do something more fancy, my answer is "use match".
Basically, with these two tools in action, I would be comfortable removing conditions. What do others think about this strategy?
----
This PR specifically implements the `unused_result` lint. I actually added two lints, `unused_result` and `unused_must_use`, and the first commit has the rationale for why `unused_result` is turned off by default.
These are either returned from public functions, and really should
appear in the documentation, but don't since they're private, or are
implementation details that are currently public.
Renamed the ```invert()``` function in ```iter.rs``` to ```flip()```, from #10632
Also renamed the ```Invert<T>``` type to ```Flip<T>```.
Some related code comments changed. Documentation that I could find has
been updated, and all the instances I could locate where the
function/type were called have been updated as well.
This is my first contribution to Rust! Apologies in advance if I've snarfed the
PR process, I'm not used to rebase.
I initially had issues with the ```codegen``` section of the tests failing, however
the ```make check``` process is not reporting any failures at this time. I think
that was a local env issue more than me facerolling my changes. :)
Renamed the invert() function in iter.rs to flip().
Also renamed the Invert<T> type to Flip<T>.
Some related code comments changed. Documentation that I could find has
been updated, and all the instances I could locate where the
function/type were called have been updated as well.
This is just an initial implementation and does not yet fully replace `~[T]`. A generic initialization syntax for containers is missing, and the slice functionality needs to be reworked to make auto-slicing unnecessary.
Traits for supporting indexing properly are also required. This also needs to be fixed to make ring buffers as easy to use as vectors.
The tests and documentation for `~[T]` can be ported over to this type when it is removed. I don't really expect DST to happen for vectors as having both `~[T]` and `Vec<T>` is overcomplicated and changing the slice representation to 3 words is not at all appealing. Unlike with traits, it's possible (and easy) to implement `RcSlice<T>` and `GcSlice<T>` without compiler help.
Native timers are a much hairier thing to deal with than green timers due to the
interface that we would like to expose (both a blocking sleep() and a
channel-based interface). I ended up implementing timers in three different ways
for the various platforms that we supports.
In all three of the implementations, there is a worker thread which does send()s
on channels for timers. This worker thread is initialized once and then
communicated to in a platform-specific manner, but there's always a shared
channel available for sending messages to the worker thread.
* Windows - I decided to use windows kernel timer objects via
CreateWaitableTimer and SetWaitableTimer in order to provide sleeping
capabilities. The worker thread blocks via WaitForMultipleObjects where one of
the objects is an event that is used to wake up the helper thread (which then
drains the incoming message channel for requests).
* Linux/(Android?) - These have the ideal interface for implementing timers,
timerfd_create. Each timer corresponds to a timerfd, and the helper thread
uses epoll to wait for all active timers and then send() for the next one that
wakes up. The tricky part in this implementation is updating a timerfd, but
see the implementation for the fun details
* OSX/FreeBSD - These obviously don't have the windows APIs, and sadly don't
have the timerfd api available to them, so I have thrown together a solution
which uses select() plus a timeout in order to ad-hoc-ly implement a timer
solution for threads. The implementation is backed by a sorted array of timers
which need to fire. As I said, this is an ad-hoc solution which is certainly
not accurate timing-wise. I have done this implementation due to the lack of
other primitives to provide an implementation, and I've done it the best that
I could, but I'm sure that there's room for improvement.
I'm pretty happy with how these implementations turned out. In theory we could
drop the timerfd implementation and have linux use the select() + timeout
implementation, but it's so inaccurate that I would much rather continue to use
timerfd rather than my ad-hoc select() implementation.
The only change that I would make to the API in general is to have a generic
sleep() method on an IoFactory which doesn't require allocating a Timer object.
For everything but windows it's super-cheap to request a blocking sleep for a
set amount of time, and it's probably worth it to provide a sleep() which
doesn't do something like allocate a file descriptor on linux.
The `malloc` family of functions may return a null pointer for a
zero-size allocation, which should not be interpreted as an
out-of-memory error.
If the implementation does not return a null pointer, then handling
this will result in memory savings for zero-size types.
This also switches some code to `malloc_raw` in order to maintain a
centralized point for handling out-of-memory in `rt::global_heap`.
Closes#11634
This will allow capturing of common things like logging messages, stdout prints
(using stdio println), and failure messages (printed to stderr). Any new prints
added to libstd should be funneled through these task handles to allow capture
as well.
Additionally, this commit redirects logging back through a `Logger` trait so the
log level can be usefully consumed by an arbitrary logger.
This commit also introduces methods to set the task-local stdout handles:
* std::io::stdio::set_stdout
* std::io::stdio::set_stderr
* std::io::logging::set_logger
These methods all return the previous logger just in case it needs to be used
for inspection.
I plan on using this infrastructure for extra::test soon, but we don't quite
have the primitives that I'd like to use for it, so it doesn't migrate
extra::test at this time.
Closes#6369
Similarly to the recent commit to do this for networking, there's no reason that
a read on a file descriptor should continue reading until the entire buffer is
full. This makes sense when dealing with literal files, but when dealing with
things like stdin this doesn't make sense.
This will allow capturing of common things like logging messages, stdout prints
(using stdio println), and failure messages (printed to stderr). Any new prints
added to libstd should be funneled through these task handles to allow capture
as well.
Additionally, this commit redirects logging back through a `Logger` trait so the
log level can be usefully consumed by an arbitrary logger.
This commit also introduces methods to set the task-local stdout handles:
* std::io::stdio::set_stdout
* std::io::stdio::set_stderr
* std::io::logging::set_logger
These methods all return the previous logger just in case it needs to be used
for inspection.
I plan on using this infrastructure for extra::test soon, but we don't quite
have the primitives that I'd like to use for it, so it doesn't migrate
extra::test at this time.
Closes#6369
libnative erroneously would attempt to fill the entire buffer in a call to
`read` before returning, when rather it should return immediately because
there's not guaranteed to be any data that will ever be received again.
Close#11328
There was a scheduling race where a child may not increment the global task
count before the parent exits, and the parent would then think that there are no
more tasks left.
Closes#11039
libnative erroneously would attempt to fill the entire buffer in a call to
`read` before returning, when rather it should return immediately because
there's not guaranteed to be any data that will ever be received again.
Close#11328
There was a scheduling race where a child may not increment the global task
count before the parent exits, and the parent would then think that there are no
more tasks left.
If the main closure failed, then the `exit_code` variable would still be `None`,
and the `unwrap()` was failing (triggering a process abort). This changes the
`unwrap()` to an `unwrap_or()` in order to prevent process abort and detect when
the native task failed.
This patchset adds intrinsics similar to the to_[be|le][16|32|64] intrinsics but for going in the reverse direction, e.g. from big/little endian to host endian. Implementation wise they do exactly the same as the corresponding to_* functions but I think it anyway make sense to have them since using the to_* functions in the reverse direction is not entirely intuitive.
The first patch adds the intrinsics and the two following changes instances of bswap* to use the [to|from]_* intrinsics instead.
For libgreen, bookeeping should not be global but rather on a per-pool basis.
Inside libnative, it's known that there must be a global counter with a
mutex/cvar.
The benefit of taking this strategy is to remove this functionality from libstd
to allow fine-grained control of it through libnative/libgreen. Notably, helper
threads in libnative can manually decrement the global count so they don't count
towards the global count of threads. Also, the shutdown process of *all* sched
pools is now dependent on the number of tasks in the pool being 0 rather than
this only being a hardcoded solution for the initial sched pool in libgreen.
This involved adding a Local::try_take() method on the Local trait in order for
the channel wakeup to work inside of libgreen. The channel send was happening
from a SchedTask when there is no Task available in TLS, and now this is
possible to work (remote wakeups are always possible, just a little slower).
For libgreen, bookeeping should not be global but rather on a per-pool basis.
Inside libnative, it's known that there must be a global counter with a
mutex/cvar.
The benefit of taking this strategy is to remove this functionality from libstd
to allow fine-grained control of it through libnative/libgreen. Notably, helper
threads in libnative can manually decrement the global count so they don't count
towards the global count of threads. Also, the shutdown process of *all* sched
pools is now dependent on the number of tasks in the pool being 0 rather than
this only being a hardcoded solution for the initial sched pool in libgreen.
This involved adding a Local::try_take() method on the Local trait in order for
the channel wakeup to work inside of libgreen. The channel send was happening
from a SchedTask when there is no Task available in TLS, and now this is
possible to work (remote wakeups are always possible, just a little slower).
Move the tests into libstd, use the `iotest!` macro to test both native and uv
bindings, and use the cloexec trick to figure out when the child process fails
in exec.
* vec::raw::to_ptr is gone
* Pausible => Pausable
* Removing @
* Calling the main task "<main>"
* Removing unused imports
* Removing unused mut
* Bringing some libextra tests up to date
* Allowing compiletest to work at stage0
* Fixing the bootstrap-from-c rmake tests
* assert => rtassert in a few cases
* printing to stderr instead of stdout in fail!()
This is a very real problem with cvars on normal systems, and all of channels
will not work if spurious wakeups are accepted. This problem is just solved with
a synchronized flag (accessed in the cvar's lock) to see whether a signal()
actually happened or whether it's spurious.
There was a race in the code previously where schedulers would *immediately*
shut down after spawning the main task (because the global task count would
still be 0). This fixes the logic by blocking the sched pool task in receving on
a port instead of spawning a task into the pool to receive on a port.
The modifications necessary were to have a "simple task" running by the time the
code is executing, but this is a simple enough thing to implement and I forsee
this being necessary to have implemented in the future anyway.
Note that this removes a number of run-pass tests which are exercising behavior
of the old runtime. This functionality no longer exists and is thoroughly tested
inside of libgreen and libnative. There isn't really the notion of "starting the
runtime" any more. The major notion now is "bootstrapping the initial task".
The scheduler pool now has a much more simplified interface. There is now a
clear distinction between creating the pool and then interacting the pool. When
a pool is created, all schedulers are not active, and only later if a spawn is
done does activity occur.
There are four operations that you can do on a pool:
1. Create a new pool. The only argument to this function is the configuration
for the scheduler pool. Currently the only configuration parameter is the
number of threads to initially spawn.
2. Spawn a task into this pool. This takes a procedure and task configuration
options and spawns a new task into the pool of schedulers.
3. Spawn a new scheduler into the pool. This will return a handle on which to
communicate with the scheduler in order to do something like a pinned task.
4. Shut down the scheduler pool. This will consume the scheduler pool, request
all of the schedulers to shut down, and then wait on all the scheduler
threads. Currently this will block the invoking OS thread, but I plan on
making 'Thread::join' not a thread-blocking call.
These operations can be used to encode all current usage of M:N schedulers, as
well as providing a simple interface through which a pool can be modified. There
is currently no way to remove a scheduler from a pool of scheduler, as there's
no way to guarantee that a scheduler has exited. This may be added in the
future, however (as necessary).
This adds a few smoke tests associated with libnative tasks (not much code to
test here anyway), and cleans up the entry points a little bit to be a little
more like libgreen.
The I/O code doesn't need much testing because that's all tested in libstd (with
the iotest! macro).
This commit introduces a new crate called "native" which will be the crate that
implements the 1:1 runtime of rust. This currently entails having an
implementation of std::rt::Runtime inside of libnative as well as moving all of
the native I/O implementations to libnative.
The current snag is that the start lang item must currently be defined in
libnative in order to start running, but this will change in the future.
Cool fact about this crate, there are no extra features that are enabled.
Note that this commit does not include any makefile support necessary for
building libnative, that's all coming in a later commit.
