This resolves issue #908.
Notable changes:
- On Windows, LLVM integrated assembler emits bad stack unwind tables when segmented stacks are enabled. However, unwind info directives in the assembly output are correct, so we generate assembly first and then run it through an external assembler, just like it is already done for Android builds.
- Linker is invoked via "g++" command instead of "gcc": g++ passes the appropriate magic parameters to the linker, which ensure correct registration of stack unwind tables in dynamic libraries.
- change all uses of Path in fn args to &P
- FileStream.read assumptions were wrong (libuv file io is non-positional)
- the above will mean that we "own" Seek impl info .. should probably
push it in UvFileDescriptor..
- needs more tests
Implement CharIterator as a separate struct, so that it can be .clone()'d. Fix `.char_range_at_reverse` so that it performs better, closer to the forwards version. This makes the reverse iterators and users like `.rfind()` perform better.
Before
test str::bench::char_iterator ... bench: 146 ns/iter (+/- 0)
test str::bench::char_iterator_ascii ... bench: 397 ns/iter (+/- 49)
test str::bench::char_iterator_rev ... bench: 576 ns/iter (+/- 8)
test str::bench::char_offset_iterator ... bench: 128 ns/iter (+/- 2)
test str::bench::char_offset_iterator_rev ... bench: 425 ns/iter (+/- 59)
After
test str::bench::char_iterator ... bench: 130 ns/iter (+/- 1)
test str::bench::char_iterator_ascii ... bench: 307 ns/iter (+/- 5)
test str::bench::char_iterator_rev ... bench: 185 ns/iter (+/- 8)
test str::bench::char_offset_iterator ... bench: 131 ns/iter (+/- 13)
test str::bench::char_offset_iterator_rev ... bench: 183 ns/iter (+/- 2)
To be able to use a string slice to represent the CharIterator, a function `slice_unchecked` is added, that does the same as `slice_bytes` but without any boundary checks.
It would be possible to implement CharIterator with pointer arithmetic to make it *much more efficient*, but since vec iterator is still improving, it's too early to attempt to re-implement it in other places. Hopefully CharIterator can be implemented on top of vec iterator without any unsafe code later.
Additional changes fix the documentation about null termination.
This adds support for performing Unicode Normalization Forms D and KD on strings.
To enable this the decomposition and canonical combining class properties are added to std::unicode.
On my system this increases libstd's size by ~250KiB.
Linux and Android share the kernel, but not the C library, so sysconf constants are different. For example, _SC_PAGESIZE is 30 on Linux, but 39 on Android.
This patch
* splits sysconf constants to sysconf module
* merges non-MIPS and MIPS sysconf constants (they are same)
* adds Android sysconf constants
This patch also lets mmap tests to pass on Android.
Fixed a memory leak caused by the singleton idle callback failing to close correctly. The problem was that the close function requires running inside a callback in the event loop, but we were trying to close the idle watcher after the loop returned from run. The fix was to just call run again to process this callback. There is an additional tweak to move the initialization logic fully into bootstrap, so tasks that do not ever call run do not have problems destructing.
libuv handles are tied to the event loop that created them. In order to perform IO, the handle must be on the thread with its home event loop. Thus, when as task wants to do IO it must first go to the IO handle's home event loop and pin itself to the corresponding scheduler while the IO action is in flight. Once the IO action completes, the task is unpinned and either returns to its home scheduler if it is a pinned task, or otherwise stays on the current scheduler.
Making new blocking IO implementations (i.e. files) thread safe is rather simple. Add a home field to the IO handle's struct in uvio and implement the HomingIO trait. Wrap every IO call in the HomingIO.home_for_io method, which will take care of the scheduling.
I'm not sure if this remains thread safe in the presence of asynchronous IO at the libuv level. If we decide to do that, then this set up should be revisited.
Instead of a furious storm of idle callbacks we just have one. This is a major performance gain - around 40% on my machine for the ping pong bench.
Also in this PR is a cleanup commit for the scheduler code. Was previously up as a separate PR, but bors load + imminent merge hell led me to roll them together. Was #8549.
Each IO handle has a home event loop, which created it.
When a task wants to use an IO handle, it must first make sure it is on that home event loop.
It uses the scheduler handle in the IO handle to send itself there before starting the IO action.
Once the IO action completes, the task restores its previous home state.
If it is an AnySched task, then it will be executed on the new scheduler.
If it has a normal home, then it will return there before executing any more code after the IO action.
Long-standing branch to remove foreign function wrappers altogether. Calls to C functions are done "in place" with no stack manipulation; the scheme relies entirely on the correct use of `#[fixed_stack_segment]` to guarantee adequate stack space. A linter is added to detect when `#[fixed_stack_segment]` annotations are missing. An `externfn!` macro is added to make it easier to declare foreign fns and wrappers in one go: this macro may need some refinement, though, for example it might be good to be able to declare a group of foreign fns. I leave that for future work (hopefully somebody else's work :) ).
Fixes#3678.
Let CharIterator be a separate type from CharOffsetIterator (so that
CharIterator can be cloned, for example).
Implement CharOffsetIterator by using the same technique as the method
subslice_offset.
Add a function like raw::slice_bytes, but it doesn't check slice
boundaries. For iterator use where we always know the begin, end indices
are in range.
See discussion in #8489, but this selects option 3 by adding a `Default` trait to be implemented by various basic types.
Once this makes it into a snapshot I think it's about time to start overhauling all current use-cases of `fmt!` to move towards `ifmt!`. The goal is to replace `%X` with `{}` in 90% of situations, and this commit should enable that.
Add size_hint() to a few Iterators that were missing it.
Update a couple of existing size_hint()s to use checked_add() instead of
saturating_add() for the upper bound.
@brson grilled me about how this bugfix worked the first time around, and it occurred to me that it didn't in the case where the task is unwinding. Now it will.
Address issue #5257, for example these values all had the same hash value:
("aaa", "bbb", "ccc")
("aaab", "bb", "ccc")
("aaabbb", "", "ccc")
IterBytes for &[A] now includes the length, before calling iter_bytes on
each element.
IterBytes for &str is now terminated by a byte that does not appear in
UTF-8. This way only one more byte is processed when hashing strings.
Address issue #5257, for example these values all had the same hash value:
("aaa", "bbb", "ccc")
("aaab", "bb", "ccc")
("aaabbb", "", "ccc")
IterBytes for &[A] now includes the length, before calling iter_bytes on
each element.
IterBytes for &str is now terminated by a byte that does not appear in
UTF-8. This way only one more byte is processed when hashing strings.
I need `Clone` for `Tm` for my latest work on [rust-http](https://github.com/chris-morgan/rust-http) (static typing for headers, and headers like `Date` are a time), so here it is.
@huonw recommended deriving DeepClone while I was at it.
I also had to implement `DeepClone` for `~str` to get a derived implementation of `DeepClone` for `Tm`; I did `@str` while I was at it, for consistency.
This allows the internal implementation details of the TLS keys to be
changed without requiring the update of all the users. (Or, applying
changes that *have* to be applied for the keys to work correctly, e.g.
forcing LLVM to not merge these constants.)
Retry of PR #8471
Replace the remaining functions marked for issue #8228 with similar functions that are iterator-based.
Change `either::{lefts, rights}` to be iterator-filtering instead of returning a vector.
Replace `map_vec`, `map_vec2`, `iter_vec2` in std::result with three functions:
* `result::collect` gathers `Iterator<Result<V, U>>` to `Result<~[V], U>`
* `result::fold` folds `Iterator<Result<T, E>>` to `Result<V, E>`
* `result::fold_` folds `Iterator<Result<T, E>>` to `Result<(), E>`
This allows the internal implementation details of the TLS keys to be
changed without requiring the update of all the users. (Or, applying
changes that have to be applied for the keys to work correctly, e.g.
forcing LLVM to not merge these constants.)
Fixes#7722
I had a couple of queries:
- Should this return an array or an iterator?
- Should this be a method on iterators or on the rng? I implemented it in RngUtils as it seemed to belong with shuffle().
If they are on the trait then it is extremely annoying to use them as
generic parameters to a function, e.g. with the iterator param on the trait
itself, if one was to pass an Extendable<int> to a function that filled it
either from a Range or a Map<VecIterator>, one needs to write something
like:
fn foo<E: Extendable<int, Range<int>> +
Extendable<int, Map<&'self int, int, VecIterator<int>>>
(e: &mut E, ...) { ... }
since using a generic, i.e. `foo<E: Extendable<int, I>, I: Iterator<int>>`
means that `foo` takes 2 type parameters, and the caller has to specify them
(which doesn't work anyway, as they'll mismatch with the iterators used in
`foo` itself).
This patch changes it to:
fn foo<E: Extendable<int>>(e: &mut E, ...) { ... }
.with_c_str() is a replacement for the old .as_c_str(), to avoid
unnecessary boilerplate.
Replace all usages of .to_c_str().with_ref() with .with_c_str().
Previous dicussions about CString suggested that interior nulls should
throw an error. This was never implemented. Add this now, using a
condition (named null_byte) to allow for recovery.
Add method .to_c_str_unchecked() that skips this check.
Replace these with three functions based on iterators: collect, fold,
and fold_. The mapping part is replaced by iterator .map(), so the part
that these functions do is to accumulate the final Result<,> value.
* `result::collect` gathers `Iterator<Result<V, U>>` to `Result<~[V], U>`
* `result::fold` folds `Iterator<Result<T, E>>` to `Result<V, E>`
* `result::fold_` folds `Iterator<Result<T, E>>` to `Result<(), E>`
Remove the only use of either::partition since it was better
accomplished with vector methods.
Update either::partition so that it sizes the vectors correctly before
it starts.
- Methodyfied the string ascii extionsion functions - They got added recently, I wrapped them in a trait.
- Added `into_owned()` method for vectors - similar to `Str`'s `into_owned()` function, allows to convert to a owned vector without making a copy if the source is a owned vector.
- Added `or_some` method to option - similar to `unwrap_or_default`, but keeps the values wrapped in an `Option`. Useful for `Option` chains, eg Iterator impls.
- Added `DoubleEndedIterator` impl to `Option` - Just for compatibility with generic Iterator functions.
- Renamed nil.rs to unit.rs - the type got renamed ages ago, it's time the source file is as well.
If they are on the trait then it is extremely annoying to use them as
generic parameters to a function, e.g. with the iterator param on the trait
itself, if one was to pass an Extendable<int> to a function that filled it
either from a Range or a Map<VecIterator>, one needs to write something
like:
fn foo<E: Extendable<int, Range<int>> +
Extendable<int, Map<&'self int, int, VecIterator<int>>>
(e: &mut E, ...) { ... }
since using a generic, i.e. `foo<E: Extendable<int, I>, I: Iterator<int>>`
means that `foo` takes 2 type parameters, and the caller has to specify them
(which doesn't work anyway, as they'll mismatch with the iterators used in
`foo` itself).
This patch changes it to:
fn foo<E: Extendable<int>>(e: &mut E, ...) { ... }
This includes a number of improvements to `ifmt!`
* Implements formatting arguments -- `{:0.5x}` works now
* Formatting now works on all integer widths, not just `int` and `uint`
* Added a large doc block to `std::fmt` which should help explain what `ifmt!` is all about
* Added floating point formatters, although they have the same pitfalls from before (they're just proof-of-concept now)
Closed a couple of issues along the way, yay! Once this gets into a snapshot, I'll start looking into removing all of `fmt`
It now actually does logging, and is compiled out when `--cfg rtdebug` is not
given to the libstd build, which it isn't by default. This makes the rt
benchmarks 18-50% faster.
Use Eq + Ord for lexicographical ordering of sequences.
For each of <, <=, >= or > as R, use::
[x, ..xs] R [y, ..ys] = if x != y { x R y } else { xs R ys }
Previous code using `a < b` and then `!(b < a)` for short-circuiting
fails on cases such as [1.0, 2.0] < [0.0/0.0, 3.0], where the first
element was effectively considered equal.
Containers like &[T] did also implement only one comparison operator `<`,
and derived the comparison results from this. This isn't correct either for
Ord.
Implement functions in `std::iterator::order::{lt,le,gt,ge,equal,cmp}` that all
iterable containers can use for lexical order.
We also visit tuple ordering, having the same problem and same solution
(but differing implementation).
what amount a T* pointer must be adjusted to reach the contents
of the box. For `~T` types, this requires knowing the type `T`,
which is not known in the case of objects.
This PR fixes#7235 and #3371, which removes trailing nulls from `str` types. Instead, it replaces the creation of c strings with a new type, `std::c_str::CString`, which wraps a malloced byte array, and respects:
* No interior nulls
* Ends with a trailing null
r? @graydon Also, notably, make rustpkgtest depend on the rustpkg executable (otherwise, tests that shell out to rustpgk might run when rustpkg doesn't exist).
This commit allows you to write:
extern mod x = "a/b/c";
which means rustc will search in the RUST_PATH for a package with
ID a/b/c, and bind it to the name `x` if it's found.
Incidentally, move get_relative_to from back::rpath into std::path
Mostly optimizing TLS accesses to bring local heap allocation performance
closer to that of oldsched. It's not completely at parity but removing the
branches involved in supporting oldsched and optimizing pthread_get/setspecific
to instead use our dedicated TCB slot will probably make up for it.
It now actually does logging, and is compiled out when `--cfg rtdebug` is not
given to the libstd build, which it isn't by default. This makes the rt
benchmarks 18-50% faster.
Mostly optimizing TLS accesses to bring local heap allocation performance
closer to that of oldsched. It's not completely at parity but removing the
branches involved in supporting oldsched and optimizing pthread_get/setspecific
to instead use our dedicated TCB slot will probably make up for it.
Basically, generic containers should not use the default methods since a
type of elements may not guarantees total order. str could use them
since u8's Ord guarantees total order. Floating point numbers are also
broken with the default methods because of NaN. Thanks for @thestinger.
Timespec also guarantees total order AIUI. I'm unsure whether
extra::semver::Identifier does so I left it alone. Proof needed.
Signed-off-by: OGINO Masanori <masanori.ogino@gmail.com>
FromStr implemented from scratch.
It is overengineered a bit, however.
Old implementation handles errors by fail!()-ing. And it has bugs, like it accepts `127.0.0.1::127.0.0.1` as IPv6 address, and does not handle all ipv4-in-ipv6 schemes. So I decided to implement parser from scratch.
This pull request converts the scheduler from a naive shared queue scheduler to a naive workstealing scheduler. The deque is still a queue inside a lock, but there is still a substantial performance gain. Fiddling with the messaging benchmark I got a ~10x speedup and observed massively reduced memory usage.
There are still *many* locations for optimization, but based on my experience so far it is a clear performance win as it is now.
This is a fairly large rollup, but I've tested everything locally, and none of
it should be platform-specific.
r=alexcrichton (bdfdbdd)
r=brson (d803c18)
r=alexcrichton (a5041d0)
r=bstrie (317412a)
r=alexcrichton (135c85e)
r=thestinger (8805baa)
r=pcwalton (0661178)
r=cmr (9397fe0)
r=cmr (caa4135)
r=cmr (6a21d93)
r=cmr (4dc3379)
r=cmr (0aa5154)
r=cmr (18be261)
r=thestinger (f10be03)
Use the definition, where R is <, <=, >=, or >
[x, ..xs] R [y, ..ys] = if x != y { x R y } else { xs R ys }
Previously, tuples would only implement < and derive the other
comparisons from it; this is incorrect. Included are several testcases
involving NaN comparisons that are now correct.
Previously, tuples would consider an element equal if both a < b and
b < a were false, this was also incorrect.
Use Eq + Ord for lexicographical ordering of sequences.
For each of <, <=, >= or > as R, use::
[x, ..xs] R [y, ..ys] = if x != y { x R y } else { xs R ys }
Previous code using `a < b` and then `!(b < a)` for short-circuiting
fails on cases such as [1.0, 2.0] < [0.0/0.0, 3.0], where the first
element was effectively considered equal.
This is a reopening of #8182, although this removes any abuse of the compiler internals. Now it's just a pure syntax extension (hard coded what the attribute names are).
According to #7887, we've decided to use the syntax of `fn map<U>(f: &fn(&T) -> U) -> U`, which passes a reference to the closure, and to `fn map_move<U>(f: &fn(T) -> U) -> U` which moves the value into the closure. This PR adds these `.map_move()` functions to `Option` and `Result`.
In addition, it has these other minor features:
* Replaces a couple uses of `option.get()`, `result.get()`, and `result.get_err()` with `option.unwrap()`, `result.unwrap()`, and `result.unwrap_err()`. (See #8268 and #8288 for a more thorough adaptation of this functionality.
* Removes `option.take_map()` and `option.take_map_default()`. These two functions can be easily written as `.take().map_move(...)`.
* Adds a better error message to `result.unwrap()` and `result.unwrap_err()`.
The two deletions are because the test cases are very old (still using `class` and modes!), and, as far as I can tell (since they are so old), the areas they test are well tested by other rpass tests.
Some general clean-up relating to deriving:
- `TotalOrd` was too eager, and evaluated the `.cmp` call for every field, even if it could short-circuit earlier.
- the pointer types didn't have impls for `TotalOrd` or `TotalEq`.
- the Makefiles didn't reach deep enough into libsyntax for dependencies.
(Split out from https://github.com/mozilla/rust/pull/8258.)
This results in throwing away alias analysis information, because LLVM
does *not* implement reasoning about these conversions yet.
We specialize zero-size types since a `getelementptr` offset will
return us the same pointer, making it broken as a simple counter.
This lazily initializes the taskgroup structs for ```spawn_unlinked``` tasks. If such a task never spawns another task linked to it (or a descendant of it), its taskgroup is simply never initialized at all. Also if an unlinked task spawns another unlinked task, neither of them will need to initialize their taskgroups. This works for the main task too.
I benchmarked this with the following test case and observed a ~~21% speedup (average over 4 runs: 7.85 sec -> 6.20 sec, 2.5 GHz)~~ 11% speedup, see comment below.
```
use std::task;
use std::cell::Cell;
use std::rt::comm;
static NUM: uint = 1024*256;
fn run(f: ~fn()) {
let mut t = task::task();
t.unlinked();
t.spawn(f);
}
fn main() {
do NUM.times {
let (p,c) = comm::oneshot();
let c = Cell::new(c);
do run { c.take().send(()); }
p.recv();
}
}
```
Better than that in rt::uv::net, because it:
* handles invalid input explicitly, without fail!()
* parses socket address, not just IP
* handles various ipv4-in-ipv6 addresses, like 2001:db8:122:344::192.0.2.33
(see http://tools.ietf.org/html/rfc6052 for example)
* rejects output like `127.0000000.0.1`
* does not allocate heap memory
* have unit tests
`fn slice_bytes` is marked unsafe since it allows violating the valid
string encoding property; but the function did also allow extending the
lifetime of the slice by mistake, since it's returning `&str`.
Use the annotation `slice_bytes<'a>(&'a str, ...) -> &'a str` so
that all uses of `slice_bytes` are region checked correctly.
Let Option be a base for a widely useful one- or zero- item iterator.
Refactor OptionIterator to support any generic element type, so the same
iterator impl can be used for both &T, &mut T and T iterators.
This is an alternative version to https://github.com/mozilla/rust/pull/8268, where instead of transitioning to `get()` completely, I transitioned to `unwrap()` completely.
My reasoning for also opening this PR is that having two different functions with identical behavior on a common datatype is bad for consistency and confusing for users, and should be solved as soon as possible. The fact that apparently half the code uses `get()`, and the other half `unwrap()` only makes it worse.
If the final naming decision ends up different, there needs to be a big renaming anyway, but until then it should at least be consistent.
---
- Made naming schemes consistent between Option, Result and Either
- Lifted the quality of the either and result module to that of option
- Changed Options Add implementation to work like the maybe Monad (return None if any of the inputs is None)
See https://github.com/mozilla/rust/issues/6002, especially my last comment.
- Removed duplicate Option::get and renamed all related functions to use the term `unwrap` instead
See also https://github.com/mozilla/rust/issues/7887.
Todo:
Adding testcases for all function in the three modules. Even without the few functions I added, the coverage wasn't complete to begin with. But I'd rather do that as a follow up PR, I've touched to much code here already, need to go through them again later.
- Made naming schemes consistent between Option, Result and Either
- Changed Options Add implementation to work like the maybe monad (return None if any of the inputs is None)
- Removed duplicate Option::get and renamed all related functions to use the term `unwrap` instead
fn slice_bytes is marked unsafe since it allows violating the valid
string encoding property; but the function did also allow extending the
lifetime of the slice by mistake, since it's returning `&str`.
Use the annotation `slice_bytes<'a>(&'a str, ...) -> &'a str` so
that all uses of slice_bytes are region checked correctly.
