Use Eq + Ord for lexicographical ordering of sequences.
For each of <, <=, >= or > as R, use::
[x, ..xs] R [y, ..ys] = if x != y { x R y } else { xs R ys }
Previous code using `a < b` and then `!(b < a)` for short-circuiting
fails on cases such as [1.0, 2.0] < [0.0/0.0, 3.0], where the first
element was effectively considered equal.
Containers like &[T] did also implement only one comparison operator `<`,
and derived the comparison results from this. This isn't correct either for
Ord.
Implement functions in `std::iterator::order::{lt,le,gt,ge,equal,cmp}` that all
iterable containers can use for lexical order.
We also visit tuple ordering, having the same problem and same solution
(but differing implementation).
I'm a bit disappointed that I couldn't figure out how to factor out more of the code implementing `extra::sync` but I feel this is an okay start. Also I added some documentation explaining that `WaitQueue` isn't thread safe, and needs an exclusive lock.
@bblum
This PR fixes#7235 and #3371, which removes trailing nulls from `str` types. Instead, it replaces the creation of c strings with a new type, `std::c_str::CString`, which wraps a malloced byte array, and respects:
* No interior nulls
* Ends with a trailing null
Basically, generic containers should not use the default methods since a
type of elements may not guarantees total order. str could use them
since u8's Ord guarantees total order. Floating point numbers are also
broken with the default methods because of NaN. Thanks for @thestinger.
Timespec also guarantees total order AIUI. I'm unsure whether
extra::semver::Identifier does so I left it alone. Proof needed.
Signed-off-by: OGINO Masanori <masanori.ogino@gmail.com>
This is a fairly large rollup, but I've tested everything locally, and none of
it should be platform-specific.
r=alexcrichton (bdfdbdd)
r=brson (d803c18)
r=alexcrichton (a5041d0)
r=bstrie (317412a)
r=alexcrichton (135c85e)
r=thestinger (8805baa)
r=pcwalton (0661178)
r=cmr (9397fe0)
r=cmr (caa4135)
r=cmr (6a21d93)
r=cmr (4dc3379)
r=cmr (0aa5154)
r=cmr (18be261)
r=thestinger (f10be03)
Write external iterators for Difference, Sym. Difference, Intersection
and Union set operations.
These iterators are generic insofar that they could work on any ordered
sequence iterators, even though they are type specialized to the
TreeSetIterator in this case.
Looking at the `check` function in the treeset tests, rustc seems
unwilling to compile a function resembling::
fn check<'a, T: Iterator<&'a int>>(... )
so the tests for these iterators are still running the legacy loop
protocol.
According to #7887, we've decided to use the syntax of `fn map<U>(f: &fn(&T) -> U) -> U`, which passes a reference to the closure, and to `fn map_move<U>(f: &fn(T) -> U) -> U` which moves the value into the closure. This PR adds these `.map_move()` functions to `Option` and `Result`.
In addition, it has these other minor features:
* Replaces a couple uses of `option.get()`, `result.get()`, and `result.get_err()` with `option.unwrap()`, `result.unwrap()`, and `result.unwrap_err()`. (See #8268 and #8288 for a more thorough adaptation of this functionality.
* Removes `option.take_map()` and `option.take_map_default()`. These two functions can be easily written as `.take().map_move(...)`.
* Adds a better error message to `result.unwrap()` and `result.unwrap_err()`.
This module provided adaptors for the old internal iterator protocol,
but they proved to be quite unreadable and are not generic enough to
handle borrowed pointers well.
Since Rust no longer defines an internal iteration protocol, I don't
think there's going to be any reuse via these adaptors.
Encoding should really only be done from [u8]<->str. The extra
convenience implementations don't really have a place, especially since
they're so trivial.
Also improved error messages in FromBase64.
The overhead of str::push_char is high enough to cripple the performance
of these two functions. I've switched them to build the output in a
~[u8] and then convert to a string later. Since we know exactly the
bytes going into the vector, we can use the unsafe version to avoid the
is_utf8 check.
I could have riced it further with vec::raw::get, but it only added
~10MB/s so I didn't think it was worth it. ToHex is still ~30% slower
than FromHex, which is puzzling.
Before:
```
test base64::test::from_base64 ... bench: 1000 ns/iter (+/- 349) = 204 MB/s
test base64::test::to_base64 ... bench: 2390 ns/iter (+/- 1130) = 63 MB/s
...
test hex::tests::bench_from_hex ... bench: 884 ns/iter (+/- 220) = 341 MB/s
test hex::tests::bench_to_hex ... bench: 2453 ns/iter (+/- 919) = 61 MB/s
```
After:
```
test base64::test::from_base64 ... bench: 1271 ns/iter (+/- 600) = 160 MB/s
test base64::test::to_base64 ... bench: 759 ns/iter (+/- 286) = 198 MB/s
...
test hex::tests::bench_from_hex ... bench: 875 ns/iter (+/- 377) = 345 MB/s
test hex::tests::bench_to_hex ... bench: 593 ns/iter (+/- 240) = 254 MB/s
```
FromHex ignores whitespace and parses either upper or lower case hex
digits. ToHex outputs lower case hex digits with no whitespace. Unlike
ToBase64, ToHex doesn't allow you to configure the output format. I
don't feel that it's super useful in this case.
- Made naming schemes consistent between Option, Result and Either
- Changed Options Add implementation to work like the maybe monad (return None if any of the inputs is None)
- Removed duplicate Option::get and renamed all related functions to use the term `unwrap` instead
It seems that relatively new code uses `Foo::new()` instead of `Foo()` so I wrote a patch to migrate some structs to the former style.
Is it a right direction? If there are any guidelines not to use new()-style, could you add them to the [style guide](https://github.com/omasanori/rust/wiki/Note-style-guide)?
The compiler-generated dtor for DList recurses deeply to drop Nodes.
For big lists this can overflow the stack.
This is a problem for the new scheduler, where split stacks are not implemented.
Thanks @blake2-ppc
WIth this patch `RUSTFLAGS='--cfg unicode' make check"` passed successfully.
* Why doesn't `#[link_name="icuuc"]` make libextra to link against libicuuc.so?
* In `extra::unicode::tests`, `use unicode; unicode::is_foo('a')` failed but `use unicode::*; is_foo('a')` succeeded. Is it right?
RWArc had a clone() method, but it was part of impl RWArc instead of
an implementation of Clone.
Stick with the explicit implementation instead of deriving Clone so we
can have a docstring.
Fixes#8052.
Added functions to cryptoutil.rs that perform an addition after shifting
the 2nd parameter by a specified constant. These function fail!() if integer
overflow will result. Updated the Sha2 implementation to use these functions.
Create a helper function in cryptoutil.rs which feeds 1,000,000 'a's into
a Digest with varying input sizes and then checks the result. This is
essentially the same as one of Sha1's existing tests, so, that test was
re-implemented using this method. New tests were added using this method for
Sha512 and Sha256.
The Sha2 compression functions were re-written to execute the message
scheduling calculations in the same loop as the rest of the compression
function. The compiler is able to generate much better code. Additionally,
innermost part of the compression functions were turned into macros to
reduce code duplicate and to make the functions more concise.
There are 2 main pieces of functionality in cryptoutil.rs:
* A set of unsafe function for efficiently reading and writing u32 and u64
values. All of these functions are fairly easy to audit to confirm that
they do what they are supposed to.
* A FixedBuffer struct. This struct keeps track of input data until there
is enough of it to execute the a function on it which expects a fixed
block of data.
The Sha2 module was rewritten to take advantage of the new functions in
cryptoutil as well as FixedBuffer. The result is that the duplicate code
for maintaining a buffer of input data is removed from the Sha512 and
Sha256 implementation. Additionally, the FixedBuffer code is much more
efficient than the previous code was.
The result_X() methods just calculate an output of a fixed size. They don't
really have much to do with running the actually hash algorithm until the very
last step - the output. It makes much more sense to put all this logic into
the Digest impls for each specific variation on the hash function.
The code was arranged so that the core Sha2 code came first, and then
all of the various implementation of Digest followed along later. The
problem is that the Sha512 compression function code is far away from
the Sha512 Digest implementation, so, if you are trying to read over
the code, you need to scroll all around the file for no good reason. The
code was rearranged so that all of the Sha512 code is in one place and
all of the Sha256 code is in another and so that all impls for a struct
are near the definition of that struct.
Change the former repetition::
for 5.times { }
to::
do 5.times { }
.times() cannot be broken with `break` or `return` anymore; for those
cases, use a numerical range loop instead.
.intersection(), .union() etc methods in trait std::container::Set use
internal iters. Remove these methods from the trait.
I reported issue #8154 for the reinstatement of iterator-based set algebra
methods to the Set trait.
For bitv and treemap, that lack Iterator implementations of set
operations, preserve them as methods directly on the types themselves.
For HashSet, these methods are replaced by the present .union_iter()
etc.
This removes a bunch of options from the task builder interface that are irrelevant to the new scheduler and were generally unused anyway. It also bumps the stack size of new scheduler tasks so that there's enough room to run rustc and changes the interface to `Thread` to not implicitly join threads on destruction, but instead require an explicit, and mandatory, call to `join`.
Main logic in ```Implement select() for new runtime pipes.```. The guts of the ```PortOne::try_recv()``` implementation are now split up across several functions, ```optimistic_check```, ```block_on```, and ```recv_ready```.
There is one weird FIXME I left open here, in the "implement select" commit -- an assertion I couldn't get to work in the receive path, on an invariant that for some reason doesn't hold with ```SharedPort```. Still investigating this.
Renamed bytes_iter to byte_iter to match other iterators
Refactored str Iterators to use DoubleEnded Iterators and typedefs instead of wrapper structs
Reordered the Iterator section
Whitespace fixup
Moved clunky `each_split_within` function to the one place in the tree where it's actually needed
Replaced all block doccomments in str with line doccomments
Drop the "Iterator" suffix for the the structs in std::iterator.
Filter, Zip, Chain etc. are shorter type names for when iterator
pipelines need their types written out in full in return value types, so
it's easier to read and write. the iterator module already forms enough
namespace.
Good evening,
This is a superset of @MaikKlein's #7969 commit, that I've fixed up to compile. I had a couple commits I wanted to do on top of @MaikKlein's work that I didn't want to bitrot.
To be more specific:
`UPPERCASETYPE` was changed to `UppercaseType`
`type_new` was changed to `Type::new`
`type_function(value)` was changed to `value.method()`
With the recent fixes to method resolution, we can now remove the
dummy type parameters used as crutches in the iterator module.
For example, the zip adaptor type is just ZipIterator<T, U> now.
This is a cleanup pull request that does:
* removes `os::as_c_charp`
* moves `str::as_buf` and `str::as_c_str` into `StrSlice`
* converts some functions from `StrSlice::as_buf` to `StrSlice::as_c_str`
* renames `StrSlice::as_buf` to `StrSlice::as_imm_buf` (and adds `StrSlice::as_mut_buf` to match `vec.rs`.
* renames `UniqueStr::as_bytes_with_null_consume` to `UniqueStr::to_bytes`
* and other misc cleanups and minor optimizations
Factor out internal methods to pop/push list nodes so that .merge() and .rotate_to_front(), .rotate_to_back() (new methods) can be implemented without allocating nodes.
With that, some cleanup changes to DList use of Option, and adding a missing Encodable implementation.
SmallIntSet is equivalent to BitvSet but with 64 times the memory
overhead. There's no reason for it to exist.
SmallIntSet's overhead should really only be 8 times, but for some
reason, `sys::size_of::<Option<()>>() == 8`, not 1.
Switched Bitv and BitvSet to external iterators. They still use some internal iterators internally (ha).
Derived clone for all Bitv types.
Removed indirection in BitvVariant. It previously held a unique pointer to the appropriate Bitv struct, even though those structs are the size of a pointer themselves. BitvVariant is the same size (16 bytes) as it was previously.
.peek_next() needs to check the element counter just like the .next()
and .next_back() iterators do.
Also clarify .insert_next() doc w.r.t double ended iteration.
Made the `iter` and `mut_iter` methods on SmallIntMap and SmallIntSet return double-ended-iterators. These iterators now implement `size_hint`.
Also the iterator tests only tested dense maps/sets, which aren't very useful. So they were changed to iterate over sparse maps/sets.
Fixes#7721
Factor out internal methods for pop/push ~Node<T>, This allows moving
nodes instead of destructuring and allocating new.
Make use of this in .merge() so that it requires no allocations when
merging two DList.
Implement method .cycle() that repeats an iterator endlessly
Implement Clone for simple iterators (without closures), including VecIterator.
> The theory is simple, the immutable iterators simply hold state
> variables (indicies or pointers) into frozen containers. We can freely
> clone these iterators, just like we can clone borrowed pointers.
Updated all users of HashMap, HashSet ::consume() to use
.consume_iter().
Since .consume_iter() takes the map or set by value, it needs awkward
extra code to in librusti's use of @mut HashMap, where the map value can
not be directly moved out.
Addresses issue #7719
Updated all users of HashMap, HashSet old .consume() to use .consume()
with a for loop.
Since .consume() takes the map or set by value, it needs awkward
extra code to in librusti's use of @mut HashMap, where the map value can
not be directly moved out.
This should get us over the hump of activating basic ratcheting on codegen tests, at least. It also puts in place optional (disabled by default) ratcheting on all #[bench] tests, and records all metrics from them to harvestable .json files in any case.
Changes int/uint range_rev to iterate over range `(hi,lo]` instead of `[hi,lo)`.
Fix#5270.
Also:
* Adds unit tests for int/uint range functions
* Updates the uses of `range_rev` to account for the new semantics. (Note that pretty much all of the updates there were strict improvements to the code in question; yay!)
* Exposes new function, `range_step_inclusive`, which does the range `[hi,lo]`, (at least when `hi-lo` is a multiple of the `step` parameter).
* Special-cases when `|step| == 1` removing unnecessary bounds-check. (I did not check whether LLVM was already performing this optimization; I figure it would be a net win to not leave that analysis to the compiler. If reviewer objects, I can easily remove that from the refactored code.)
(This pull request is a rebased version of PR #7524, which went stale due to recent unrelated changes to num libraries.)
Note that Version's `le` is not "less than or equal to" now, since `lt`
ignores build metadata. I think the new ordering algorithm satisfies
strict weak ordering which C++ STL requires, instead of strict total
ordering.
Teach `extra::term` to support more terminal attributes than just color.
Fix the compiler diagnostic messages to print in bold instead of bright white. This matches Clang's output.
Cache the term::Terminal instead of re-parsing for every diagnostic (fixes#6827).
If the TLS key is 0-sized, then the linux linker is apparently smart enough to
put everything at the same pointer. OSX on the other hand, will reserve some
space for all of them. To get around this, the TLS key now actuall consumes
space to ensure that it gets a unique pointer
Fixes most of #4989. I didn't add Persistent{Set,Map} since the only
persistent data structure is fun_treemap and its functionality is
currently too limited to build a trait out of.
Unify the mutable iterators too. Switch the ListInsertion trait to use
method .insert_next() and .peek_next() for list mutation. .insert_next()
inserts an element into the list that will not appear in iteration, of
course; so the length of the iteration can not change during iteration.
Did not properly allow runs from the `other` list to be merged in. The
test case was using a wrong expected value.
Edited docs for merge so they explain more clearly what it does.
Also make sure insert_ordered is marked pub.
cc #6004 and #3273
This is a rewrite of TLS to get towards not requiring `@` when using task local storage. Most of the rewrite is straightforward, although there are two caveats:
1. Changing `local_set` to not require `@` is blocked on #7673
2. The code in `local_pop` is some of the most unsafe code I've written. A second set of eyes should definitely scrutinize it...
The public-facing interface currently hasn't changed, although it will have to change because `local_data::get` cannot return `Option<T>`, nor can it return `Option<&T>` (the lifetime isn't known). This will have to be changed to be given a closure which yield `&T` (or as an Option). I didn't do this part of the api rewrite in this pull request as I figured that it could wait until when `@` is fully removed.
This also doesn't deal with the issue of using something other than functions as keys, but I'm looking into using static slices (as mentioned in the issues).
An iterator that allows mutating the list is very useful but needs care
to not be unsound. ListIteration exposes only insert_before (used for
insert_ordered) and peek_next so far.
This is an owned sendable linked list which allows insertion and
deletion at both ends, with fast traversal through iteration, and fast
append/prepend.
It is indended to replace the previous managed DList with exposed list
nodes. It does not match it feature by feature, but DList could grow
more methods if needed.
This is much faster for strings, and eventually when there is a
buffered reader of some sort.
Reading example.json 100 times before was around 1.18s.
After:
- reading from string 0.68s
- reading from file 1.08s (extra time is all in io::Reader)
r? @graydon, @nikomatsakis, @pcwalton, or @catamorphism
Sorry this is so huge, but it's been accumulating for about a month. There's lots of stuff here, mostly oriented toward enabling multithreaded scheduling and improving compatibility between the old and new runtimes. Adds task pinning so that we can create the 'platform thread' in servo.
[Here](e1555f9b56/src/libstd/rt/mod.rs (L201)) is the current runtime setup code.
About half of this has already been reviewed.
The free-standing functions in f32, f64, i8, i16, i32, i64, u8, u16,
u32, u64, float, int, and uint are replaced with generic functions in
num instead.
This means that instead of having to know everywhere what the type is, like
~~~
f64::sin(x)
~~~
You can simply write code that uses the type-generic versions in num instead, this works for all types that implement the corresponding trait in num.
~~~
num::sin(x)
~~~
Note 1: If you were previously using any of those functions, just replace them
with the corresponding function with the same name in num.
Note 2: If you were using a function that corresponds to an operator, use the
operator instead.
Note 3: This is just https://github.com/mozilla/rust/pull/7090 reopened against master.
Correct treatment of irrefutable patterns. The old code was wrong in many, many ways. `ref` bindings didn't work, it sometimes copied when it should have moved, the borrow checker didn't even look at such patterns at all, we weren't consistent about preventing values with destructors from being pulled apart, etc.
Fixes#3224.
Fixes#3225.
Fixes#3255.
Fixes#6225.
Fixes#6386.
r? @catamorphism
Avoids the overhead of read_char for every character.
Benchmark reading example.json 10 times from
https://code.google.com/p/rapidjson/wiki/Performance
Before: 2.55s
After: 0.16s
Regression testing is already done by isrustfastyet.
The free-standing functions in f32, f64, i8, i16, i32, i64, u8, u16,
u32, u64, float, int, and uint are replaced with generic functions in
num instead.
If you were previously using any of those functions, just replace them
with the corresponding function with the same name in num.
Note: If you were using a function that corresponds to an operator, use
the operator instead.
It's broken/unmaintained and needs to be rewritten to avoid managed
pointers and needless copies. A full rewrite is necessary and the API
will need to be redone so it's not worth keeping this around (#7628).
Closes#2236, #2744
Where * = tcp, ip, url.
Formerly, extra::net::* were aliases of extra::net_*, but were the
recommended path to use. Thus, the documentation talked of the `net_*`
modules while everything else was written expecting `net::*`.
This moves the actual modules so that `extra::net::*` is the actual
location of the modules.
This will naturally break any code which used `extra::net_*` directly.
They should be altered to use `extra::net::*` (which has been the
documented way of doing things for some time).
This ensures that there is one, and only one, obvious way of doing
things.
Change the signature of Iterator.size_hint() to always have a lower bound.
Implement .size_hint() on all remaining iterators (if it differs from the default).
It's broken/unmaintained and needs to be rewritten to avoid managed
pointers and needless copies. A full rewrite is necessary and the API
will need to be redone so it's not worth keeping this around.
Closes#2236, #2744
Adding iterators for extra::smallintmap
Working on mutability error
Ran into ICE
More mutability problems
Working through mutability issue
working on getting tests passing
SmallIntMa tests passing
Added SmallIntSet iterators, and the tests are passing
Stripped trailing spaces
Removed extra use directive
The deque is split at the marker lo, or logical index 0. Move the
shortest part (split by lo) when growing. This way add_front is just as
fast as add_back, on average.
The previous implementation of reverse iterators used modulus (%) of
negative indices, which did work but was fragile due to dependency on
the divisor.
The deque is determined by vec self.elts.len(), self.nelts, and self.lo,
and self.hi is calculated from these.
self.hi is just the raw index of element number `self.nelts`
Fix some issues with the deque being very slow, keep the same vec around
instead of constructing a new. Move as few elements as possible, so the
self.lo point is not moved after grow.
[o o o o o|o o o]
hi...^ ^.... lo
grows to
[. . . . .|o o o o o o o o|. . .]
^.. lo ^.. hi
If the deque is append-only, it will result in moving no elements on
grow. If the deque is prepend-only, all will be moved each time.
The bench tests added show big improvements:
Timed using `rust build -O --test extra.rs && ./extra --bench deque`
Old version:
test deque::tests::bench_add_back ... bench: 4976 ns/iter (+/- 9)
test deque::tests::bench_add_front ... bench: 4108 ns/iter (+/- 18)
test deque::tests::bench_grow ... bench: 416964 ns/iter (+/- 4197)
test deque::tests::bench_new ... bench: 408 ns/iter (+/- 12)
With this commit:
test deque::tests::bench_add_back ... bench: 12 ns/iter (+/- 0)
test deque::tests::bench_add_front ... bench: 16 ns/iter (+/- 0)
test deque::tests::bench_grow ... bench: 1515 ns/iter (+/- 30)
test deque::tests::bench_new ... bench: 419 ns/iter (+/- 3)
Avoids the overhead of read_char for every character.
Benchmark reading example.json 10 times from
https://code.google.com/p/rapidjson/wiki/Performance
Before: 2.55s
After: 0.16s
Regression testing is already done by isrustfastyet.
