libtest: Split HumanFormatter into {Pretty,Terse}
libtest: Fixed padding of benchmarks when not benchmarking
libtest: Fixed benchmarks' names not showing in terse-mode
libtest: Formatting
libtest: Json format now outputs failed tests' stdouts.
libtest: Json format now outputs failed tests' stdouts.
libtest: Json formatter now spews individiual events, not as an array
libtest: JSON fixes
libtest: Better JSON escaping
libtest: Test start event is printed on time
CloudABI doesn't make any distinction between TTYs and ordinary pipes.
While there, remove the redundant implementation used by Redox. It can
use the same stub function.
This structure doesn't seem to be used by libtest itself. It is used by
compiletest, but never passed on to anything externally. This makes it
easier to get the testing framework to work for CloudABI crossbuilds, as
CloudABI currently lacks PathBuf, which is used by TestPaths.
This commit adds a new target to the compiler: wasm32-unknown-unknown. This
target is a reimagining of what it looks like to generate WebAssembly code from
Rust. Instead of using Emscripten which can bring with it a weighty runtime this
instead is a target which uses only the LLVM backend for WebAssembly and a
"custom linker" for now which will hopefully one day be direct calls to lld.
Notable features of this target include:
* There is zero runtime footprint. The target assumes nothing exists other than
the wasm32 instruction set.
* There is zero toolchain footprint beyond adding the target. No custom linker
is needed, rustc contains everything.
* Very small wasm modules can be generated directly from Rust code using this
target.
* Most of the standard library is stubbed out to return an error, but anything
related to allocation works (aka `HashMap`, `Vec`, etc).
* Naturally, any `#[no_std]` crate should be 100% compatible with this new
target.
This target is currently somewhat janky due to how linking works. The "linking"
is currently unconditional whole program LTO (aka LLVM is being used as a
linker). Naturally that means compiling programs is pretty slow! Eventually
though this target should have a linker.
This target is also intended to be quite experimental. I'm hoping that this can
act as a catalyst for further experimentation in Rust with WebAssembly. Breaking
changes are very likely to land to this target, so it's not recommended to rely
on it in any critical capacity yet. We'll let you know when it's "production
ready".
---
Currently testing-wise this target is looking pretty good but isn't complete.
I've got almost the entire `run-pass` test suite working with this target (lots
of tests ignored, but many passing as well). The `core` test suite is still
getting LLVM bugs fixed to get that working and will take some time. Relatively
simple programs all seem to work though!
---
It's worth nothing that you may not immediately see the "smallest possible wasm
module" for the input you feed to rustc. For various reasons it's very difficult
to get rid of the final "bloat" in vanilla rustc (again, a real linker should
fix all this). For now what you'll have to do is:
cargo install --git https://github.com/alexcrichton/wasm-gc
wasm-gc foo.wasm bar.wasm
And then `bar.wasm` should be the smallest we can get it!
---
In any case for now I'd love feedback on this, particularly on the various
integration points if you've got better ideas of how to approach them!
Rationale:
We use --quiet mode when testing a PR in the CI. Also, we use `stamp` to
prefix every line with a timestamp. Previously, when testing in --quiet
mode, we will only print a dot for each test without any line breaks.
Combined with `stamp`, this means we'd need to wait for all tests to
complete before writing the output. On Travis CI, if we don't print
anything within 30 minutes, the job will be forcefully canceled. This
makes it very easy to spuriously-timeout when testing non-default images
like arm-android using the CI. This commit tries to workaround the issue
by printing a new line every 100 dots, forcing `stamp` to emit something
to reset Travis's countdown.
This change allows the user to add an `#[allow_fail]` attribute to
tests that will cause the test to compile & run, but if the test fails
it will not cause the entire test run to fail. The test output will
show the failure, but in yellow instead of red, and also indicate that
it was an allowed failure.
This commit deletes the in-tree `getopts` crate in favor of the crates.io-based
`getopts` crate. The main difference here is with a new builder-style API, but
otherwise everything else remains relatively standard.
When `RUST_BACKTRACE=1`, remove all frames after
`__rust_maybe_catch_panic`. Tested on `main`, threads, tests and
benches. Cleaning of the top of the stacktrace is let to a future PR.
Fixes#40201
See #41815
treat setting the number of test-threads to 0 as an error
It is currently possible to call `cargo test -- --test-threads=0` which will cause cargo to hang until aborted. This change will fix that and will report an appropriate error to the user.
Do not run outer setup part of benchmarks multiple times to fix issue 20142
Fix#20142
This is my first real rust code, so I expect the quality is quite bad. Please let me know in which ways it is horrible and I'll fix it.
Previously the whole benchmark function was rerun many times, but with this change, only the callback passed to iter is rerun. This improves performances by saving benchmark startup time. The setup used to be called a minimum of 101 times, and now only runs once.
I wasn't sure exactly what should be done for the case where iter is never called, so I left a FIXME for that: currently it does not error, and I added tests to cover that.
I have left the algorithm and statistics unchanged: I don't like how the minimum number of runs is 301 (that's bad for very slow benchmarks) but I consider such changes out of scope for this fix.
Running test with cargo test -- --test-threads=0 causes cargo to
hang as 0 is a valid usize. Adding zero threads as a special case
to the error handling.
Historically this was done to accommodate bugs in lints, but there hasn't been a
bug in a lint since this feature was added which the warnings affected. Let's
completely purge warnings from all our stages by denying warnings in all stages.
This will also assist in tracking down `stage0` code to be removed whenever
we're updating the bootstrap compiler.
This option lists all the tests and benchmarks a binary provides. By default the listing
is sent to stdout, but if --logfile is also specified, it is written there.
If filters are specified, they're applied before the output is emitted.
Filter matching is by substring by default. This makes it impossible
to run a single test if its name is a substring of some other test.
For example, its not possible to run just "mymod::test" with these
tests:
mymod::test
mymod::test1
mymod::test_module::moretests
You could declare by convention that no test has a name that's a
substring of another test, but that's not really practical.
This PR adds the "--exact" flag, to make filter matching exactly
match the complete name.
Most of the Rust community agrees that the vec! macro is clearer when
called using square brackets [] instead of regular brackets (). Most of
these ocurrences are from before macros allowed using different types of
brackets.
There is one left unchanged in a pretty-print test, as the pretty
printer still wants it to have regular brackets.
* Don't spawn two threads for all tests, just one now that `catch_unwind` is
stable.
* Remove usage of the unstable `box` keyword
* Remove usage of the unstable `FnBox` trait
libtest: add a --skip flag to the test runner
This flag takes a FILTER argument and instructs the test runner to skip
the tests whose names contain the word FILTER. --skip can be used
several times.
---
My motivation for submitting this is that while using [smoke] to run `std` unit tests for cross
targets I found that a few of the tests always fail due to limitations in QEMU (it can't handle too
many threads) and I'd like to skip these problematic tests from the command line to be able to run
the rest of the unit tests.
[smoke]: https://github.com/japaric/smoke
I know there is another mechanism to skip tests: `#[ignore]` but this doesn't work in my use case
because I can't (easily) modify the source of the standard libraries to `#[ignore]` some tests. And
even if I could, the change would involve conditionally ignoring some tests for some targets but
that's not a perfect solution because those tests should pass if executed on real hardware so they
should not be `#[ignored]` in that scenario.
r? @alexcrichton
cc @brson
This flag takes a FILTER argument and instructs the test runner to skip
the tests whose names contain the word FILTER. --skip can be used
several times.
Add --test-threads option to test binaries
This change allows parallelism of test runs to be specified by a
command line flag names --test-threads in addition to the existing
environment variable RUST_TEST_THREADS. Fixes#25636.
This change allows parallelism of test runs to be specified by a
command line flag names --test-threads in addition to the existing
environment variable RUST_TEST_THREADS. Fixes#25636.
This is a spiritual succesor to #34268/8531d581, in which we replaced a
number of matches of None to the unit value with `if let` conditionals
where it was judged that this made for clearer/simpler code (as would be
recommended by Manishearth/rust-clippy's `single_match` lint). The same
rationale applies to matches of None to the empty block.
This commit is an implementation of [RFC 1513] which allows applications to
alter the behavior of panics at compile time. A new compiler flag, `-C panic`,
is added and accepts the values `unwind` or `panic`, with the default being
`unwind`. This model affects how code is generated for the local crate, skipping
generation of landing pads with `-C panic=abort`.
[RFC 1513]: https://github.com/rust-lang/rfcs/blob/master/text/1513-less-unwinding.md
Panic implementations are then provided by crates tagged with
`#![panic_runtime]` and lazily required by crates with
`#![needs_panic_runtime]`. The panic strategy (`-C panic` value) of the panic
runtime must match the final product, and if the panic strategy is not `abort`
then the entire DAG must have the same panic strategy.
With the `-C panic=abort` strategy, users can expect a stable method to disable
generation of landing pads, improving optimization in niche scenarios,
decreasing compile time, and decreasing output binary size. With the `-C
panic=unwind` strategy users can expect the existing ability to isolate failure
in Rust code from the outside world.
Organizationally, this commit dismantles the `sys_common::unwind` module in
favor of some bits moving part of it to `libpanic_unwind` and the rest into the
`panicking` module in libstd. The custom panic runtime support is pretty similar
to the custom allocator support with the only major difference being how the
panic runtime is injected (takes the `-C panic` flag into account).
/# This is a combination of 16 commits.
/# The first commit's message is:
allow RUST_BACKTRACE=disabled to act as if unset
When RUST_BACKTRACE is set to "disabled" then this acts as if the env.
var is unset.
/# This is the 2nd commit message:
case insensitive "DiSaBLeD" RUST_BACKTRACE value
previously it expected a lowercase "disabled" to treat the env. var as
unset
/# This is the 3rd commit message:
RUST_BACKTRACE=0 acts as if unset
previously RUST_BACKTRACE=disabled was doing the same thing
/# This is the 4th commit message:
RUST_BACKTRACE=0|n|no|off acts as if unset
previously only RUST_BACKTRACE=0 acted as if RUST_BACKTRACE was unset
Now added more options (case-insensitive): 'n','no' and 'off'
eg. RUST_BACKTRACE=oFF
/# This is the 5th commit message:
DRY on the value of 2
DRY=don't repeat yourself
Because having to remember to keep the two places of '2' in sync is not
ideal, even though this is a simple enough case.
/# This is the 6th commit message:
Revert "DRY on the value of 2"
This reverts commit 95a0479d5cf72a2b2d9d21ec0bed2823ed213fef.
Nevermind this DRY on 2, because we already have a RY on 1,
besides the code is less readable this way...
/# This is the 7th commit message:
attempt to document unsetting RUST_BACKTRACE
/# This is the 8th commit message:
curb allocations when checking for RUST_BACKTRACE
this means we don't check for case-insensitivity anymore
/# This is the 9th commit message:
as decided, RUST_BACKTRACE=0 turns off backtrace
/# This is the 10th commit message:
RUST_TEST_NOCAPTURE=0 acts as if unset
(that is, capture is on)
Any other value acts as if nocapture is enabled (that is, capture is off)
/# This is the 11th commit message:
update other RUST_TEST_NOCAPTURE occurrences
apparently only one place needs updating
/# This is the 12th commit message:
update RUST_BACKTRACE in man page
/# This is the 13th commit message:
handle an occurrence of RUST_BACKTRACE
/# This is the 14th commit message:
ensure consistency with new rules for backtrace
/# This is the 15th commit message:
a more concise comment for RUST_TEST_NOCAPTURE
/# This is the 16th commit message:
update RUST_TEST_NOCAPTURE in man page
Automated conversion using the untry tool [1] and the following command:
```
$ find -name '*.rs' -type f | xargs untry
```
at the root of the Rust repo.
[1]: https://github.com/japaric/untry
Shorter output for `rustc --test` binaries.
Until now, a program created with `rustc --test` prints at least one line per test. This can be very verbose, especially with [data-driven tests](https://internals.rust-lang.org/t/test-and-external-test-harnesses/3145) when hundreds or thousands of tests is not rare.
This changes the default output to one character per test (except metrics and benchmarks results which have additional data to show):
```
Running target/debug/wpt-75c594dc1e6e6187
running 314 tests
..............................................................................
..............................................................................
..............................................................................
..............................................................................
..
test result: ok. 314 passed; 0 failed; 0 ignored; 0 measured
```
<s>The previous behavior is available by passing `--verbose` to the test program. Maybe `cargo test --verbose` could be changed to do that?</s> **Edit:** the default is now unchanged, `-q` or `--quiet` enables the new output.
A program created with `rustc --test` prints at least one line per test.
This can be very verbose, especially with [data-driven tests](
https://internals.rust-lang.org/t/test-and-external-test-harnesses/3145)
when hundreds or thousands of tests is not rare.
This adds a `-q` or `--quiet` option that changes the output
to one character instead of one line per test
(except metrics and benchmarks results which have additional data to
show):
```
Running target/debug/wpt-75c594dc1e6e6187
running 314 tests
..............................................................................
..............................................................................
..............................................................................
..............................................................................
..
test result: ok. 314 passed; 0 failed; 0 ignored; 0 measured
```
This is a breaking change since the `test::TestOpts` struct
now has one more field.
Right now everything in TARGET_CRATES is built by default for all non-fulldeps
tests and is distributed by default for all target standard library packages.
Currenly this includes a number of unstable crates which are rarely used such as
`graphviz` and `rbml`>
This commit trims down the set of `TARGET_CRATES`, moves a number of tests to
`*-fulldeps` as a result, and trims down the dependencies of libtest so we can
distribute fewer crates in the `rust-std` packages.
This commit is the result of the FCPs ending for the 1.8 release cycle for both
the libs and the lang suteams. The full list of changes are:
Stabilized
* `braced_empty_structs`
* `augmented_assignments`
* `str::encode_utf16` - renamed from `utf16_units`
* `str::EncodeUtf16` - renamed from `Utf16Units`
* `Ref::map`
* `RefMut::map`
* `ptr::drop_in_place`
* `time::Instant`
* `time::SystemTime`
* `{Instant,SystemTime}::now`
* `{Instant,SystemTime}::duration_since` - renamed from `duration_from_earlier`
* `{Instant,SystemTime}::elapsed`
* Various `Add`/`Sub` impls for `Time` and `SystemTime`
* `SystemTimeError`
* `SystemTimeError::duration`
* Various impls for `SystemTimeError`
* `UNIX_EPOCH`
* `ops::{Add,Sub,Mul,Div,Rem,BitAnd,BitOr,BitXor,Shl,Shr}Assign`
Deprecated
* Scoped TLS (the `scoped_thread_local!` macro)
* `Ref::filter_map`
* `RefMut::filter_map`
* `RwLockReadGuard::map`
* `RwLockWriteGuard::map`
* `Condvar::wait_timeout_with`
Closes#27714Closes#27715Closes#27746Closes#27748Closes#27908Closes#29866
around a set of paths called `TestPaths`
This commit is not quite standalone; it basically contains all the
borrowing plumbing bits, the interesting stuff comes in the next commit.
Backtraces, and the compilation of libbacktrace for asmjs, are disabled.
This port doesn't use jemalloc so, like pnacl, it disables jemalloc *for all targets*
in the configure file.
It disables stack protection.
This pull request adds support for [Illumos](http://illumos.org/)-based operating systems: SmartOS, OpenIndiana, and others. For now it's x86-64 only, as I'm not sure if 32-bit installations are widespread. This PR is based on #28589 by @potatosalad, and also closes#21000, #25845, and #25846.
Required changes in libc are already merged: https://github.com/rust-lang-nursery/libc/pull/138
Here's a snapshot required to build a stage0 compiler:
https://s3-eu-west-1.amazonaws.com/nbaksalyar/rustc-sunos-snapshot.tar.gz
It passes all checks from `make check`.
There are some changes I'm not quite sure about, e.g. macro usage in `src/libstd/num/f64.rs` and `DirEntry` structure in `src/libstd/sys/unix/fs.rs`, so any comments on how to rewrite it better would be greatly appreciated.
Also, LLVM configure script might need to be patched to build it successfully, or a pre-built libLLVM should be used. Some details can be found here: https://llvm.org/bugs/show_bug.cgi?id=25409
Thanks!
r? @brson
This commit removes the `-D warnings` flag being passed through the makefiles to
all crates to instead be a crate attribute. We want these attributes always
applied for all our standard builds, and this is more amenable to Cargo-based
builds as well.
Note that all `deny(warnings)` attributes are gated with a `cfg(stage0)`
attribute currently to match the same semantics we have today
test: Increase resolution of MB/s stat for bench runs close to 1 second
MB/s was based on the number of iterations performed in a second, when
the iteration duration nears 1 second (1e9 ns), the resolution of the
MB/s stat decreases.
MB/s was based on the number of iterations performed in a second, when
the iteration duration nears 1 second (1e9 ns), the resolution of the
MB/s stat decreases.
This commit is the standard API stabilization commit for the 1.6 release cycle.
The list of issues and APIs below have all been through their cycle-long FCP and
the libs team decisions are listed below
Stabilized APIs
* `Read::read_exact`
* `ErrorKind::UnexpectedEof` (renamed from `UnexpectedEOF`)
* libcore -- this was a bit of a nuanced stabilization, the crate itself is now
marked as `#[stable]` and the methods appearing via traits for primitives like
`char` and `str` are now also marked as stable. Note that the extension traits
themeselves are marked as unstable as they're imported via the prelude. The
`try!` macro was also moved from the standard library into libcore to have the
same interface. Otherwise the functions all have copied stability from the
standard library now.
* The `#![no_std]` attribute
* `fs::DirBuilder`
* `fs::DirBuilder::new`
* `fs::DirBuilder::recursive`
* `fs::DirBuilder::create`
* `os::unix::fs::DirBuilderExt`
* `os::unix::fs::DirBuilderExt::mode`
* `vec::Drain`
* `vec::Vec::drain`
* `string::Drain`
* `string::String::drain`
* `vec_deque::Drain`
* `vec_deque::VecDeque::drain`
* `collections::hash_map::Drain`
* `collections::hash_map::HashMap::drain`
* `collections::hash_set::Drain`
* `collections::hash_set::HashSet::drain`
* `collections::binary_heap::Drain`
* `collections::binary_heap::BinaryHeap::drain`
* `Vec::extend_from_slice` (renamed from `push_all`)
* `Mutex::get_mut`
* `Mutex::into_inner`
* `RwLock::get_mut`
* `RwLock::into_inner`
* `Iterator::min_by_key` (renamed from `min_by`)
* `Iterator::max_by_key` (renamed from `max_by`)
Deprecated APIs
* `ErrorKind::UnexpectedEOF` (renamed to `UnexpectedEof`)
* `OsString::from_bytes`
* `OsStr::to_cstring`
* `OsStr::to_bytes`
* `fs::walk_dir` and `fs::WalkDir`
* `path::Components::peek`
* `slice::bytes::MutableByteVector`
* `slice::bytes::copy_memory`
* `Vec::push_all` (renamed to `extend_from_slice`)
* `Duration::span`
* `IpAddr`
* `SocketAddr::ip`
* `Read::tee`
* `io::Tee`
* `Write::broadcast`
* `io::Broadcast`
* `Iterator::min_by` (renamed to `min_by_key`)
* `Iterator::max_by` (renamed to `max_by_key`)
* `net::lookup_addr`
New APIs (still unstable)
* `<[T]>::sort_by_key` (added to mirror `min_by_key`)
Closes#27585Closes#27704Closes#27707Closes#27710Closes#27711Closes#27727Closes#27740Closes#27744Closes#27799Closes#27801
cc #27801 (doesn't close as `Chars` is still unstable)
Closes#28968
