Rationale:
We use --quiet mode when testing a PR in the CI. Also, we use `stamp` to
prefix every line with a timestamp. Previously, when testing in --quiet
mode, we will only print a dot for each test without any line breaks.
Combined with `stamp`, this means we'd need to wait for all tests to
complete before writing the output. On Travis CI, if we don't print
anything within 30 minutes, the job will be forcefully canceled. This
makes it very easy to spuriously-timeout when testing non-default images
like arm-android using the CI. This commit tries to workaround the issue
by printing a new line every 100 dots, forcing `stamp` to emit something
to reset Travis's countdown.
This change allows the user to add an `#[allow_fail]` attribute to
tests that will cause the test to compile & run, but if the test fails
it will not cause the entire test run to fail. The test output will
show the failure, but in yellow instead of red, and also indicate that
it was an allowed failure.
This commit deletes the in-tree `getopts` crate in favor of the crates.io-based
`getopts` crate. The main difference here is with a new builder-style API, but
otherwise everything else remains relatively standard.
When `RUST_BACKTRACE=1`, remove all frames after
`__rust_maybe_catch_panic`. Tested on `main`, threads, tests and
benches. Cleaning of the top of the stacktrace is let to a future PR.
Fixes#40201
See #41815
treat setting the number of test-threads to 0 as an error
It is currently possible to call `cargo test -- --test-threads=0` which will cause cargo to hang until aborted. This change will fix that and will report an appropriate error to the user.
Do not run outer setup part of benchmarks multiple times to fix issue 20142
Fix#20142
This is my first real rust code, so I expect the quality is quite bad. Please let me know in which ways it is horrible and I'll fix it.
Previously the whole benchmark function was rerun many times, but with this change, only the callback passed to iter is rerun. This improves performances by saving benchmark startup time. The setup used to be called a minimum of 101 times, and now only runs once.
I wasn't sure exactly what should be done for the case where iter is never called, so I left a FIXME for that: currently it does not error, and I added tests to cover that.
I have left the algorithm and statistics unchanged: I don't like how the minimum number of runs is 301 (that's bad for very slow benchmarks) but I consider such changes out of scope for this fix.
Running test with cargo test -- --test-threads=0 causes cargo to
hang as 0 is a valid usize. Adding zero threads as a special case
to the error handling.
Historically this was done to accommodate bugs in lints, but there hasn't been a
bug in a lint since this feature was added which the warnings affected. Let's
completely purge warnings from all our stages by denying warnings in all stages.
This will also assist in tracking down `stage0` code to be removed whenever
we're updating the bootstrap compiler.
This option lists all the tests and benchmarks a binary provides. By default the listing
is sent to stdout, but if --logfile is also specified, it is written there.
If filters are specified, they're applied before the output is emitted.
Filter matching is by substring by default. This makes it impossible
to run a single test if its name is a substring of some other test.
For example, its not possible to run just "mymod::test" with these
tests:
mymod::test
mymod::test1
mymod::test_module::moretests
You could declare by convention that no test has a name that's a
substring of another test, but that's not really practical.
This PR adds the "--exact" flag, to make filter matching exactly
match the complete name.
Most of the Rust community agrees that the vec! macro is clearer when
called using square brackets [] instead of regular brackets (). Most of
these ocurrences are from before macros allowed using different types of
brackets.
There is one left unchanged in a pretty-print test, as the pretty
printer still wants it to have regular brackets.
* Don't spawn two threads for all tests, just one now that `catch_unwind` is
stable.
* Remove usage of the unstable `box` keyword
* Remove usage of the unstable `FnBox` trait
libtest: add a --skip flag to the test runner
This flag takes a FILTER argument and instructs the test runner to skip
the tests whose names contain the word FILTER. --skip can be used
several times.
---
My motivation for submitting this is that while using [smoke] to run `std` unit tests for cross
targets I found that a few of the tests always fail due to limitations in QEMU (it can't handle too
many threads) and I'd like to skip these problematic tests from the command line to be able to run
the rest of the unit tests.
[smoke]: https://github.com/japaric/smoke
I know there is another mechanism to skip tests: `#[ignore]` but this doesn't work in my use case
because I can't (easily) modify the source of the standard libraries to `#[ignore]` some tests. And
even if I could, the change would involve conditionally ignoring some tests for some targets but
that's not a perfect solution because those tests should pass if executed on real hardware so they
should not be `#[ignored]` in that scenario.
r? @alexcrichton
cc @brson
This flag takes a FILTER argument and instructs the test runner to skip
the tests whose names contain the word FILTER. --skip can be used
several times.
Add --test-threads option to test binaries
This change allows parallelism of test runs to be specified by a
command line flag names --test-threads in addition to the existing
environment variable RUST_TEST_THREADS. Fixes#25636.
This change allows parallelism of test runs to be specified by a
command line flag names --test-threads in addition to the existing
environment variable RUST_TEST_THREADS. Fixes#25636.
This is a spiritual succesor to #34268/8531d581, in which we replaced a
number of matches of None to the unit value with `if let` conditionals
where it was judged that this made for clearer/simpler code (as would be
recommended by Manishearth/rust-clippy's `single_match` lint). The same
rationale applies to matches of None to the empty block.
