libtest: Split HumanFormatter into {Pretty,Terse}
libtest: Fixed padding of benchmarks when not benchmarking
libtest: Fixed benchmarks' names not showing in terse-mode
libtest: Formatting
libtest: Json format now outputs failed tests' stdouts.
libtest: Json format now outputs failed tests' stdouts.
libtest: Json formatter now spews individiual events, not as an array
libtest: JSON fixes
libtest: Better JSON escaping
libtest: Test start event is printed on time
CloudABI doesn't make any distinction between TTYs and ordinary pipes.
While there, remove the redundant implementation used by Redox. It can
use the same stub function.
Tiny fixes to make compiletest work for CloudABI cross builds
I'm currently working toward getting a `src/ci/docker` container working to do isolated/automated builds and testing of `x86_64-unknown-cloudabi`. This is working pretty well, but still requires some fixes to `libtest` and `compiletest`. Here is the first set of fixes that I had to apply.
This structure doesn't seem to be used by libtest itself. It is used by
compiletest, but never passed on to anything externally. This makes it
easier to get the testing framework to work for CloudABI crossbuilds, as
CloudABI currently lacks PathBuf, which is used by TestPaths.
This commit adds a new target to the compiler: wasm32-unknown-unknown. This
target is a reimagining of what it looks like to generate WebAssembly code from
Rust. Instead of using Emscripten which can bring with it a weighty runtime this
instead is a target which uses only the LLVM backend for WebAssembly and a
"custom linker" for now which will hopefully one day be direct calls to lld.
Notable features of this target include:
* There is zero runtime footprint. The target assumes nothing exists other than
the wasm32 instruction set.
* There is zero toolchain footprint beyond adding the target. No custom linker
is needed, rustc contains everything.
* Very small wasm modules can be generated directly from Rust code using this
target.
* Most of the standard library is stubbed out to return an error, but anything
related to allocation works (aka `HashMap`, `Vec`, etc).
* Naturally, any `#[no_std]` crate should be 100% compatible with this new
target.
This target is currently somewhat janky due to how linking works. The "linking"
is currently unconditional whole program LTO (aka LLVM is being used as a
linker). Naturally that means compiling programs is pretty slow! Eventually
though this target should have a linker.
This target is also intended to be quite experimental. I'm hoping that this can
act as a catalyst for further experimentation in Rust with WebAssembly. Breaking
changes are very likely to land to this target, so it's not recommended to rely
on it in any critical capacity yet. We'll let you know when it's "production
ready".
---
Currently testing-wise this target is looking pretty good but isn't complete.
I've got almost the entire `run-pass` test suite working with this target (lots
of tests ignored, but many passing as well). The `core` test suite is still
getting LLVM bugs fixed to get that working and will take some time. Relatively
simple programs all seem to work though!
---
It's worth nothing that you may not immediately see the "smallest possible wasm
module" for the input you feed to rustc. For various reasons it's very difficult
to get rid of the final "bloat" in vanilla rustc (again, a real linker should
fix all this). For now what you'll have to do is:
cargo install --git https://github.com/alexcrichton/wasm-gc
wasm-gc foo.wasm bar.wasm
And then `bar.wasm` should be the smallest we can get it!
---
In any case for now I'd love feedback on this, particularly on the various
integration points if you've got better ideas of how to approach them!
Rationale:
We use --quiet mode when testing a PR in the CI. Also, we use `stamp` to
prefix every line with a timestamp. Previously, when testing in --quiet
mode, we will only print a dot for each test without any line breaks.
Combined with `stamp`, this means we'd need to wait for all tests to
complete before writing the output. On Travis CI, if we don't print
anything within 30 minutes, the job will be forcefully canceled. This
makes it very easy to spuriously-timeout when testing non-default images
like arm-android using the CI. This commit tries to workaround the issue
by printing a new line every 100 dots, forcing `stamp` to emit something
to reset Travis's countdown.
This change allows the user to add an `#[allow_fail]` attribute to
tests that will cause the test to compile & run, but if the test fails
it will not cause the entire test run to fail. The test output will
show the failure, but in yellow instead of red, and also indicate that
it was an allowed failure.
This commit deletes the in-tree `getopts` crate in favor of the crates.io-based
`getopts` crate. The main difference here is with a new builder-style API, but
otherwise everything else remains relatively standard.
When `RUST_BACKTRACE=1`, remove all frames after
`__rust_maybe_catch_panic`. Tested on `main`, threads, tests and
benches. Cleaning of the top of the stacktrace is let to a future PR.
Fixes#40201
See #41815
treat setting the number of test-threads to 0 as an error
It is currently possible to call `cargo test -- --test-threads=0` which will cause cargo to hang until aborted. This change will fix that and will report an appropriate error to the user.
Do not run outer setup part of benchmarks multiple times to fix issue 20142
Fix#20142
This is my first real rust code, so I expect the quality is quite bad. Please let me know in which ways it is horrible and I'll fix it.
Previously the whole benchmark function was rerun many times, but with this change, only the callback passed to iter is rerun. This improves performances by saving benchmark startup time. The setup used to be called a minimum of 101 times, and now only runs once.
I wasn't sure exactly what should be done for the case where iter is never called, so I left a FIXME for that: currently it does not error, and I added tests to cover that.
I have left the algorithm and statistics unchanged: I don't like how the minimum number of runs is 301 (that's bad for very slow benchmarks) but I consider such changes out of scope for this fix.
Running test with cargo test -- --test-threads=0 causes cargo to
hang as 0 is a valid usize. Adding zero threads as a special case
to the error handling.
Historically this was done to accommodate bugs in lints, but there hasn't been a
bug in a lint since this feature was added which the warnings affected. Let's
completely purge warnings from all our stages by denying warnings in all stages.
This will also assist in tracking down `stage0` code to be removed whenever
we're updating the bootstrap compiler.
This option lists all the tests and benchmarks a binary provides. By default the listing
is sent to stdout, but if --logfile is also specified, it is written there.
If filters are specified, they're applied before the output is emitted.
