Commit Graph

114 Commits

Author SHA1 Message Date
Mark Rousskov
22c3a71de1 Switch bootstrap cfgs 2022-02-25 08:00:52 -05:00
Antonio Yang
bb3b5574cd Include ignore message in libtest output
As an example:

    #[test]
    #[ignore = "not yet implemented"]
    fn test_ignored() {
        ...
    }

Will now render as:

    running 2 tests
    test tests::test_ignored ... ignored, not yet implemented

    test result: ok. 1 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.00s
2022-02-24 17:36:36 -05:00
Matthias Krüger
d855121a44
Rollup merge of #93479 - smoelius:master, r=yaahc
Use `optflag` for `--report-time`

Essentially, what is described here:
https://github.com/rust-lang/rust/issues/64888#issuecomment-1008047228

There is one difference. The comment proposes to add a
`--report-time-color` option. This change instead uses libtest's
existing `--color` option for that purpose.
2022-02-17 06:29:59 +01:00
Mara Bos
252ff5ead0
Rollup merge of #93416 - name1e5s:chore/remove_allow_fail, r=m-ou-se
remove `allow_fail` test flag

close #93345
2022-02-07 14:08:34 +00:00
Yuki Okushi
95efb2b578
Rollup merge of #93600 - last-partizan:fix-junit-formatter, r=yaahc
fix: Remove extra newlines from junit output

This PR fixes extra newline in junit output https://github.com/rust-lang/rust/issues/93454
2022-02-03 22:20:28 +09:00
Serg Tereshchenko
088474a408 fix: Remove extra newlines from junit output 2022-02-02 22:14:03 +02:00
Samuel E. Moelius III
96d96a7ac4 Use optflag for --report-time
Essentially, what is described here:
https://github.com/rust-lang/rust/issues/64888#issuecomment-1008047228

There is one difference. The comment proposes to add a
`--report-time-color` option. This change instead uses libtest's
existing `--color` option for that purpose.
2022-01-30 10:25:40 -05:00
Jane Lusby
91ffbc43b1 Change Termination::report return type to ExitCode 2022-01-28 12:53:36 -08:00
yuhaixin.hx
0b8c9fbf0b add allow_fail field in TestDesc to pass check 2022-01-28 19:40:49 +08:00
yuhaixin.hx
6562069ebe remove allow_fail test flag 2022-01-28 18:31:49 +08:00
Lucas Kent
08829853d3 eplace usages of vec![].into_iter with [].into_iter 2022-01-09 14:09:25 +11:00
bors
23ce5fc465 Auto merge of #92068 - fee1-dead:libcore2021, r=m-ou-se
Switch all libraries to the 2021 edition

The fix for https://github.com/rust-lang/rust/issues/88638#issuecomment-996620107 is to simply add const-stability for these functions.

r? `@m-ou-se`

Closes #88638.
2022-01-08 21:41:48 +00:00
Yoshua Wuyts
3632f41c78 Stabilize #[feature(available_parallelism)] 2022-01-07 01:07:10 +01:00
bjorn3
e015b9ee80 Remove unused allow deprecated 2021-12-29 18:06:01 +01:00
bjorn3
f710bc5fc0 Remove unused feature gates from libtest 2021-12-29 16:36:22 +01:00
bjorn3
1ea4810c84 Remove #![crate_name] attribute from libtest
The crate name is already set in Cargo.toml. The comment says there is
some logic in the compiler that reads #![crate_name] and not
--crate-name, but I can't find it. Removing it seems to work fine.
2021-12-29 16:36:08 +01:00
bjorn3
75d8339cdd Replace TDynBenchFn with Fn(&mut Bencher) 2021-12-29 16:17:50 +01:00
Deadbeef
06a1c14d52
Switch all libraries to the 2021 edition 2021-12-23 19:03:47 +08:00
Alex Crichton
d2a3c24a95 Update more rustc/libtest things for wasm64
* Add wasm64 variants for inline assembly along the same lines as wasm32
* Update a few directives in libtest to check for `target_family`
  instead of `target_arch`
* Update some rustc codegen and typechecks specialized for wasm32 to
  also work for wasm64.
2021-11-10 08:35:42 -08:00
Matthias Krüger
28ef4169cc clippy::perf fixes 2021-11-04 21:07:56 +01:00
Jubilee
37f17bca7c
Rollup merge of #89082 - smoelius:master, r=kennytm
Implement #85440 (Random test ordering)

This PR adds `--shuffle` and `--shuffle-seed` options to `libtest`. The options are similar to the [`-shuffle` option](c894b442d1/src/testing/testing.go (L1482-L1499)) that was recently added to Go.

Here are the relevant parts of the help message:
```
        --shuffle       Run tests in random order
        --shuffle-seed SEED
                        Run tests in random order; seed the random number
                        generator with SEED
...
By default, the tests are run in alphabetical order. Use --shuffle or set
RUST_TEST_SHUFFLE to run the tests in random order. Pass the generated
"shuffle seed" to --shuffle-seed (or set RUST_TEST_SHUFFLE_SEED) to run the
tests in the same order again. Note that --shuffle and --shuffle-seed do not
affect whether the tests are run in parallel.
```
Is an RFC needed for this?
2021-10-07 20:26:12 -07:00
Manish Goregaokar
b4615b5bf9
Rollup merge of #89324 - yoshuawuyts:hardware-parallelism, r=m-ou-se
Rename `std:🧵:available_conccurrency` to `std:🧵:available_parallelism`

_Tracking issue: https://github.com/rust-lang/rust/issues/74479_

This PR renames  `std:🧵:available_conccurrency` to `std:🧵:available_parallelism`.

## Rationale

The API was initially named `std:🧵:hardware_concurrency`, mirroring the [C++ API of the same name](https://en.cppreference.com/w/cpp/thread/thread/hardware_concurrency). We eventually decided to omit any reference to the word "hardware" after [this comment](https://github.com/rust-lang/rust/pull/74480#issuecomment-662045841). And so we ended up with `available_concurrency` instead.

---

For a talk I was preparing this week I was reading through ["Understanding and expressing scalable concurrency" (A. Turon, 2013)](http://aturon.github.io/academic/turon-thesis.pdf), and the following passage stood out to me (emphasis mine):

> __Concurrency is a system-structuring mechanism.__ An interactive system that deals with disparate asynchronous events is naturally structured by division into concurrent threads with disparate responsibilities. Doing so creates a better fit between problem and solution, and can also decrease the average latency of the system by preventing long-running computations from obstructing quicker ones.

> __Parallelism is a resource.__ A given machine provides a certain capacity for parallelism, i.e., a bound on the number of computations it can perform simultaneously. The goal is to maximize throughput by intelligently using this resource. For interactive systems, parallelism can decrease latency as well.

_Chapter 2.1: Concurrency is not Parallelism. Page 30._

---

_"Concurrency is a system-structuring mechanism. Parallelism is a resource."_ — It feels like this accurately captures the way we should be thinking about these APIs. What this API returns is not "the amount of concurrency available to the program" which is a property of the program, and thus even with just a single thread is effectively unbounded. But instead it returns "the amount of _parallelism_ available to the program", which is a resource hard-constrained by the machine's capacity (and can be further restricted by e.g. operating systems).

That's why I'd like to propose we rename this API from `available_concurrency` to `available_parallelism`. This still meets the criteria we previously established of not attempting to define what exactly we mean by "hardware", "threads", and other such words. Instead we only talk about "concurrency" as an abstract resource available to our program.

r? `@joshtriplett`
2021-10-06 12:33:17 -07:00
Samuel E. Moelius III
32b6ac5b44 Check allow_unstable before checking environment variables 2021-09-30 12:57:34 -04:00
Samuel E. Moelius III
a6738c7231 Add tests 2021-09-29 21:51:59 -04:00
Samuel E. Moelius III
fa23d4fe93 Implement #85440 2021-09-29 21:51:46 -04:00
Guillaume Gomez
e601554dc0
Rollup merge of #89235 - yaahc:junit-formatting, r=kennytm
make junit output more consistent with default format

The default format of libtest includes new-lines between each section to ensure the label output from cargo is on it's own line

<pre><font color="#A1B56C"><b>❯</b></font> <font color="#A1B56C">cargo</font><font color="#D8D8D8"> </font><font color="#A1B56C">test</font>
<font color="#A1B56C"><b>   Compiling</b></font> test-test v0.1.0 (/home/jlusby/tmp/test-test)
<font color="#A1B56C"><b>    Finished</b></font> test [unoptimized + debuginfo] target(s) in 0.59s
<font color="#A1B56C"><b>     Running</b></font> unittests (target/debug/deps/test_test-639f369234319c09)

running 1 test
test tests::it_works ... <font color="#A1B56C">ok</font>

test result: <font color="#A1B56C">ok</font>. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

<font color="#A1B56C"><b>   Doc-tests</b></font> test-test

running 0 tests

test result: <font color="#A1B56C">ok</font>. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

</pre>

But when the junit outputter was added to libtest these newlines were omitted, resulting in some "fun" output when run via cargo.

Note the `Doc-tests` text at the end of the first line of xml.

<pre><font color="#A1B56C"><b>❯</b></font> <font color="#A1B56C">cargo</font><font color="#D8D8D8"> </font><font color="#A1B56C">test</font><font color="#D8D8D8"> </font><font color="#A1B56C">--</font><font color="#D8D8D8"> </font><font color="#A1B56C">-Zunstable-options</font><font color="#D8D8D8"> </font><font color="#A1B56C">--format</font><font color="#D8D8D8"> </font><font color="#A1B56C">junit</font>
<font color="#A1B56C"><b>    Finished</b></font> test [unoptimized + debuginfo] target(s) in 0.00s
<font color="#A1B56C"><b>     Running</b></font> unittests (target/debug/deps/test_test-639f369234319c09)
&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt;&lt;testsuites&gt;&lt;testsuite name=&quot;test&quot; package=&quot;test&quot; id=&quot;0&quot; errors=&quot;0&quot; failures=&quot;0&quot; tests=&quot;1&quot; skipped=&quot;0&quot; &gt;&lt;testcase classname=&quot;tests&quot; name=&quot;it_works&quot; time=&quot;0&quot;/&gt;&lt;system-out/&gt;&lt;system-err/&gt;&lt;/testsuite&gt;&lt;/testsuites&gt;<font color="#A1B56C"><b>   Doc-tests</b></font> test-test
&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt;&lt;testsuites&gt;&lt;testsuite name=&quot;test&quot; package=&quot;test&quot; id=&quot;0&quot; errors=&quot;0&quot; failures=&quot;0&quot; tests=&quot;0&quot; skipped=&quot;0&quot; &gt;&lt;system-out/&gt;&lt;system-err/&gt;&lt;/testsuite&gt;&lt;/testsuites&gt;

</pre>

After this PR the junit output includes the same style of newlines as the pretty format

<pre><font color="#A1B56C"><b>❯</b></font> <font color="#A1B56C">cargo</font><font color="#D8D8D8"> </font><font color="#A1B56C">test</font><font color="#D8D8D8"> </font><font color="#A1B56C">--</font><font color="#D8D8D8"> </font><font color="#A1B56C">-Zunstable-options</font><font color="#D8D8D8"> </font><font color="#A1B56C">--format</font><font color="#D8D8D8"> </font><font color="#A1B56C">junit</font>
<font color="#A1B56C"><b>   Compiling</b></font> test-test v0.1.0 (/home/jlusby/tmp/test-test)
<font color="#A1B56C"><b>    Finished</b></font> test [unoptimized + debuginfo] target(s) in 0.39s
<font color="#A1B56C"><b>     Running</b></font> unittests (target/debug/deps/test_test-42c2320bb9450c69)

&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt;&lt;testsuites&gt;&lt;testsuite name=&quot;test&quot; package=&quot;test&quot; id=&quot;0&quot; errors=&quot;0&quot; failures=&quot;0&quot; tests=&quot;1&quot; skipped=&quot;0&quot; &gt;&lt;testcase classname=&quot;tests&quot; name=&quot;it_works&quot; time=&quot;0&quot;/&gt;&lt;system-out/&gt;&lt;system-err/&gt;&lt;/testsuite&gt;&lt;/testsuites&gt;

<font color="#A1B56C"><b>   Doc-tests</b></font> test-test

&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt;&lt;testsuites&gt;&lt;testsuite name=&quot;test&quot; package=&quot;test&quot; id=&quot;0&quot; errors=&quot;0&quot; failures=&quot;0&quot; tests=&quot;0&quot; skipped=&quot;0&quot; &gt;&lt;system-out/&gt;&lt;system-err/&gt;&lt;/testsuite&gt;&lt;/testsuites&gt;

</pre>
2021-09-28 20:00:15 +02:00
Yoshua Wuyts
6cc91cb3d8 Rename std:🧵:available_onccurrency to std:🧵:available_parallelism 2021-09-28 14:59:33 +02:00
Jane Lusby
0911069feb
Apply suggestions from code review
Co-authored-by: kennytm <kennytm@gmail.com>
2021-09-27 14:50:35 -07:00
Luca Barbato
160b93903c Expose the std_detect env_override feature 2021-09-25 20:30:25 +02:00
Jane Lusby
7779eb74c8 make junit output more consistent with default format 2021-09-24 14:45:09 -07:00
Matthias Krüger
c1e96085d3 don't clone types that are Copy (clippy::clone_on_copy) 2021-09-11 10:18:56 +02:00
Fabian Wolff
79adda930f Ignore automatically derived impls of Clone and Debug in dead code analysis 2021-09-09 19:49:07 +02:00
Jade
3cf820e17d rfc3052: Remove authors field from Cargo manifests
Since RFC 3052 soft deprecated the authors field anyway, hiding it from
crates.io, docs.rs, and making Cargo not add it by default, and it is
not generally up to date/useful information, we should remove it from
crates in this repo.
2021-07-29 14:56:05 -07:00
Charles Lew
4486795d02 Remove unused stuff and switch to pub(crate) whenever possible. 2021-07-18 22:00:41 +08:00
Charles Lew
950f569c91 Fix compilation errors. 2021-07-18 20:51:47 +08:00
Charles Lew
9c11113b4f Move library/term/src to library/test/src/term/. 2021-07-18 20:49:18 +08:00
Mark Rousskov
06661ba759 Update to new bootstrap compiler 2021-06-28 11:30:49 -04:00
Smitty
bdfcb88e8b Use HTTPS links where possible 2021-06-23 16:26:46 -04:00
Deadbeef
e4b3131584
Use as_secs_f64 in JunitFormatter 2021-06-17 03:23:17 +08:00
bors
3740ba2a7d Auto merge of #84863 - ABouttefeux:libtest, r=m-ou-se
Show test type during prints

Test output can sometimes be confusing. For example doctest with the no_run argument are displayed the same way than test that are run.

During #83857 I got the feedback that test output can be confusing.

For the moment test output is
```
test $DIR/test-type.rs - f (line 12) ... ignored
test $DIR/test-type.rs - f (line 15) ... ok
test $DIR/test-type.rs - f (line 21) ... ok
test $DIR/test-type.rs - f (line 6) ... ok
```

I propose to change output by indicating the test type as
```
test $DIR/test-type.rs - f (line 12) ... ignored
test $DIR/test-type.rs - f (line 15) - compile ... ok
test $DIR/test-type.rs - f (line 21) - compile fail ... ok
test $DIR/test-type.rs - f (line 6) ... ok
```
by indicating the test type after the test name (and in the case of doctest after the function name and line) and before the "...".

------------

Note: this is a proof of concept, the implementation is probably not optimal as the properties added in `TestDesc` are only use in the display and does not represent actual change of behavior, maybe `TestType::DocTest` could have fields
2021-06-06 09:13:59 +00:00
Joshua Nelson
7411a9e7cc rustdoc: link to stable/beta docs consistently in documentation
## User-facing changes

- Intra-doc links to primitives that currently go to rust-lang.org/nightly/std/primitive.x.html will start going to channel that rustdoc was built with. Nightly will continue going to /nightly; Beta will link to /beta; stable compilers will link to /1.52.1 (or whatever version they were built as).
- Cross-crate links from std to core currently go to /nightly unconditionally. They will start going to /1.52.0 on stable channels (but remain the same on nightly channels).
- Intra-crate links from std to std (or core to core) currently go to the same URL they are hosted at; they will continue to do so. Notably, this is different from everything else because it can preserve the distinction between /stable and /1.52.0 by using relative links.

Note that "links" includes both intra-doc links and rustdoc's own
automatically generated hyperlinks.

 ## Implementation changes

- Update the testsuite to allow linking to /beta and /1.52.1 in docs
- Use an html_root_url for the standard library that's dependent on the channel

  This avoids linking to nightly docs on stable.

- Update rustdoc to use channel-dependent links for primitives from an
  unknown crate

- Set DOC_RUST_LANG_ORG_CHANNEL from bootstrap to ensure it's in sync
- Include doc.rust-lang.org in the channel
2021-06-04 14:18:21 -04:00
bors
1c6868aa21 Auto merge of #84568 - andoriyu:libtest/junit_formatter, r=yaahc
feat(libtest): Add JUnit formatter

tracking issue: https://github.com/rust-lang/rust/issues/85563

Add an alternative formatter to `libtest`. Formatter produces valid xml that later can be interpreted as JUnit report.

Caveats:

- `timestamp` is required by schema, but every viewer/parser ignores it. Attribute is not set to avoid depending on chrono;
- Running all "suits" (unit tests, doc-tests and integration tests) will produce a mess;
- I couldn't find a way to get integration test binary name, so it's just goes by "integration";

Sample output for unit tests (pretty printed by 3rd party tool):
```
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="test" package="test" id="0" errors="0" failures="0" tests="13" skipped="1">
    <testcase classname="results::tests" name="test_completed_bad" time="0"/>
    <testcase classname="results::tests" name="suite_started" time="0"/>
    <testcase classname="results::tests" name="suite_ended_ok" time="0"/>
    <testcase classname="results::tests" name="suite_ended_bad" time="0"/>
    <testcase classname="junit::tests" name="test_failed_output" time="0"/>
    <testcase classname="junit::tests" name="test_simple_output" time="0"/>
    <testcase classname="junit::tests" name="test_multiple_outputs" time="0"/>
    <testcase classname="results::tests" name="test_completed_ok" time="0"/>
    <testcase classname="results::tests" name="test_stared" time="0"/>
    <testcase classname="junit::tests" name="test_generate_xml_no_error_single_testsuite" time="0"/>
    <testcase classname="results::tests" name="test_simple_output" time="0"/>
    <testcase classname="test" name="should_panic" time="0"/>
    <system-out/>
    <system-err/>
  </testsuite>
</testsuites>
```

Sample output for integration tests (pretty printed by 3rd party tool):

```
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="test" package="test" id="0" errors="0" failures="0" tests="1" skipped="0">
    <testcase classname="integration" name="test_add" time="0"/>
    <system-out/>
    <system-err/>
  </testsuite>
</testsuites>
```

Sample output for Doc-tests (pretty printed by 3rd party tool):

```
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="test" package="test" id="0" errors="0" failures="0" tests="1" skipped="0">
    <testcase classname="src/lib.rs" name="(line 2)" time="0"/>
    <system-out/>
    <system-err/>
  </testsuite>
</testsuites>
```
2021-05-27 21:14:55 +00:00
Aliénore Bouttefeux
6de13c3ffc change based on review 2021-05-18 18:18:28 +02:00
Aliénore Bouttefeux
3d81806352 remove mode for run and ignore tests 2021-05-16 22:33:41 +02:00
Alan Egerton
67e8f12307
Expose Concurrent (private type in public i'face) 2021-05-14 13:28:56 +01:00
Aliénore Bouttefeux
f6b8b78063 add bootstrap cfg 2021-05-09 13:37:09 +02:00
Aliénore Bouttefeux
6e99cb3989 change based on review 2021-05-03 20:17:15 +02:00
Aliénore Bouttefeux
347ed001e8 proof of concept add test type on prints 2021-05-03 15:22:19 +02:00
Andrey Cherkashin
9f83e2290a Better output for junit formatter 2021-04-30 17:16:09 -07:00
Andrey Cherkashin
4b4d06ae82
Update junit.rs 2021-04-26 11:42:13 -07:00