Allow building rustdoc without first building rustc (MVP)
## Motivation
The compile times for rustc are extremely long and a major issue for
recruiting new contributors to rustdoc. People interested in joining
often give up after running into issues with submodules or python
versions. stage1 rustdoc fundamentally doesn't care about bootstrapping
or stages, it just needs `rustc_private` available.
## Summary of Changes
- Add an opt-in `[rust] download_rustc` option
- Determine the version of the compiler to download using `log --author=bors`
- Do no work for any component other than `Rustdoc` for any stage. Instead, copy the CI artifacts from the downloaded sysroot stage0/ to stage0-sysroot/ or stage1/ in `Sysroot`. This is done with an `ENABLE_DOWNLOAD_STAGE1` constant which is off by default.
- Don't download different versions of rustfmt or cargo - those should still use the beta version (rustfmt especially).
The vast majority of work is done in bootstrap.py, which downloads the artifacts and extracts them to stage0/ in place of the beta compiler. Rustbuild just takes care of copying the artifacts to stage1 if necessary.
## Future work
- I turned off verification for the commit tarballs because the .sha256 URLs gave a 404. This seems not ideal, it would be nice to start signing them.
- This will break if you rebase an old enough branch (I think commits are kept at most 160 days?). This doesn't need to be supported, but it would be nice to give a reasonable error. https://github.com/rust-lang/rust/pull/79540#issuecomment-751481291
- Right now, every time you rebase, stage0 tools (bootstrap, tidy, ...) will have to be recompiled. Additionally running `x.py setup tools` will compile rustbuild twice. Instead, this should download a separate beta compiler for stage0 and only use CI artifacts for stage1 onward. https://github.com/rust-lang/rust/pull/79540#issuecomment-757047321
- Add `x.py setup tools` to enable this conveniently (it doesn't make sense to use this for compiler developers). cb5d8c8522
- Compile a new version of tracing so that rustdoc still gets debug logging (since CI artifacts always disable `debug` and `trace` logging). https://github.com/rust-lang/rust/pull/79540#issuecomment-742756411, 6a5d512420
- Right now only rustdoc is ever rebuilt. This is not ideal and should probably at least compile compiler tools (rustfmt, clippy, miri). https://github.com/rust-lang/rust/pull/79540#issuecomment-775634693
- Using `git log --author=bors` sometimes breaks. This should use `git merge-base` instead. https://github.com/rust-lang/rust/pull/79540#discussion_r572572280
- It would be nice to support cross-compiling the standard library. Right now this gives an assertion failure I think.
Some of this work has already been done in (the history for) 673476c785.
Fix derived PartialOrd operators
The derived implementation of `partial_cmp` compares matching fields one
by one, stopping the computation when the result of a comparison is not
equal to `Some(Equal)`.
On the other hand the derived implementation for `lt`, `le`, `gt` and
`ge` continues the computation when the result of a field comparison is
`None`, consequently those operators are not transitive and inconsistent
with `partial_cmp`.
Fix the inconsistency by using the default implementation that fall-backs
to the `partial_cmp`. This also avoids creating very deeply nested
closures that were quite costly to compile.
Fixes#81373.
Helps with #81278, #80118.
Rollup of 11 pull requests
Successful merges:
- #72209 (Add checking for no_mangle to unsafe_code lint)
- #80732 (Allow Trait inheritance with cycles on associated types take 2)
- #81697 (Add "every" as a doc alias for "all".)
- #81826 (Prefer match over combinators to make some Box methods inlineable)
- #81834 (Resolve typedef in HashMap lldb pretty-printer only if possible)
- #81841 ([rustbuild] Output rustdoc-json-types docs )
- #81849 (Expand the docs for ops::ControlFlow a bit)
- #81876 (parser: Fix panic in 'const impl' recovery)
- #81882 (⬆️ rust-analyzer)
- #81888 (Fix pretty printer macro_rules with semicolon.)
- #81896 (Remove outdated comment in windows' mutex.rs)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
- Use the same compiler for stage0 and stage1. This should be fixed at
some point (so bootstrap isn't constantly rebuilt).
- Make sure `x.py build` and `x.py check` work.
- Use `git merge-base` to determine the most recent commit to download.
- Copy stage0 to the various sysroots in `Sysroot`, and delegate to
Sysroot in Assemble. Leave all other code unchanged.
- Rename date -> key
This can also be a commit hash, so 'date' is no longer a good name.
- Add the commented-out option to config.toml.example
- Disable all steps by default when `download-rustc` is enabled
Most steps don't make sense when downloading a compiler, because they'll
be pre-built in the sysroot. Only enable the ones that might be useful,
in particular Rustdoc and all `check` steps.
At some point, this should probably enable other tools, but rustdoc is
enough to test out `download-rustc`.
- Don't print 'Skipping' twice in a row
Bootstrap forcibly enables a dry run if it isn't already set, so
previously it would print the message twice:
```
Skipping bootstrap::compile::Std because it is not enabled for `download-rustc`
Skipping bootstrap::compile::Std because it is not enabled for `download-rustc`
```
Now it correctly only prints once.
## Future work
- Add FIXME about supporting beta commits
- Debug logging will never work. This should be fixed.
Remove outdated comment in windows' mutex.rs
After https://github.com/rust-lang/rust/pull/81250, this `Mutex` no longer falls back to the `ReentrantMutex` implementation, so this comment is no longer relevant.
Fix pretty printer macro_rules with semicolon.
The pretty printer was not including the trailing semicolon for a macro_rules definition that used parenthesis or brackets, which results in invalid code. This adds the semicolon in those two cases.
parser: Fix panic in 'const impl' recovery
The panic happens when in recovery parsing a full `impl`
(`parse_item_impl`) fails and we drop the `DiagnosticBuilder` for the
recovery suggestion and return the `parse_item_impl` error.
We now raise the original error "expected identifier found `impl`" when
parsing the `impl` fails.
Note that the regression test is slightly simplified version of the
original repro in #81806, to make the error output smaller and more
resilient to unrelated changes in parser error messages.
Fixes#81806
Expand the docs for ops::ControlFlow a bit
Since I was writing some examples for an RFC anyway.
And I almost made the mistake of reordering the variants, so added a note and a test about that.
Resolve typedef in HashMap lldb pretty-printer only if possible
Fixes https://github.com/rust-lang/rust/issues/81814
Previously, `GetTypedefedType` was invoked unconditionally.
But this did not work in case of `rust-lldb` without Rust patches since there was no typedef.
Prefer match over combinators to make some Box methods inlineable
Hopefully this patch would make two snippets generated identical code: <https://rust.godbolt.org/z/fjrj4E>.
Allow Trait inheritance with cycles on associated types take 2
This reverts the revert of #79209 and fixes the ICEs that's occasioned by that PR exposing some problems that are addressed in #80648 and #79811.
For easier review I'd say, check only the last commit, the first one is just a revert of the revert of #79209 which was already approved.
This also could be considered part or the actual fix of #79560 but I guess for that to be closed and fixed completely we would need to land #80648 and #79811 too.
r? `@nikomatsakis`
cc `@Aaron1011`
The derived implementation of `partial_cmp` compares matching fields one
by one, stopping the computation when the result of a comparison is not
equal to `Some(Equal)`.
On the other hand the derived implementation for `lt`, `le`, `gt` and
`ge` continues the computation when the result of a field comparison is
`None`, consequently those operators are not transitive and inconsistent
with `partial_cmp`.
Fix the inconsistency by using the default implementation that fall-backs
to the `partial_cmp`. This also avoids creating very deeply nested
closures that were quite costly to compile.
parse_format: treat r" as a literal
This PR changes `format_args!` internal parsing machinery to treat raw strings starting `r"` as a literal.
Currently `"` and `r#` are recognised as valid starting combinations for string literals, but `r"` is not.
This was noticed when debugging https://github.com/rust-lang/rust/issues/67984#issuecomment-753413156
As well as fixing the behavior observed in that comment, this improves diagnostic spans for `r"` formatting strings.
improve error message for disallowed ptr-to-int casts in const eval
Improves an error message as [suggested](https://github.com/rust-lang/rust/issues/80875#issuecomment-762754580) in #80875.
Does the wording make enough sense? I tried to follow precedent for error message style while maintaining brevity.
It seems like the rest of the `ConstEvalErrKind::NeedsRfc` error messages could be improved as well. I could give that a go if this approach works.
Closes#80875
faster few span methods
Touched few methods, so it should be (hopefully) faster.
First two changes: instead splitting string from start and taking only last piece, split it from the end.
Last: swapped conditions, to first check boolean parameter.
libtest: allow multiple filters
Libtest ignores any filters after the first. This changes it so that if multiple filters are passed, it will test against all of them.
This also affects compiletest to do the same.
Closes#30422
BTree: remove Ord bound where it is absent elsewhere
Some btree methods don't really need an Ord bound and don't have one, while some methods that more obviously don't need it, do have one.
An example of the former is `iter`, even though it explicitly exposes the work of the Ord implementation (["sorted by key"](https://doc.rust-lang.org/std/collections/struct.BTreeMap.html#method.iter) - but I'm not suggesting it should have the Ord bound). An example of the latter is `new`, which doesn't involve any keys whatsoever.
The panic happens when in recovery parsing a full `impl`
(`parse_item_impl`) fails and we drop the `DiagnosticBuilder` for the
recovery suggestion and return the `parse_item_impl` error.
We now raise the original error "expected identifier found `impl`" when
parsing the `impl` fails.
Note that the regression test is slightly simplified version of the
original repro in #81806, to make the error output smaller and more
resilient to unrelated changes in parser error messages.
Fixes#81806
Stabilize remaining integer methods as `const fn`
This pull request stabilizes the following methods as `const fn`:
- `i*::checked_div`
- `i*::checked_div_euclid`
- `i*::checked_rem`
- `i*::checked_rem_euclid`
- `i*::div_euclid`
- `i*::overflowing_div`
- `i*::overflowing_div_euclid`
- `i*::overflowing_rem`
- `i*::overflowing_rem_euclid`
- `i*::rem_euclid`
- `i*::wrapping_div`
- `i*::wrapping_div_euclid`
- `i*::wrapping_rem`
- `i*::wrapping_rem_euclid`
- `u*::checked_div`
- `u*::checked_div_euclid`
- `u*::checked_rem`
- `u*::checked_rem_euclid`
- `u*::div_euclid`
- `u*::overflowing_div`
- `u*::overflowing_div_euclid`
- `u*::overflowing_rem`
- `u*::overflowing_rem_euclid`
- `u*::rem_euclid`
- `u*::wrapping_div`
- `u*::wrapping_div_euclid`
- `u*::wrapping_rem`
- `u*::wrapping_rem_euclid`
These can all be implemented on the current stable (1.49). There are two unstable details: const likely/unlikely and unchecked division/remainder. Both of these are for optimizations, and are in no way required to make the methods function; there is no exposure of these details publicly. Per comments below, it seems best practice is to stabilize the intrinsics. As such, `intrinsics::unchecked_div` and `intrinsics::unchecked_rem` have been stabilized as `const` as part of this pull request as well. The methods themselves remain unstable.
I believe part of the reason these were not stabilized previously was the behavior around division by 0 and modulo 0. After testing on nightly, the diagnostic for something like `const _: i8 = 5i8 % 0i8;` is similar to that of `const _: i8 = 5i8.rem_euclid(0i8);` (assuming the appropriate feature flag is enabled). As such, I believe these methods are ready to be stabilized as `const fn`.
This pull request represents the final methods mentioned in #53718. As such, this PR closes#53718.
`@rustbot` modify labels to +A-const-fn, +T-libs
Implement `--extern-location`
This PR implements `--extern-location` as a followup to #72342 as part of the implementation of #57274. The goal of this PR is to allow rustc, in coordination with the build system, to present a useful diagnostic about how to remove an unnecessary dependency from a dependency specification file (eg Cargo.toml).
EDIT: Updated to current PR state.
The location is specified for each named crate - that is, for a given `--extern foo[=path]` there can also be `--extern-location foo=<location>`. It supports ~~three~~ two styles of location:
~~1. `--extern-location foo=file:<path>:<line>` - a file path and line specification
1. `--extern-location foo=span:<path>:<start>:<end>` - a span specified as a file and start and end byte offsets~~
1. `--extern-location foo=raw:<anything>` - a raw string which is included in the output
1. `--extern-location foo=json:<anything>` - an arbitrary Json structure which is emitted via Json diagnostics in a `tool_metadata` field.
~~1 & 2 are turned into an internal `Span`, so long as the path exists and is readable, and the location is meaningful (within the file, etc). This is used as the `Span` for a fix suggestion which is reported like other fix suggestions.~~
`raw` and `json` are for the case where the location isn't best expressed as a file and location within that file. For example, it could be a rule name and the name of a dependency within that rule. `rustc` makes no attempt to parse the raw string, and simply includes it in the output diagnostic text. `json` is only included in json diagnostics. `raw` is emitted as text and also as a json string in `tool_metadata`.
If no `--extern-location` option is specified then it will emit a default json structure consisting of `{"name": name, "path": path}` corresponding to the name and path in `--extern name=path`.
This is a prototype/RFC to make some of the earlier conversations more concrete. It doesn't stand on its own - it's only useful if implemented by Cargo and other build systems. There's also a ton of implementation details which I'd appreciate a second eye on as well.
~~**NOTE** The first commit in this PR is #72342 and should be ignored for the purposes of review. The first commit is a very simplistic implementation which is basically raw-only, presented as a MVP. The second implements the full thing, and subsequent commits are incremental fixes.~~
cc `@ehuss` `@est31` `@petrochenkov` `@estebank`
...so we can skip serializing `tool_metadata` if it hasn't been set.
This makes the output a bit cleaner, and avoiding having to update a
bunch of unrelated tests.
This allows a build system to indicate a location in its own dependency
specification files (eg Cargo's `Cargo.toml`) which can be reported
along side any unused crate dependency.
This supports several types of location:
- 'json' - provide some json-structured data, which is included in the json diagnostics
in a `tool_metadata` field
- 'raw' - emit the provided string into the output. This also appears as a json string in
`tool_metadata`.
If no `--extern-location` is explicitly provided then a default json entry of the form
`"tool_metadata":{"name":<cratename>,"path":<cratepath>}` is emitted.
Improve SIMD type element count validation
Resolvesrust-lang/stdsimd#53.
These changes are motivated by `stdsimd` moving in the direction of const generic vectors, e.g.:
```rust
#[repr(simd)]
struct SimdF32<const N: usize>([f32; N]);
```
This makes a few changes:
* Establishes a maximum SIMD lane count of 2^16 (65536). This value is arbitrary, but attempts to validate lane count before hitting potential errors in the backend. It's not clear what LLVM's maximum lane count is, but cranelift's appears to be much less than `usize::MAX`, at least.
* Expands some SIMD intrinsics to support arbitrary lane counts. This resolves the ICE in the linked issue.
* Attempts to catch invalid-sized vectors during typeck when possible.
Unresolved questions:
* Generic-length vectors can't be validated in typeck and are only validated after monomorphization while computing layout. This "works", but the errors simply bail out with no context beyond the name of the type. Should these errors instead return `LayoutError` or otherwise provide context in some way? As it stands, users of `stdsimd` could trivially produce monomorphization errors by making zero-length vectors.
cc `@bjorn3`
