c23f045a8b
Don't run `everybody_loops` for rustdoc; instead ignore resolution errors r? @eddyb cc @petrochenkov, @GuillaumeGomez, @Manishearth, @ecstatic-morse, @marmeladema ~~Blocked on https://github.com/rust-lang/rust/pull/73743~~ Merged. ~~Blocked on crater run.~~ Crater popped up some ICEs ([now fixed](https://github.com/rust-lang/rust/pull/73566#issuecomment-656934851)). See [crater run](https://crater-reports.s3.amazonaws.com/pr-73566/index.html), [ICEs](https://github.com/rust-lang/rust/pull/73566#issuecomment-653619212). ~~Blocked on #74070 so that we don't make typeck_tables_of public when it shouldn't be.~~ Merged. Closes #71820, closes #71104, closes #65863. ## What is the motivation for this change? As seen from a lengthy trail of PRs and issues (https://github.com/rust-lang/rust/pull/73532, https://github.com/rust-lang/rust/pull/73103, https://github.com/rust-lang/rust/issues/71820, https://github.com/rust-lang/rust/issues/71104), `everybody_loops` is causing bugs in rustdoc. The main issue is that it does not preserve the validity of the `DefId` tree, meaning that operations on DefIds may unexpectedly fail when called later. This is blocking intra-doc links (see https://github.com/rust-lang/rust/pull/73101). This PR starts by removing `everybody_loops`, fixing #71104 and #71820. However, that brings back the bugs seen originally in https://github.com/rust-lang/rust/pull/43348: Since libstd documents items for all platforms, the function bodies sometimes do not type check. Here are the errors from documenting `libstd` with `everybody_loops` disabled and no other changes: ```rust error[E0433]: failed to resolve: could not find `handle` in `sys` --> src/libstd/sys/windows/ext/process.rs:13:27 | 13 | let handle = sys::handle::Handle::new(handle as *mut _); | ^^^^^^ could not find `handle` in `sys` error[E0425]: cannot find function `symlink_inner` in module `sys::fs` --> src/libstd/sys/windows/ext/fs.rs:544:14 | 544 | sys::fs::symlink_inner(src.as_ref(), dst.as_ref(), false) | ^^^^^^^^^^^^^ not found in `sys::fs` error[E0425]: cannot find function `symlink_inner` in module `sys::fs` --> src/libstd/sys/windows/ext/fs.rs:564:14 | 564 | sys::fs::symlink_inner(src.as_ref(), dst.as_ref(), true) | ^^^^^^^^^^^^^ not found in `sys::fs` ``` ## Why does this need changes to `rustc_resolve`? Normally, this could be avoided by simply not calling the `typeck_item_bodies` pass. However, the errors above happen before type checking, in name resolution itself. Since name resolution is intermingled with macro expansion, and rustdoc needs expansion to happen before it knows all items to be documented, there needs to be someway to ignore _resolution_ errors in function bodies. An alternative solution suggested by @petrochenkov was to not run `everybody_loops` on anything containing a nested `DefId`. This would solve some of the immediate issues, but isn't bullet-proof: the following functions still could not be documented if the items in the body failed to resolve: - Functions containing a nested `DefId` (https://github.com/rust-lang/rust/issues/71104) - ~~Functions returning `impl Trait` (https://github.com/rust-lang/rust/pull/43878)~~ These ended up not resolving anyway with this PR. - ~~`const fn`, because `loop {}` in `const fn` is unstable (https://github.com/rust-lang/rust/issues/43636)~~ `const_loop` was just stabilized. This also isn't exactly what rustdoc wants, which is to avoid looking at function bodies in the first place. ## What changes were made? The hack implemented in this PR is to add an option to ignore all resolution errors in function bodies. This is enabled only for rustdoc. Since resolution errors are ignored, the MIR generated will be invalid, as can be seen in the following ICE: ```rust error: internal compiler error: broken MIR in DefId(0:11 ~ doc_cfg[8787]::uses_target_feature[0]) ("return type"): bad type [type error] --> /home/joshua/src/rust/src/test/rustdoc/doc-cfg.rs:51:1 | 51 | / pub unsafe fn uses_target_feature() { 52 | | content::should::be::irrelevant(); 53 | | } | |_^ ``` Fortunately, rustdoc does not need to access MIR in order to generate documentation. Therefore this also removes the call to `analyze()` in `rustdoc::run_core`. This has the side effect of not generating all lints by default. Most lints are safe to ignore (does rustdoc really need to run liveness analysis?) but `missing_docs` in particular is disabled when it should not be. Re-running `missing_docs` specifically does not help, because it causes the typechecking pass to be run, bringing back the errors from #24658: ``` error[E0599]: no method named `into_handle` found for struct `sys::unix::pipe::AnonPipe` in the current scope --> src/libstd/sys/windows/ext/process.rs:71:27 | 71 | self.into_inner().into_handle().into_raw() as *mut _ | ^^^^^^^^^^^ method not found in `sys::unix::pipe::AnonPipe` | ``` Because of #73743, we only run typeck on demand. So this only causes an issue for functions returning `impl Trait`, which were already special cased by `ReplaceFunctionWithBody`. However, it now considers `async fn f() -> T` to be considered `impl Future<Output = T>`, where before it was considered to have a concrete `T` type. ## How will this affect future changes to rustdoc? - Any new changes to rustdoc will not be able to perform type checking without bringing back resolution errors in function bodies. + As a corollary, any new lints cannot require or perform type checking. In some cases this may require refactoring other parts of the compiler to perform type-checking only on-demand, see for example #73743. + As a corollary, rustdoc can never again call `tcx.analysis()` unless this PR is reverted altogether. ## Current status - ~~I am not yet sure how to bring back `missing_docs` without running typeck. @eddyb suggested allowing lints to opt-out of type-checking, which would probably be another rabbit hole.~~ The opt-out was implemented in https://github.com/rust-lang/rust/pull/73743. However, of the rustc lints, now _only_ missing_docs is run and no other lints: https://github.com/rust-lang/rust/pull/73566#issuecomment-650213058. We need a team decision on whether that's an acceptable tradeoff. Note that all rustdoc lints are still run (`intra_doc_link_resolution_failure`, etc). **UPDATE**: This was deemed acceptable in https://github.com/rust-lang/rust/pull/73566#issuecomment-655750237 - ~~The implementation of optional errors in `rustc_resolve` is very brute force, it should probably be moved from `LateResolver` to `Resolver` to avoid duplicating the logic in many places.~~ I'm mostly happy with it now. - This no longer allows errors in `async fn f() -> T`. This caused breakage in 50 crates out of a full crater run, all of which (that I looked at) didn't compile when run with rustc directly. In other words, it used to be that they could not be compiled but could still be documented; now they can't be documented either. This needs a decision from the rustdoc team on whether this is acceptable breakage. **UPDATE**: This was deemed acceptable in https://github.com/rust-lang/rust/pull/73566#issuecomment-655750237 - ~~This makes `fn typeck_tables_of` in `rustc_typeck` public. This is not desired behavior, but needs the changes from https://github.com/rust-lang/rust/pull/74070 in order to be fixed.~~ Reverted. |
||
---|---|---|
.github | ||
src | ||
.gitattributes | ||
.gitignore | ||
.gitmodules | ||
.mailmap | ||
Cargo.lock | ||
Cargo.toml | ||
CODE_OF_CONDUCT.md | ||
config.toml.example | ||
configure | ||
CONTRIBUTING.md | ||
COPYRIGHT | ||
LICENSE-APACHE | ||
LICENSE-MIT | ||
README.md | ||
RELEASES.md | ||
rustfmt.toml | ||
triagebot.toml | ||
x.py |
This is the main source code repository for Rust. It contains the compiler, standard library, and documentation.
Note: this README is for users rather than contributors.
Quick Start
Read "Installation" from The Book.
Installing from Source
Note: If you wish to contribute to the compiler, you should read the Getting Started of the rustc-dev-guide instead of this section.
The Rust build system uses a Python script called x.py
to build the compiler,
which manages the bootstrapping process. More information about it can be found
by running ./x.py --help
or reading the rustc dev guide.
Building on a Unix-like system
-
Make sure you have installed the dependencies:
g++
5.1 or later orclang++
3.5 or laterpython
3 or 2.7- GNU
make
3.81 or later cmake
3.4.3 or latercurl
git
ssl
which comes inlibssl-dev
oropenssl-devel
pkg-config
if you are compiling on Linux and targeting Linux
-
Clone the source with
git
:$ git clone https://github.com/rust-lang/rust.git $ cd rust
-
Configure the build settings:
The Rust build system uses a file named
config.toml
in the root of the source tree to determine various configuration settings for the build. Copy the defaultconfig.toml.example
toconfig.toml
to get started.$ cp config.toml.example config.toml
If you plan to use
x.py install
to create an installation, it is recommended that you set theprefix
value in the[install]
section to a directory.Create install directory if you are not installing in default directory
-
Build and install:
$ ./x.py build && ./x.py install
When complete,
./x.py install
will place several programs into$PREFIX/bin
:rustc
, the Rust compiler, andrustdoc
, the API-documentation tool. This install does not include Cargo, Rust's package manager. To build and install Cargo, you may run./x.py install cargo
or set thebuild.extended
key inconfig.toml
totrue
to build and install all tools.
Building on Windows
There are two prominent ABIs in use on Windows: the native (MSVC) ABI used by Visual Studio, and the GNU ABI used by the GCC toolchain. Which version of Rust you need depends largely on what C/C++ libraries you want to interoperate with: for interop with software produced by Visual Studio use the MSVC build of Rust; for interop with GNU software built using the MinGW/MSYS2 toolchain use the GNU build.
MinGW
MSYS2 can be used to easily build Rust on Windows:
-
Grab the latest MSYS2 installer and go through the installer.
-
Run
mingw32_shell.bat
ormingw64_shell.bat
from wherever you installed MSYS2 (i.e.C:\msys64
), depending on whether you want 32-bit or 64-bit Rust. (As of the latest version of MSYS2 you have to runmsys2_shell.cmd -mingw32
ormsys2_shell.cmd -mingw64
from the command line instead) -
From this terminal, install the required tools:
# Update package mirrors (may be needed if you have a fresh install of MSYS2) $ pacman -Sy pacman-mirrors # Install build tools needed for Rust. If you're building a 32-bit compiler, # then replace "x86_64" below with "i686". If you've already got git, python, # or CMake installed and in PATH you can remove them from this list. Note # that it is important that you do **not** use the 'python2' and 'cmake' # packages from the 'msys2' subsystem. The build has historically been known # to fail with these packages. $ pacman -S git \ make \ diffutils \ tar \ mingw-w64-x86_64-python \ mingw-w64-x86_64-cmake \ mingw-w64-x86_64-gcc
-
Navigate to Rust's source code (or clone it), then build it:
$ ./x.py build && ./x.py install
MSVC
MSVC builds of Rust additionally require an installation of Visual Studio 2017
(or later) so rustc
can use its linker. The simplest way is to get the
Visual Studio, check the “C++ build tools” and “Windows 10 SDK” workload.
(If you're installing cmake yourself, be careful that “C++ CMake tools for Windows” doesn't get included under “Individual components”.)
With these dependencies installed, you can build the compiler in a cmd.exe
shell with:
> python x.py build
Currently, building Rust only works with some known versions of Visual Studio. If you have a more recent version installed and the build system doesn't understand, you may need to force rustbuild to use an older version. This can be done by manually calling the appropriate vcvars file before running the bootstrap.
> CALL "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvars64.bat"
> python x.py build
Building rustc with older host toolchains
It is still possible to build Rust with the older toolchain versions listed below, but only if the LLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN option is set to true in the config.toml file.
- Clang 3.1
- Apple Clang 3.1
- GCC 4.8
- Visual Studio 2015 (Update 3)
Toolchain versions older than what is listed above cannot be used to build rustc.
Specifying an ABI
Each specific ABI can also be used from either environment (for example, using the GNU ABI in PowerShell) by using an explicit build triple. The available Windows build triples are:
- GNU ABI (using GCC)
i686-pc-windows-gnu
x86_64-pc-windows-gnu
- The MSVC ABI
i686-pc-windows-msvc
x86_64-pc-windows-msvc
The build triple can be specified by either specifying --build=<triple>
when
invoking x.py
commands, or by copying the config.toml
file (as described
in Installing From Source), and modifying the
build
option under the [build]
section.
Configure and Make
While it's not the recommended build system, this project also provides a
configure script and makefile (the latter of which just invokes x.py
).
$ ./configure
$ make && sudo make install
When using the configure script, the generated config.mk
file may override the
config.toml
file. To go back to the config.toml
file, delete the generated
config.mk
file.
Building Documentation
If you’d like to build the documentation, it’s almost the same:
$ ./x.py doc
The generated documentation will appear under doc
in the build
directory for
the ABI used. I.e., if the ABI was x86_64-pc-windows-msvc
, the directory will be
build\x86_64-pc-windows-msvc\doc
.
Notes
Since the Rust compiler is written in Rust, it must be built by a precompiled "snapshot" version of itself (made in an earlier stage of development). As such, source builds require a connection to the Internet, to fetch snapshots, and an OS that can execute the available snapshot binaries.
Snapshot binaries are currently built and tested on several platforms:
Platform / Architecture | x86 | x86_64 |
---|---|---|
Windows (7, 8, 10, ...) | ✓ | ✓ |
Linux (2.6.18 or later) | ✓ | ✓ |
macOS (10.7 Lion or later) | ✓ | ✓ |
You may find that other platforms work, but these are our officially supported build environments that are most likely to work.
Getting Help
The Rust community congregates in a few places:
- Stack Overflow - Direct questions about using the language.
- users.rust-lang.org - General discussion and broader questions.
- /r/rust - News and general discussion.
Contributing
If you are interested in contributing to the Rust project, please take a look at the Getting Started guide in the [rustc-dev-guide].
License
Rust is primarily distributed under the terms of both the MIT license and the Apache License (Version 2.0), with portions covered by various BSD-like licenses.
See LICENSE-APACHE, LICENSE-MIT, and COPYRIGHT for details.
Trademark
The Rust programming language is an open source, community project governed by a core team. It is also sponsored by the Mozilla Foundation (“Mozilla”), which owns and protects the Rust and Cargo trademarks and logos (the “Rust Trademarks”).
If you want to use these names or brands, please read the media guide.
Third-party logos may be subject to third-party copyrights and trademarks. See Licenses for details.