78a5d5b54e
This series of commits adds the initial implementation of a new build system for the compiler and standard library based on Cargo. The high-level architecture now looks like: 1. The `./configure` script is run with `--enable-rustbuild` and other standard configuration options. 2. A `Makefile` is generate which proxies commands to the new build system. 3. The new build system has a Python script entry point which manages downloading both a Rust and Cargo nightly. This initial script also manages building the build system itself (which is written in Rust). 4. The build system, written in rust and called `bootstrap`, architects how to call `cargo` and manages building all native libraries and such. One might reasonably ask "why rewrite the build system?", which is a good question! The Rust project has used Makefiles for as long as I can remember at least, and while ugly and difficult to use are undeniably robust as they contain years worth of tweaking and tuning for working on as many platforms in as many situation as possible. The rationale behind this PR, however is: * The makefiles are impenetrable to all but a few people on this planet. This means that contributions to the build system are almost nonexistent, and furthermore if a build system change is needed it's incredibly difficult to figure out how to do so. This hindrance prevents us from doing some "perhaps fancier" things we may wish to do in make. * Our build system, while portable, is unfortunately not infinitely portable everywhere. For example the recently-introduced MSVC target is quite unlikely to have `make` installed by default (e.g. it requires building inside of an MSYS2 shell currently). Conversely, the portability of make comes at a cost of crazy and weird hacks to work around all sorts of versions of software everywhere, especially when it comes to the configure script and makefiles. By rewriting this logic in one of the most robust platforms there is, Rust, we get to assuage all of these worries for free! * There's a standard tool to build Rust crates, Cargo, but the standard library and compiler don't use it. This means that they cannot benefit easily from the crates.io ecosystem, nor can the ecosystem benefit from a standard way to build this repository itself. Moving to Cargo should help assuage both of these needs. This has the added benefit of making the compiler more approachable for newbies as working on the compiler will just happen to be working on a large Cargo project, all the same standard tools and tricks will apply. * There's a huge amount of portability information in the main distribution, for example around cross compiling, compiling on new OSes, etc. Pushing this logic into standard crates (like `gcc`) enables the community to immediately benefit from new build logic. Despite these benefits, it's going to be a long road to actually replace our current build system. This PR is just the beginning and doesn't implement the full suite of functionality as the current one, but there are many more to follow! The current implementation strategy hopes to look like: 1. Land a second build system in-tree that can be itereated on an and contributed to. This will not be used just yet in terms of gating new commits to the repo. 2. Over time, bring the second build system to feature parity with the old build system, start setting up CI for both build systems. 3. At some point in the future, switch the default to the new build system, but keep the old one around. 4. At some further point in the future, delete the entire old build system. --- Alright, so with all that out of the way, here's some more info on this PR itself. The inital build system here is contained in the `src/bootstrap` directory and just adds the necessary minimum bits to bootstrap the compiler itself. There is currently no support for building documentation, running tests, or installing, but the implemented support is: * Compiling LLVM with `cmake` instead of `./configure` + `make`. The LLVM project is removing their autotools build system, so we'd have to make this transition eventually anyway. * Compiling compiler-rt with `cmake` as well (for the same rationale as above). * Adding `Cargo.toml` to map out the dependency graph to all crates, and also adding `build.rs` files where appropriate. For example `alloc_jemalloc` has a script to build jemalloc, `flate` has a script to build `miniz.c`, `std` will build `libbacktrace`, etc. * Orchestrating all the calls to `cargo` to build the standard distribution, following the normal bootstrapping process. This also tracks dependencies between steps to ensure cross-compilation targets happen as well. * Configuration is intended to eventually be done through a `config.toml` file, so support is implemented for this. The most likely vector of configuration for now, however, is likely through `config.mk` (what `./configure` emits), so the build system currently parses this information. There's still quite a few steps left to do, and I'll open up some follow-up issues (as well as a tracking issue) for this migration, but hopefully this is a great start to get going! This PR is currently tested on all the Windows/Linux/OSX triples for x86\_64 and x86, but more portability is always welcome! --- Future functionality left to implement * [ ] Re-verify that multi-host builds work * [ ] Verify android build works * [ ] Verify iOS build work (mostly compiler-rt) * [ ] Verify sha256 and ideally gpg of downloaded nightly compiler and nightly rustc * [ ] Implement testing -- this is a huge bullet point with lots of sub-bullets * [ ] Build and generate documentation (plus the various tools we have in-tree) * [ ] Move various src/etc scripts into Rust -- not sure how this interacts with `make` build system * [ ] Implement `make install` - like testing this is also quite massive * [x] Deduplicate version information with makefiles
The Rust Programming Language
This is the main source code repository for Rust. It contains the compiler, standard library, and documentation.
Quick Start
Read "Installing Rust" from The Book.
Building from Source
-
Make sure you have installed the dependencies:
g++4.7 orclang++3.xpython2.7 or later (but not 3.x)- GNU
make3.81 or later curlgit
-
Clone the source with
git:$ git clone https://github.com/rust-lang/rust.git $ cd rust
-
Build and install:
$ ./configure $ make && make installNote: You may need to use
sudo make installif you do not normally have permission to modify the destination directory. The install locations can be adjusted by passing a--prefixargument toconfigure. Various other options are also supported – pass--helpfor more information on them.When complete,
make installwill place several programs into/usr/local/bin:rustc, the Rust compiler, andrustdoc, the API-documentation tool. This install does not include Cargo, Rust's package manager, which you may also want to build.
Building on Windows
There are two prominent ABIs in use on Windows: the native (MSVC) ABI used by Visual Studio, and the GNU ABI used by the GCC toolchain. Which version of Rust you need depends largely on what C/C++ libraries you want to interoperate with: for interop with software produced by Visual Studio use the MSVC build of Rust; for interop with GNU software built using the MinGW/MSYS2 toolchain use the GNU build.
MinGW
MSYS2 can be used to easily build Rust on Windows:
-
Grab the latest MSYS2 installer and go through the installer.
-
From the MSYS2 terminal, install the
mingw64toolchain and other required tools.# Update package mirrors (may be needed if you have a fresh install of MSYS2) $ pacman -Sy pacman-mirrors
Download MinGW from
here, and choose the
threads=win32,exceptions=dwarf/seh flavor when installing. After installing,
add its bin directory to your PATH. This is due to #28260, in the future,
installing from pacman should be just fine.
# Make git available in MSYS2 (if not already available on path)
$ pacman -S git
$ pacman -S base-devel
-
Run
mingw32_shell.batormingw64_shell.batfrom wherever you installed MSYS2 (i.e.C:\msys), depending on whether you want 32-bit or 64-bit Rust. -
Navigate to Rust's source code, configure and build it:
$ ./configure $ make && make install
MSVC
MSVC builds of Rust additionally require an installation of Visual Studio 2013
(or later) so rustc can use its linker. Make sure to check the “C++ tools”
option. In addition, cmake needs to be installed to build LLVM.
With these dependencies installed, the build takes two steps:
$ ./configure
$ make && make install
Building Documentation
If you’d like to build the documentation, it’s almost the same:
./configure
$ make docs
Building the documentation requires building the compiler, so the above details will apply. Once you have the compiler built, you can
$ make docs NO_REBUILD=1
To make sure you don’t re-build the compiler because you made a change to some documentation.
The generated documentation will appear in a top-level doc directory,
created by the make rule.
Notes
Since the Rust compiler is written in Rust, it must be built by a precompiled "snapshot" version of itself (made in an earlier state of development). As such, source builds require a connection to the Internet, to fetch snapshots, and an OS that can execute the available snapshot binaries.
Snapshot binaries are currently built and tested on several platforms:
| Platform \ Architecture | x86 | x86_64 |
|---|---|---|
| Windows (7, 8, Server 2008 R2) | ✓ | ✓ |
| Linux (2.6.18 or later) | ✓ | ✓ |
| OSX (10.7 Lion or later) | ✓ | ✓ |
You may find that other platforms work, but these are our officially supported build environments that are most likely to work.
Rust currently needs between 600MiB and 1.5GiB to build, depending on platform. If it hits swap, it will take a very long time to build.
There is more advice about hacking on Rust in CONTRIBUTING.md.
Getting Help
The Rust community congregates in a few places:
- Stack Overflow - Direct questions about using the language.
- users.rust-lang.org - General discussion and broader questions.
- /r/rust - News and general discussion.
Contributing
To contribute to Rust, please see CONTRIBUTING.
Rust has an IRC culture and most real-time collaboration happens in a variety of channels on Mozilla's IRC network, irc.mozilla.org. The most popular channel is #rust, a venue for general discussion about Rust, and a good place to ask for help.
License
Rust is primarily distributed under the terms of both the MIT license and the Apache License (Version 2.0), with portions covered by various BSD-like licenses.
See LICENSE-APACHE, LICENSE-MIT, and COPYRIGHT for details.