LLVM upgrade
As discussed in https://internals.rust-lang.org/t/need-help-with-emscripten-port/3154/46 I'm trying to update the used LLVM checkout in Rust.
I basically took @shepmaster's code and applied it on top (though I did the commits manually, the [original commits have better descriptions](https://github.com/rust-lang/rust/compare/master...avr-rust:avr-support).
With these changes I was able to build rustc. `make check` throws one last error on `run-pass/issue-28950.rs`. Output: https://gist.github.com/badboy/bcdd3bbde260860b6159aa49070a9052
I took the metadata changes as is and they seem to work, though it now uses the module in another step. I'm not sure if this is the best and correct way.
Things to do:
* [x] ~~Make `run-pass/issue-28950.rs` pass~~ unrelated
* [x] Find out how the `PositionIndependentExecutable` setting is now used
* [x] Is the `llvm::legacy` still the right way to do these things?
cc @brson @alexcrichton
The targets are:
- `arm-unknown-linux-musleabi`
- `arm-unknown-linux-musleabihf`
- `armv7-unknown-linux-musleabihf`
These mirror the existing `gnueabi` targets.
All of these targets produce fully static binaries, similar to the
x86 MUSL targets.
For now these targets can only be used with `--rustbuild` builds, as
https://github.com/rust-lang/compiler-rt/pull/22 only made the
necessary compiler-rt changes in the CMake configs, not the plain
GNU Make configs.
I've tested these targets GCC 5.3.0 compiled again musl-1.1.12
(downloaded from http://musl.codu.org/). An example `./configure`
invocation is:
```
./configure \
--enable-rustbuild
--target=arm-unknown-linux-musleabi \
--musl-root="$MUSL_ROOT"
```
where `MUSL_ROOT` points to the `arm-linux-musleabi` prefix.
Usually that path will be of the form
`/foobar/arm-linux-musleabi/arm-linux-musleabi`.
Usually the cross-compile toolchain will live under
`/foobar/arm-linux-musleabi/bin`. That path should either by added
to your `PATH` variable, or you should add a section to your
`config.toml` as follows:
```
[target.arm-unknown-linux-musleabi]
cc = "/foobar/arm-linux-musleabi/bin/arm-linux-musleabi-gcc"
cxx = "/foobar/arm-linux-musleabi/bin/arm-linux-musleabi-g++"
```
As a prerequisite you'll also have to put a cross-compiled static
`libunwind.a` library in `$MUSL_ROOT/lib`. This is similar to [how
the x86_64 MUSL targets are built]
(https://doc.rust-lang.org/book/advanced-linking.html).
llvm, rt: build using the Ninja generator if available
The Ninja generator generally builds much faster than make. It may also
be used on Windows to have a vast speed improvement over the Visual
Studio generators.
Currently hidden behind an `--enable-ninja` flag because it does not
obey the top-level `-j` or `-l` flags given to `make`.
Use --release-channel=stable by default on releases
> Release tarballs should be compilable with just basic ./configure ;
> make ; sudo make install without having to pass special flags to
> configure. This is the case of the --release-channel option, that must
> be changed in the releases.
This commit detects the presence of .git, as it happens on other parts
of `configure` to assume it is a tarball. Then it changes the default
value stored, before parsing the arguments, while still allowing it to
be overriden before any action verifying the flag is done.
Closes#28322
The Ninja generator generally builds much faster than make. It may also
be used on Windows to have a vast speed improvement over the Visual
Studio generators.
Currently hidden behind an `--enable-ninja` flag because it does not
obey the top-level `-j` or `-l` flags given to `make`.
configure: Check for valid Python on MinGW as well
The LLVM build system is somewhat picky about which Python is used to build it
as it's known to be incompatible with the default `python2` package that ships
with MinGW. This was previously detected for MSVC builds but the logic was left
out for MinGW by accident (now that we've switched to cmake builds for LLVM
everywhere).
This corrects the `./configure` check and also updates the `README.md`
accordingly. Additionally, a number of instructions were updated to work with
the most recent copy of MSYS2.
Closes https://github.com/rust-lang/rust/issues/28260Closes#34489
Actually got it working this time, and it was again just a problem specifying
the llvm-tblgen binary. We need to point it at the $CFG_BUILD target's tblgen
and then we also needed to correct the path a bit.
The LLVM build system is somewhat picky about which Python is used to build it
as it's known to be incompatible with the default `python2` package that ships
with MinGW. This was previously detected for MSVC builds but the logic was left
out for MinGW by accident (now that we've switched to cmake builds for LLVM
everywhere).
This corrects the `./configure` check and also updates the `README.md`
accordingly. Additionally, a number of instructions were updated to work with
the most recent copy of MSYS2.
Closes#34489
Convert makefiles to build LLVM/compiler-rt with CMake
This is certainly buggy, but I have successfully built on x86_64-unknown-linux-gnu and x86_64-pc-windows-gnu. I haven't built successfully on mac yet, and I've seen mysterious test failures on Linux, but I'm interested in throwing this at the bots to see what they think.
We no C++ and an incredibly small amount of C code as part of the build, so
there's not really much need for us to strictly check the version of compilers
as we're not really stressing anything. LLVM is a pretty huge chunk of C++ but
it should be the responsibility of LLVM to ensure that it can build with a
particular clang/gcc version, not ours (as this logic changes over time).
These version checks seem to basically just by us a regular stream of PRs every
six weeks or so when a new version is releases, so they're not really buying us
much. As a result, remove them and we can add then back piecemeal perhaps as a
blacklist if we really need to.
> Release tarballs should be compilable with just basic ./configure ;
> make ; sudo make install without having to pass special flags to
> configure. This is the case of the --release-channel option, that must
> be changed in the releases.
This commit detects the presence of .git, as it happens on other parts
of `configure` to assume it is a tarball. Then it changes the default
value stored, before parsing the arguments, while still allowing it to
be overriden before any action verifying the flag is done.
Closes#28322
In Linux distributions, it is often necessary to rebuild packages for
cases like applying new patches or linking against new system libraries.
In this scenario, the rustc in the distro build environment may already
match the current release that we're trying to rebuild. Thus we don't
want to use the prior release's bootstrap key, nor `--cfg stage0` for
the prior unstable features.
The new `configure --enable-local-rebuild` option specifies that we are
rebuilding from the current release. The current bootstrap key is used
for the local rustc, and current stage1 features are also assumed.
Sanity check Python on OSX for LLDB tests
Two primary changes:
* Don't get past the configure stage if `python` isn't coming from `/usr/bin`
* Call `debugger.Terminate()` to prevent segfaults on newer versions of LLDB.
Closes#32994
mk: Hardcode the bootstrap key for each release
Starting with the 1.10.0 release we would like to bootstrap all compilers from
the previous stable release. For example the 1.10.0 compiler should bootstrap
from the literal 1.9.0 release artifacts. To do this, however, we need a way to
enable unstable features temporarily in a stable compiler (as the released
compiler is stable), but it turns out we already have a way to do that!
At compile time the configure script selects a `CFG_BOOTSTRAP_KEY` variable
value and then exports it into the makefiles. If the `RUSTC_BOOTSTRAP_KEY`
environment variable is set to this value, then the compiler is allowed to
"cheat" and use unstable features.
This method of choosing the bootstrap key, however, is problematic for the
intention of bootstrapping from the previous release. Each time a 1.9.0 compiler
is created, a new bootstrap key will be selected. That means that the 1.10.0
compiler will only compile from *our* literal release artifacts. Instead
distributions would like to bootstrap from their own compilers, so instead we
simply hardcode the bootstrap key for each release.
This patch uses the same `CFG_FILENAME_EXTRA` value (a hash of the release
string) as the bootstrap key. Consequently all 1.9.0 compilers, no matter where
they are compiled, will have the same bootstrap key. Additionally we won't need
to keep updating this as it'll be based on the release number anyway.
Once the 1.9.0 beta has been created, we can update the 1.10.0 nightly sources
(the `master` branch at that time) to bootstrap from that release using this
hard-coded bootstrap key. We will likely just hardcode into the makefiles what
the previous bootstrap key was and we'll change that whenever the stage0
compiler is updated.
Starting with the 1.10.0 release we would like to bootstrap all compilers from
the previous stable release. For example the 1.10.0 compiler should bootstrap
from the literal 1.9.0 release artifacts. To do this, however, we need a way to
enable unstable features temporarily in a stable compiler (as the released
compiler is stable), but it turns out we already have a way to do that!
At compile time the configure script selects a `CFG_BOOTSTRAP_KEY` variable
value and then exports it into the makefiles. If the `RUSTC_BOOTSTRAP_KEY`
environment variable is set to this value, then the compiler is allowed to
"cheat" and use unstable features.
This method of choosing the bootstrap key, however, is problematic for the
intention of bootstrapping from the previous release. Each time a 1.9.0 compiler
is created, a new bootstrap key will be selected. That means that the 1.10.0
compiler will only compile from *our* literal release artifacts. Instead
distributions would like to bootstrap from their own compilers, so instead we
simply hardcode the bootstrap key for each release.
This patch uses the same `CFG_FILENAME_EXTRA` value (a hash of the release
string) as the bootstrap key. Consequently all 1.9.0 compilers, no matter where
they are compiled, will have the same bootstrap key. Additionally we won't need
to keep updating this as it'll be based on the release number anyway.
Once the 1.9.0 beta has been created, we can update the 1.10.0 nightly sources
(the `master` branch at that time) to bootstrap from that release using this
hard-coded bootstrap key. We will likely just hardcode into the makefiles what
the previous bootstrap key was and we'll change that whenever the stage0
compiler is updated.
Our `codegen` test suite requires the LLVM `FileCheck` utility but unfortunately
this isn't always available in all custom LLVM roots (e.g. those specified via
`--llvm-root`). This commit adds a `./configure` option called
`--disable-codegen-tests` which will manually disable running these tests. In
the case that this option is passed we can forgo the need for the `FileCheck`
executable. Note that we still require `FileCheck` by default as we will attempt
to run these tests.
Closes#28667
Detect the triple in the configure script for probing MSVC shenanigans and also
be sure to use `llvm-config` from the build host and not the target when
configuring compiler-rt.
Rust 1.7.0 and newer appears to require LLVM 3.6.0 or newer when
building against a version that's out of the tree with the --llvm-root
flag.
Signed-off-by: Doug Goldstein <cardoe@cardoe.com>
The `--disable-jemalloc` configure option has a failure mode where it will
create a distribution that is not compatible with other compilers. For example
the nightly for Linux will assume that it will link to jemalloc by default as
an allocator for executable crates. If, however, a standard library is used
which was built via `./configure --disable-jemalloc` then this will fail
because the jemalloc crate wasn't built.
While this seems somewhat reasonable as a niche situation, the same mechanism is
used for disabling jemalloc for platforms that just don't support it. For
example if the rumprun target is compiled then the sibiling Linux target *also*
doesn't have jemalloc. This is currently a problem for our cross-build nightlies
which build many targets. If rumprun is also built, it will disable jemalloc for
all targets, which isn't desired.
This commit moves the platform-specific disabling of jemalloc as hardcoded logic
into the makefiles that is scoped per-platform. This way when configuring
multiple targets **without the `--disable-jemalloc` option specified** all
targets will get jemalloc as they should.
In other words, enforce what was documented in #30626 (and also stop blaming it on LLVM, we have at least one Python script of our own).
Also, there is no Python later than 2.7 and there never will be.
This commit adds a `--enable-rustbuild` option to the configure script which
will copy a different `Makefile.in` into place to intercept all `make`
invocations.
Currently this makefile only has one target, but it's expected to be filled out
quite a bit over time!
Backtraces, and the compilation of libbacktrace for asmjs, are disabled.
This port doesn't use jemalloc so, like pnacl, it disables jemalloc *for all targets*
in the configure file.
It disables stack protection.
This target covers MIPS devices that run the trunk version of OpenWRT.
The x86_64-unknown-linux-musl target always links statically to C libraries. For
the mips(el)-unknown-linux-musl target, we opt for dynamic linking (like most of
other targets do) to keep binary size down.
As for the C compiler flags used in the build system, we use the same flags used
for the mips(el)-unknown-linux-gnu target.
r? @alexcrichton
This target covers MIPS devices that run the trunk version of OpenWRT.
The x86_64-unknown-linux-musl target always links statically to C libraries. For
the mips(el)-unknown-linux-musl target, we opt for dynamic linking (like most of
other targets do) to keep binary size down.
As for the C compiler flags used in the build system, we use the same flags used
for the mips(el)-unknown-linux-gnu target.
The purpose of the translation item collector is to find all monomorphic instances of functions, methods and statics that need to be translated into LLVM IR in order to compile the current crate.
So far these instances have been discovered lazily during the trans path. For incremental compilation we want to know the set of these instances in advance, and that is what the trans::collect module provides.
In the future, incremental and regular translation will be driven by the collector implemented here.
We've been seeing a lot of timeouts in tests on the bots and investigation ended
pointing to jemalloc/jemalloc#315 as the culprit. Unfortunately it looks like
that doesn't seem to have a fix on the way soon, so let's temporarily downgrade
back to the previous version of jemalloc we were using (where #30434 was the
most recent upgrade)
We've been seeing a lot of timeouts in tests on the bots and investigation ended
pointing to jemalloc/jemalloc#315 as the culprit. Unfortunately it looks like
that doesn't seem to have a fix on the way soon, so let's temporarily downgrade
back to the previous version of jemalloc we were using (where #30434 was the
most recent upgrade)
This mixes in additional information into the hash that is
passed to -C extra-filename. It can be used to further distinguish
the standard libraries if they must be installed next to each
other.
Closes#29559
On some weird setup where $SHELL is a relative path (can happen under GNU
Screen,) `file -L "$BIN_TO_PROBE"` fails and $CFG_CPUTYPE is wrongly set to
i686. We should not only check its string value but also permission on
filesystem.
This commit changes our distribution and in-tree sources to pass the `-C rpath`
flag by default during compiles. This means that from-source builds, including
our release channels, will have this option enabled as well. Motivated
by #29941, this change means that the compiler should be usable as-is on all
platforms just after extraction or installation. This experience is already true
on Windows but on Unixes you still need to set up LD_LIBRARY_PATH or the
equivalent, which can often be unfortunate.
This option was originally turned off by default for Linux distributions who
tend to take care of these sorts of details themselves, so it is expected that
all those builds of Rust will want to pass `--disable-rpath` to the configure
script to preserve that behavior.
Closes#29941
On some weird setup where $SHELL is a relative path (can happen under GNU
Screen,) `file -L "$BIN_TO_PROBE"` fails and $CFG_CPUTYPE is wrongly set to
i686. We should not only check its string value but also permission on
filesystem.
This handles cases when the LLVM used isn't configured will the 'usual'
targets. Also, cases where LLVM is shared are also handled (ie with
`LD_LIBRARY_PATH` etc).
This commit changes our distribution and in-tree sources to pass the `-C rpath`
flag by default during compiles. This means that from-source builds, including
our release channels, will have this option enabled as well. Motivated
by #29941, this change means that the compiler should be usable as-is on all
platforms just after extraction or installation. This experience is already true
on Windows but on Unixes you still need to set up LD_LIBRARY_PATH or the
equivalent, which can often be unfortunate.
This option was originally turned off by default for Linux distributions who
tend to take care of these sorts of details themselves, so it is expected that
all those builds of Rust will want to pass `--disable-rpath` to the configure
script to preserve that behavior.
Closes#29941
The Rust build scripts do work if the source directory contains spaces. I tried to make it work with spaces. I managed to get the Rust's and LLVM's configure scripts to work with spaces in the path, but I could not figure out how to get the Rust makefiles working.
So for now, this PR updates Rust's `configure` to abort if the source path contains spaces. I also added a note about spaces in the source path to the README.
I think this should close#18477 for now.
Debian wants to build all binaries with particular hardening flags.
The Rust makefiles are inconsistent in which architectures they
correctly include CFLAGS/etc from the enivoronment (see mk/cfg/*).
This patch adds LDFLAGS, and then unconditionally prepends
CFLAGS/LDFLAGS/etc to the build commands.
Part of #28710
Landing pads during stage0 are now enabled by defaullt. Since this has its downsides and upsides either way, I made it possible to change the option through configure.
For most parts, rumprun currently looks like NetBSD, as they share the same
libc and drivers. However, being a unikernel, rumprun does not support
process management, signals or virtual memory, so related functions
might fail at runtime. Stack guards are disabled exactly for this reason.
Code for rumprun is always cross-compiled, it uses always static
linking and needs a custom linker.
it makes rustc compatible with gcc installation that are using
`--program-transform-name' configure flag (on OpenBSD for example).
- detects at configure the name of stdc++ library on the system
- use the detected name in llvm makefile (with enable-static-stdcpp),
and pass it to mklldeps.py
- generate mklldeps.rs using this detected name
note that CFG_STDCPP_NAME is about stdc++ name, not about libc++. If
using libc++, the default name will be `stdc++', but it won't be used
when linking.
Redirect stdout on the python bogosity detector. This is printing
pwd to the terminal currently.
Reformat the bogus python/cmake messages so they format correctly.
echo does not always escape newlines (it doesn't here), and multiline
strings don't whitespace munch.
r? @alexcrichton
Redirect stdout on the python bogosity detector. This is printing
pwd to the terminal currently.
Reformat the bogus python/cmake messages so they format correctly.
echo does not always escape newlines (it doesn't here), and multiline
strings don't whitespace munch.
under OpenBSD, it could be have present multiples versions of gcc compiler:
- gcc-4.2 (with patchs) : c/c++ compiler present in `/usr/bin`. It is unusable to build recent LLVM (so rustc too).
- gcc/g++ -4.9 : c/c++ compiler, installed as third-party with "ports" mechanism. The compiler is installed in `/usr/local` as `egcc` (and `eg++`).
this PR adds probing for `egcc` for `CFG_GCC`, if the first probed `gcc` is too old. It will set `CC` and `CXX` too, in order to pass the variables to LLVM configure if builded.
please note this PR is a first step in order to build rustc under OpenBSD with unpatched tree.
r? @alexcrichton
cc @brson @dhuseby
Visual Studio 2015, recently released, includes the Universal CRT, a different
flavor than was provided before. The binaries and header files for this library
are included in new locations not previously known about by gcc-rs, and this
commit adds support for the necessary probing to find these.
Unfortunately there are no prior examples of this probing to be found in
frameworks like CMake or clang, so this is done is a bit of a sketchy method
today. It assumes that the installation is in a relatively standard format and
then blindly looks for the location of the UCRT. I'd love to switch this over to
using registry keys for probing, but I was currently unable to find such keys.
This should enable the compiler to work outside VS 2015 dev tools prompts.
Adds support to the configure script for detecting Visual Studio 2015 being
installed and builds LLVM/uses cl with that compiler. The compiler will
automatically use this MSVC linker anyway because it's the highest version.
This has travis build LLVM and rustc up to stage1, but not run any tests. It seems wasteful to have the ultimate might of travis running on every PR just to check for whitespace errors. This is a pure subset of the bootstrap, so it shouldn't ever spuriously break.
`make tidy` still runs first, so we still get \"fast errors\" on bad style. However once make tidy passes, the build will simply keep running to try to make rustc. `tidy` takes ~3 mins to complete, so if your build runs longer you can be confident we've gone on to build LLVM/rustc. In principle this is configured to use ccache (it shows up in the build logs as uploaded/downloaded), but I found no actual performance changes in using it.
Maybe someone at @travis-ci knows what's up with that.
For reference, here is a successful build with ccache enabled: https://travis-ci.org/Gankro/rust/builds/70821237
and one without: https://travis-ci.org/Gankro/rust/builds/70812814
Builds seem to take about 41mins regardless.
r? @alexcrichton
Tidy is still run first for failing fast on the easy stuff.
To accomplish this we have travis actually persist ccache across builds. This
has LLVM built within 6 minutes, and all of stage1 built within 18.
Caching should work on fresh PRs (cache acquired from the master branch).
Because all we persist is ccache, there is minimal danger of persisting corrupt
build state.
I had to mangle `configure` a bit to make --enable-ccache work when custom
compilers are provide via CC and CXX.
This commit modifies the configure script and our makefiles to support building
32-bit MSVC targets. The MSVC toolchain is now parameterized over whether it can
produce a 32-bit or 64-bit binary. The configure script was updated to export
more variables at configure time, and the makefiles were rejiggered to
selectively reexport the relevant environment variables for the applicable
targets they're going to run for.
