Implement synchronization scheme for incr. comp. directory
This PR implements a copy-on-write-based synchronization scheme for the incremental compilation cache directory. For technical details, see the documentation at the beginning of `rustc_incremental/persist/fs.rs`.
The PR contains unit tests for some functions but for testing whether the scheme properly handles races, a more elaborate test setup would be needed. It would probably involve a small tool that allows to manipulate the incremental compilation directory in a controlled way and then letting a compiler instance run against directories in different states. I don't know if it's worth the trouble of adding another test category to `compiletest`, but I'd be happy to do so.
Fixes#32754Fixes#34957
initial support for s390x
A new target, `s390x-unknown-linux-gnu`, has been added to the compiler
and can be used to build no_core/no_std Rust programs.
Known limitations:
- librustc_trans/cabi_s390x.rs is missing. This means no support for
`extern "C" fn`.
- No support for this arch in libc. This means std can't be cross
compiled for this target.
r? @alexcrichton
This time I couldn't test running a binary cross compiled to this target under QEMU because the qemu-s390x that ships with Ubuntu 16.04 SIGABRTs with every s390x binary I run it with.
Change in binary size of `librustc_llvm.so`:
Without this commit (stage1): 41895736 bytes
With this commit (stage1): 42899016 bytes
~2.4% increase
A new target, `s390x-unknown-linux-gnu`, has been added to the compiler
and can be used to build no_core/no_std Rust programs.
Known limitations:
- librustc_trans/cabi_s390x.rs is missing. This means no support for
`extern "C" fn`.
- No support for this arch in libc. This means std can be cross compiled
for this target.
add mips-uclibc targets
These targets cover OpenWRT 15.05 devices, which use the soft float ABI
and the uclibc library. None of the other built-in mips targets covered
those devices (mips-gnu is hard float and glibc-based, mips-musl is
musl-based).
With this commit one can now build std for these devices using these
commands:
```
$ configure --enable-rustbuild --target=mips-unknown-linux-uclibc
$ make
```
cc #35673
---
r? @alexcrichton
cc @felixalias This is the target the rust-tessel project should be using.
Note that the libc crate doesn't support the uclibc library and will have to be updated. We are lucky that uclibc and glibc are somewhat similar and one can build std and even run the libc-test test suite with the current, unmodified libc. About that last part, I tried to run the libc-test and got a bunch of compile errors. I don't intend to fix them but I'll post some instruction about how to run libc-test in the rust-lang/libc issue tracker.
Kicking off libproc_macro
This PR introduces `libproc_macro`, which is currently quite bare-bones (just a few macro construction tools and an initial `quote!` macro).
This PR also introduces a few test cases for it, and an additional `shim` file (at `src/libsyntax/ext/proc_macro_shim.rs` to allow a facsimile usage of Macros 2.0 *today*!
These targets cover OpenWRT 15.05 devices, which use the soft float ABI
and the uclibc library. None of the other built-in mips targets covered
those devices (mips-gnu is hard float and glibc-based, mips-musl is
musl-based).
With this commit one can now build std for these devices using these
commands:
```
$ configure --enable-rustbuild --target=mips-unknown-linux-uclibc
$ make
```
cc #35673
add GNU make files for arm-unknown-linux-musleabi
For Yocto (Embedded Linux meta distro) Rust is provided via the [meta-rust layer](https://github.com/meta-rust/meta-rust). In this project there have been patches to add `arm-unknown-linux-musleabi`. Rust recently acquired that support via #35060 but only for rustbuild. meta-rust is currently only able to build Rust support with the existing GNU Makefiles. This adds `arm-unknown-linux-musleabi` support to Rust for the GNU Makefiles until meta-rust is able to sort out why using rustbuild does not work for it.
/cc @srwalter @derekstraka @jmesmon @japaric
add -mrelax-relocations=no to i686-musl and i586-gnu
I've been experiencing #34978 with these two targets. This applies the
hack in #35178 to these targets as well.
r? @alexcrichton
The arm-unknown-linux-musleabi target used in meta-rust for Yocto didn't
explicitly set the arch to ARMv6 and soft float but was instead done via
target spec files and never had the compiler running on the target.
Add -mrelax-relocations=no hacks to fix musl build
* this is just a start, dunno if it will work, but I'll just push it out to get feedback (my rust is still building 😢)
* I don't know much about rustbuild, so i just added that flag in there. it's a total hack, don't judge me
* I suspect the places in the musl .mk files are sufficient (but we may also need it present when building std), I'm not sure, needs more testing.
LLVM upgrade
As discussed in https://internals.rust-lang.org/t/need-help-with-emscripten-port/3154/46 I'm trying to update the used LLVM checkout in Rust.
I basically took @shepmaster's code and applied it on top (though I did the commits manually, the [original commits have better descriptions](https://github.com/rust-lang/rust/compare/master...avr-rust:avr-support).
With these changes I was able to build rustc. `make check` throws one last error on `run-pass/issue-28950.rs`. Output: https://gist.github.com/badboy/bcdd3bbde260860b6159aa49070a9052
I took the metadata changes as is and they seem to work, though it now uses the module in another step. I'm not sure if this is the best and correct way.
Things to do:
* [x] ~~Make `run-pass/issue-28950.rs` pass~~ unrelated
* [x] Find out how the `PositionIndependentExecutable` setting is now used
* [x] Is the `llvm::legacy` still the right way to do these things?
cc @brson @alexcrichton
Add ARM MUSL targets
Rebase of #33189.
I tested this by producing a std for `arm-unknown-linux-musleabi` then I cross compiled Hello world to said target. Checked that the produced binary was statically linked and verified that the binary worked under QEMU.
This depends on rust-lang/libc#341. I'll have to update this PR after that libc PR is merged.
I'm also working on generating ARM musl cross toolchain via crosstool-ng. Once I verified those work, I'll send a PR to rust-buildbot.
r? @alexcrichton
cc @timonvo
The targets are:
- `arm-unknown-linux-musleabi`
- `arm-unknown-linux-musleabihf`
- `armv7-unknown-linux-musleabihf`
These mirror the existing `gnueabi` targets.
All of these targets produce fully static binaries, similar to the
x86 MUSL targets.
For now these targets can only be used with `--rustbuild` builds, as
https://github.com/rust-lang/compiler-rt/pull/22 only made the
necessary compiler-rt changes in the CMake configs, not the plain
GNU Make configs.
I've tested these targets GCC 5.3.0 compiled again musl-1.1.12
(downloaded from http://musl.codu.org/). An example `./configure`
invocation is:
```
./configure \
--enable-rustbuild
--target=arm-unknown-linux-musleabi \
--musl-root="$MUSL_ROOT"
```
where `MUSL_ROOT` points to the `arm-linux-musleabi` prefix.
Usually that path will be of the form
`/foobar/arm-linux-musleabi/arm-linux-musleabi`.
Usually the cross-compile toolchain will live under
`/foobar/arm-linux-musleabi/bin`. That path should either by added
to your `PATH` variable, or you should add a section to your
`config.toml` as follows:
```
[target.arm-unknown-linux-musleabi]
cc = "/foobar/arm-linux-musleabi/bin/arm-linux-musleabi-gcc"
cxx = "/foobar/arm-linux-musleabi/bin/arm-linux-musleabi-g++"
```
As a prerequisite you'll also have to put a cross-compiled static
`libunwind.a` library in `$MUSL_ROOT/lib`. This is similar to [how
the x86_64 MUSL targets are built]
(https://doc.rust-lang.org/book/advanced-linking.html).
Add MIR Optimization Tests
I've starting working on the infrastructure for testing MIR optimizations.
The plan now is to have a set of test cases (written in Rust), compile them with -Z dump-mir, and check the MIR before and after each pass.
The compiler-rt build system has been a never ending cause of pain for Rust
unfortunately:
* The build system is very difficult to invoke and configure to only build
compiler-rt, especially across platforms.
* The standard build system doesn't actually do what we want, not working for
some of our platforms and requiring a significant number of patches on our end
which are difficult to apply when updating compiler-rt.
* Compiling compiler-rt requires LLVM to be compiled, which... is a big
dependency! This also means that over time compiler-rt is not guaranteed to
build against older versions of LLVM (or newer versions), and we often want to
work with multiple versions of LLVM simultaneously.
The makefiles and rustbuild already know how to compile C code, the code here is
far from the *only* C code we're compiling. This patch jettisons all logic to
work with compiler-rt's build system and just goes straight to the source. We
just list all files manually (copied from compiler-rt's
lib/builtins/CMakeLists.txt) and compile them into an archive.
It's likely that this means we'll fail to pick up new files when we upgrade
compiler-rt, but that seems like a much less significant cost to pay than what
we're currently paying.
cc #34400, first steps towards that
llvm, rt: build using the Ninja generator if available
The Ninja generator generally builds much faster than make. It may also
be used on Windows to have a vast speed improvement over the Visual
Studio generators.
Currently hidden behind an `--enable-ninja` flag because it does not
obey the top-level `-j` or `-l` flags given to `make`.
If local-rust is the same as the current version, then force a local-rebuild
In Debian, we would like the option to build/rebuild the current release from
*either* the current or previous stable release. So we use enable-local-rust
instead of enable-local-rebuild, and read the bootstrap key dynamically from
whatever is installed locally.
In general, it does not make much sense to allow enable-local-rust without also
setting the bootstrap key, since the build would fail otherwise.
(The way I detect "the bootstrap key of [the local] rustc installation" is a bit hacky, suggestions welcome.)
Soon the LLVM upgrade (#34743) will require an updated CMake installation, and
the easiest way to do this was to upgrade the Ubuntu version of the bots to
16.04. This in turn brings in a new MIPS compiler on the linux-cross builder,
which is now from the "official" ubuntu repositories. Unfortunately these
new compilers don't support compiling with the `-msoft-float` flag like we're
currently passing, causing compiles to fail.
This commit removes these flags as it's not clear *why* they're being passed, as
the mipsel targets also don't have it. At least if it's not supported by a
debian default compiler, perhaps it's not too relevant to support?
The Ninja generator generally builds much faster than make. It may also
be used on Windows to have a vast speed improvement over the Visual
Studio generators.
Currently hidden behind an `--enable-ninja` flag because it does not
obey the top-level `-j` or `-l` flags given to `make`.
mk: Request -march=i686 on i686 Linux
Apparently the gcc on our dist bot is so old and/or obscure that the default
`-m32` switch doesn't think it can generate i686 code (or something like that).
The compiler-rt build system probes for the `__i686__` define in GCC to compile
for an i686 (vs i386) target, so this was failing on the bots.
This tweaks instead to pass `-march=i686` on i686-unknown-linux-gnu to C code to
ensure that we're compiling for i686 instead of i386. This should hopefully not
actually have an impact other than maybe doing some random optimization it
wasn't able to do so before. In theory this isn't making the target less
compatible as all Rust code is already compiled for i686.
Hopefully closes#34572
Apparently the gcc on our dist bot is so old and/or obscure that the default
`-m32` switch doesn't think it can generate i686 code (or something like that).
The compiler-rt build system probes for the `__i686__` define in GCC to compile
for an i686 (vs i386) target, so this was failing on the bots.
This tweaks instead to pass `-march=i686` on i686-unknown-linux-gnu to C code to
ensure that we're compiling for i686 instead of i386. This should hopefully not
actually have an impact other than maybe doing some random optimization it
wasn't able to do so before. In theory this isn't making the target less
compatible as all Rust code is already compiled for i686.
Hopefully closes#34572
Currently if an LLVM build is interrupted *after* it creates the llvm-config
binary but before it's done it puts us in an inconsistent state where we think
LLVM is compiled but it's not actually. This tweaks our logic to only consider
LLVM done building once it's actually done building.
This should hopefully alleviate problems on the bots where if we interrupt at
the wrong time it doesn't corrupt the build directory.
Try to fix the nightlies
They look to be failing right after the CMake PR landed. I've diagnosed and confirmed the first issue fixed, the second is a bit of a shot in the dark to see if it fixes things.
* Implement the clean-llvm target for those cases where makefiles are being used
* Have all cross-compiled LLVMs depend on the **host** LLVM as they'll require
the llvm-tablegen executable from there
