Commit Graph

382 Commits

Author SHA1 Message Date
Simon Sapin
b2027ef17c Deprecate the std_unicode crate 2018-04-12 00:13:51 +02:00
Alec Mocatta
18e9532098
Add --enable-debug flag to musl CI build script
Building for x86_64-unknown-linux-musl currently results in an executable lacking debug information for musl libc itself. If you request a backtrace in GDB while control flow is within musl – including sycalls made by musl – the result looks like:

#0  0x0000000000434b46 in __cp_end ()
#1  0x0000000000432dbd in __syscall_cp_c ()
#2  0x0000000000000000 in ?? ()

i.e. not very helpful. Adding --enable-debug resolves this, and --enable-optimize re-enables optimisations which default to off given the previous flag.
2018-04-10 18:44:47 +01:00
kennytm
649f431acf
Give a name to every CI job.
Bots that read the log can simply look for `[CI_JOB_NAME=...]` to find out
the job's name.
2018-04-06 01:09:59 +08:00
kennytm
f4511e2437
Rollup merge of #49674 - alexcrichton:no-incremental-rustc, r=michaelwoerister
ci: Remove x86_64-gnu-incremental builder

This builder is starting to time out frequently causing PRs to bounce and
otherwise doesn't seem to be catching too many bugs, so this commit removes it
entirely. We've had a number of timeouts in the last few weeks related to this
builder:

* https://travis-ci.org/rust-lang/rust/jobs/360947582
* https://travis-ci.org/rust-lang/rust/jobs/360464190
* https://travis-ci.org/rust-lang/rust/jobs/359946975
* https://travis-ci.org/rust-lang/rust/jobs/361213241
* https://travis-ci.org/rust-lang/rust/jobs/362346279
* https://travis-ci.org/rust-lang/rust/jobs/362072331

On a good run this builder takes about 2h15m, which is already too long for
Travis and the variable build times end up pushing it beyond the 3h limit
occasionally.

The timeouts here are somewhat expected in that an incrementally compiled rustc
compiler isn't optimized like a normal rustc, disallowing inlining between
codegen units and losing lots of optimization opportunities.
2018-04-05 18:38:53 +08:00
kennytm
b146e33518
Rollup merge of #49563 - japaric:std-thumb, r=alexcrichton
add a dist builder to build rust-std components for the THUMB targets

the rust-std component only contains the core and compiler-builtins (+c +mem) crates

cc #49382

- I'm not entirely sure if this PR alone will produce rust-std components installable by rustup or if something else needs to be changed
- I could have done the THUMB builds in an existing builder / image; I wasn't sure if that was a good idea so I added a new image
- I could build other crates like alloc into the rust-std component but, AFAICT, that would require calling Cargo a second time (one for alloc and one for compiler-builtins), or have alloc depend on compiler-builtins (#49503 will perform that change) *and* have alloc resurface the "c" and "mem" Cargo features.

r? @alexcrichton
2018-04-05 18:37:25 +08:00
Alex Crichton
621ccf8917 ci: Remove x86_64-gnu-incremental builder
This builder is starting to time out frequently causing PRs to bounce and
otherwise doesn't seem to be catching too many bugs, so this commit removes it
entirely. We've had a number of timeouts in the last few weeks related to this
builder:

* https://travis-ci.org/rust-lang/rust/jobs/360947582
* https://travis-ci.org/rust-lang/rust/jobs/360464190
* https://travis-ci.org/rust-lang/rust/jobs/359946975
* https://travis-ci.org/rust-lang/rust/jobs/361213241
* https://travis-ci.org/rust-lang/rust/jobs/362346279
* https://travis-ci.org/rust-lang/rust/jobs/362072331

On a good run this builder takes about 2h15m, which is already too long for
Travis and the variable build times end up pushing it beyond the 3h limit
occasionally.

The timeouts here are somewhat expected in that an incrementally compiled rustc
compiler isn't optimized like a normal rustc, disallowing inlining between
codegen units and losing lots of optimization opportunities.
2018-04-04 17:35:42 -07:00
Jorge Aparicio
d0693548cd merge dist-various-3 into dist-various-1 2018-04-03 08:26:24 +02:00
Alex Crichton
64f7e11fc3 Update sccache to its master branch
Ideally I'd like to soon enable sccache for rustbuild itself and some of the
stage0 tools, but for that to work we'll need some better Rust support than the
pretty old version we were previously using!
2018-04-02 12:24:50 -07:00
Jorge Aparicio
68b54a5f43 add a dist-thumb builder to build rust-std for the THUMB targets
the rust-std component only contains the core and compiler-builtins (+c +mem) crates

cc #49382
2018-04-01 18:50:21 +02:00
Alex Crichton
bd28641344 rustbuild: Disable docs on cross-compiled builds
This commit disables building documentation on cross-compiled compilers, for
example ARM/MIPS/PowerPC/etc. Currently I believe we're not getting much use out
of these documentation artifacts and they often take 10-15 minutes total to
build as it requires building rustdoc/rustbook and then also generating all the
documentation, especially for the reference and the book itself.

In an effort to cut down on the amount of work that we're doing on dist CI
builders in light of recent timeouts this was some relatively low hanging fruit
to cut which in theory won't have much impact on the ecosystem in the hopes that
the documentation isn't used too heavily anyway.

While initial analysis in #48827 showed only shaving 5 minutes off local builds
the same 5 minute conclusion was drawn from #48826 which ended up having nearly
a half-hour impact on the bots. In that sense I'm hoping that we can land this
and test out what happens on CI to see how it affects timing.

Note that all tier 1 platforms, Windows, Mac, and Linux, will continue to
generate documentation.
2018-03-25 23:31:41 -07:00
bors
e5bf0428d1 Auto merge of #49284 - alexcrichton:use-our-own-cache, r=kennytm
ci: Don't use Travis caches for docker images

This commit moves away from caching on Travis to our own caching on S3 for
caching docker layers between builds. Unfortunately the Travis caches have over
time had a few critical pain points:

* Caches are only updated for successful builds, meaning that if a build times
  out or fails in a different location the sucessfully-created docker images
  isn't always cached. While this makes sense as a general rule of caches it
  hurts our use cases.

* Caches are per-branch and builder which means that we don't have a separate
  cache on each release channel. All our merges go through the `auto` branch
  which means that they're all sharing the same cache, even those for merging to
  master/beta. This means that PRs which switch between master/beta will keep
  rebuilting and having cache misses.

* Caches have historically been invaliated somewhat regularly a little more
  aggressively than we'd want (I think).

* We don't always need to update the contents of the cache if the Docker image
  didn't change at all, and saving off the docker layers can sometimes be quite
  expensive.

For all these reasons this commit drops the usage of Travis's built-in caching
support. Instead our own caching is used by storing blobs to S3. Normally this
would be a very risky endeavour but we're basically priming a cache for a cache
(docker) so if we get this wrong the failure mode is longer builds, not stale
caches. We'll notice that pretty quickly and hopefully fix it!

The logic here is inserted directly into the `src/ci/docker/run.sh` script to
download an image based on a shasum of the `Dockerfile` and other assorted files.
This blob, if found, is loaded into docker and we record what layers were
inserted. After docker finishes the build (hopefully quickly with lots of cache
hits) we then see the sha of the final image. If it's one of the layers we
loaded then there's no need to update the cache. Otherwise we upload our layers
to the global cache, possibly overwriting what we previously just downloaded.

This is hopefully a step towards mitigating #49278 although it doesn't
completely fix it as it means we'll still probably have to retry builds that
bust the cache.
2018-03-25 02:42:58 +00:00
kennytm
311814a1a9
Rollup merge of #49193 - davidtwco:issue-29893, r=alexcrichton
Host compiler documentation

Fixes #29893. Rust Central Station PR: rust-lang/rust-central-station#40

r? @alexcrichton
2018-03-25 01:26:33 +08:00
kennytm
c5264a5932
Rollup merge of #49120 - Zoxc:parallel-ci, r=alexcrichton
Add a CI job for parallel rustc using x.py check

r? @alexcrichton
2018-03-25 01:26:28 +08:00
Alex Crichton
a09e9e9a2a ci: Don't use Travis caches for docker images
This commit moves away from caching on Travis to our own caching on S3 for
caching docker layers between builds. Unfortunately the Travis caches have over
time had a few critical pain points:

* Caches are only updated for successful builds, meaning that if a build times
  out or fails in a different location the sucessfully-created docker images
  isn't always cached. While this makes sense as a general rule of caches it
  hurts our use cases.

* Caches are per-branch and builder which means that we don't have a separate
  cache on each release channel. All our merges go through the `auto` branch
  which means that they're all sharing the same cache, even those for merging to
  master/beta. This means that PRs which switch between master/beta will keep
  rebuilting and having cache misses.

* Caches have historically been invaliated somewhat regularly a little more
  aggressively than we'd want (I think).

* We don't always need to update the contents of the cache if the Docker image
  didn't change at all, and saving off the docker layers can sometimes be quite
  expensive.

For all these reasons this commit drops the usage of Travis's built-in caching
support. Instead our own caching is used by storing blobs to S3. Normally this
would be a very risky endeavour but we're basically priming a cache for a cache
(docker) so if we get this wrong the failure mode is longer builds, not stale
caches. We'll notice that pretty quickly and hopefully fix it!

The logic here is inserted directly into the `src/ci/docker/run.sh` script to
download an image based on a shasum of the `Dockerfile` and other assorted files.
This blob, if found, is loaded into docker and we record what layers were
inserted. After docker finishes the build (hopefully quickly with lots of cache
hits) we then see the sha of the final image. If it's one of the layers we
loaded then there's no need to update the cache. Otherwise we upload our layers
to the global cache, possibly overwriting what we previously just downloaded.

This is hopefully a step towards mitigating #49278 although it doesn't
completely fix it as it means we'll still probably have to retry builds that
bust the cache.
2018-03-22 18:31:45 -07:00
Alex Crichton
d889957dab rustc: Add a #[wasm_import_module] attribute
This commit adds a new attribute to the Rust compiler specific to the wasm
target (and no other targets). The `#[wasm_import_module]` attribute is used to
specify the module that a name is imported from, and is used like so:

    #[wasm_import_module = "./foo.js"]
    extern {
        fn some_js_function();
    }

Here the import of the symbol `some_js_function` is tagged with the `./foo.js`
module in the wasm output file. Wasm-the-format includes two fields on all
imports, a module and a field. The field is the symbol name (`some_js_function`
above) and the module has historically unconditionally been `"env"`. I'm not
sure if this `"env"` convention has asm.js or LLVM roots, but regardless we'd
like the ability to configure it!

The proposed ES module integration with wasm (aka a wasm module is "just another
ES module") requires that the import module of wasm imports is interpreted as an
ES module import, meaning that you'll need to encode paths, NPM packages, etc.
As a result, we'll need this to be something other than `"env"`!

Unfortunately neither our version of LLVM nor LLD supports custom import modules
(aka anything not `"env"`). My hope is that by the time LLVM 7 is released both
will have support, but in the meantime this commit adds some primitive
encoding/decoding of wasm files to the compiler. This way rustc postprocesses
the wasm module that LLVM emits to ensure it's got all the imports we'd like to
have in it.

Eventually I'd ideally like to unconditionally require this attribute to be
placed on all `extern { ... }` blocks. For now though it seemed prudent to add
it as an unstable attribute, so for now it's not required (as that'd force usage
of a feature gate). Hopefully it doesn't take too long to "stabilize" this!

cc rust-lang-nursery/rust-wasm#29
2018-03-22 13:16:38 -07:00
bors
5092c6b01a Auto merge of #49264 - kennytm:rollup, r=kennytm
Rollup of 23 pull requests

- Successful merges: #48374, #48596, #48759, #48939, #49029, #49069, #49093, #49109, #49117, #49140, #49158, #49188, #49189, #49209, #49211, #49216, #49225, #49231, #49234, #49242, #49244, #49105, #49038
- Failed merges:
2018-03-22 16:58:57 +00:00
kennytm
a4bc859011
Revert "Temporarily disable dist-ing RLS, Rustfmt and Clippy."
This reverts commit 9f792e199b.
2018-03-22 22:43:56 +08:00
kennytm
08526c8a9d
Rollup merge of #49093 - Zoxc:speedup, r=aidanhs
Update submodules in parallel
2018-03-22 22:43:35 +08:00
bors
b176285ba7 Auto merge of #49094 - alexcrichton:print-step-duration, r=kennytm
ci: Print out how long each step takes on CI

This commit updates CI configuration to inform rustbuild that it should print
out how long each step takes on CI. This'll hopefully allow us to track the
duration of steps over time and follow regressions a bit more closesly (as well
as have closer analysis of differences between two builds).

cc #48829
2018-03-22 09:46:06 +00:00
kennytm
9f792e199b
Temporarily disable dist-ing RLS, Rustfmt and Clippy.
Unfortunately we don't have sufficient time to rebuild the cache *and*
distribute everything in `dist-x86_64-linux alt`, the debug assertions are
really slow.

We will re-enable them after the PR has been successfully merged, thus
successfully updating the cache (freeing up 40 minutes), giving us enough
time to build these tools.
2018-03-22 09:45:48 +08:00
kennytm
56a19a9eec
Handle redirects correctly. 2018-03-22 05:28:11 +08:00
kennytm
78bcbb0f96
Download the GCC artifacts from the HTTP server instead of FTP server.
The former seems much more stable, in case the cache becomes invalidated.
2018-03-22 05:28:04 +08:00
David Wood
1392179cdc
Configure the dist-x86_64-linux builder to produce compiler documentation 2018-03-21 19:02:54 +00:00
Alex Crichton
1b5eb17d61 ci: Print out how long each step takes on CI
This commit updates CI configuration to inform rustbuild that it should print
out how long each step takes on CI. This'll hopefully allow us to track the
duration of steps over time and follow regressions a bit more closesly (as well
as have closer analysis of differences between two builds).

cc #48829
2018-03-20 07:17:37 -07:00
John Kåre Alsaker
1578b1edc0 Update submodules in parallel 2018-03-19 04:50:30 +01:00
John Kåre Alsaker
efa9016390 Add a CI job for parallel rustc using x.py check 2018-03-18 17:04:50 +01:00
Alex Crichton
60eb308b42 ci: Run fewer tests on asmjs
Many tests run on the asmjs builder like compile-fail, ui, parse-fail, etc,
aren't actually specific to asm.js. Instead of running redundant test suites
this commit changes things up to only run tests that actually emit JS we then
pass to node.
2018-03-16 12:53:51 -07:00
Alex Crichton
1c4b9c1034 ci: Disable optimized tests for asm.js
Since all tests are compiled with LTO effectively in Emscripten this commit
disables optimizations to hopefully squeeze some more time out of the CI
builders.

Closes #48826
2018-03-16 12:47:49 -07:00
kennytm
68a602efa9
Rollup merge of #48892 - alexcrichton:thinlto-again, r=Mark-Simulacrum
rustbuild: Remove ThinLTO-related configuration

This commit removes some ThinLTO/codegen unit cruft primarily only needed during
the initial phase where we were adding ThinLTO support to rustc itself. The
current bootstrap compiler knows about ThinLTO and has it enabled by default for
multi-CGU builds which are also enabled by default. One CGU builds (aka
disabling ThinLTO) can be achieved by configuring the number of codegen units to
1 for a particular builds.

This also changes the defaults for our dist builders to go back to multiple
CGUs. Unfortunately we're seriously bleeding for cycle time on the bots right
now so we need to recover any time we can.
2018-03-16 01:49:41 +08:00
kennytm
c67e553384
Print /proc/cpuinfo and /proc/meminfo before starting to build. 2018-03-10 19:57:03 +08:00
Alex Crichton
ff227c4a2d rustbuild: Remove ThinLTO-related configuration
This commit removes some ThinLTO/codegen unit cruft primarily only needed during
the initial phase where we were adding ThinLTO support to rustc itself. The
current bootstrap compiler knows about ThinLTO and has it enabled by default for
multi-CGU builds which are also enabled by default. One CGU builds (aka
disabling ThinLTO) can be achieved by configuring the number of codegen units to
1 for a particular builds.

This also changes the defaults for our dist builders to go back to multiple
CGUs. Unfortunately we're seriously bleeding for cycle time on the bots right
now so we need to recover any time we can.
2018-03-09 13:21:37 -08:00
Alex Crichton
7f465abd4f appveyor: Shard more slow MSVC builders
Take two more slow builders and split them in two to get them under 2 hrs
2018-03-07 07:29:44 -08:00
bors
4a316e7483 Auto merge of #48630 - alexcrichton:more-sccache, r=kennytm
rustbuild: Pass `ccache` to build scripts

This is a re-attempt at #48192 hopefully this time with 100% less randomly
[blocking builds for 20 minutes][block]. To work around #48192 the sccache
server is started in the `run.sh` script very early on in the compilation
process.

[block]: https://github.com/rust-lang/rust/issues/48192
2018-03-04 07:09:41 +00:00
Alex Crichton
d69b24805b rust: Import LLD for linking wasm objects
This commit imports the LLD project from LLVM to serve as the default linker for
the `wasm32-unknown-unknown` target. The `binaryen` submoule is consequently
removed along with "binaryen linker" support in rustc.

Moving to LLD brings with it a number of benefits for wasm code:

* LLD is itself an actual linker, so there's no need to compile all wasm code
  with LTO any more. As a result builds should be *much* speedier as LTO is no
  longer forcibly enabled for all builds of the wasm target.
* LLD is quickly becoming an "official solution" for linking wasm code together.
  This, I believe at least, is intended to be the main supported linker for
  native code and wasm moving forward. Picking up support early on should help
  ensure that we can help LLD identify bugs and otherwise prove that it works
  great for all our use cases!
* Improvements to the wasm toolchain are currently primarily focused around LLVM
  and LLD (from what I can tell at least), so it's in general much better to be
  on this bandwagon for bugfixes and new features.
* Historical "hacks" like `wasm-gc` will soon no longer be necessary, LLD
  will [natively implement][gc] `--gc-sections` (better than `wasm-gc`!) which
  means a postprocessor is no longer needed to show off Rust's "small wasm
  binary size".

LLD is added in a pretty standard way to rustc right now. A new rustbuild target
was defined for building LLD, and this is executed when a compiler's sysroot is
being assembled. LLD is compiled against the LLVM that we've got in tree, which
means we're currently on the `release_60` branch, but this may get upgraded in
the near future!

LLD is placed into rustc's sysroot in a `bin` directory. This is similar to
where `gcc.exe` can be found on Windows. This directory is automatically added
to `PATH` whenever rustc executes the linker, allowing us to define a `WasmLd`
linker which implements the interface that `wasm-ld`, LLD's frontend, expects.

Like Emscripten the LLD target is currently only enabled for Tier 1 platforms,
notably OSX/Windows/Linux, and will need to be installed manually for compiling
to wasm on other platforms. LLD is by default turned off in rustbuild, and
requires a `config.toml` option to be enabled to turn it on.

Finally the unstable `#![wasm_import_memory]` attribute was also removed as LLD
has a native option for controlling this.

[gc]: https://reviews.llvm.org/D42511
2018-03-03 20:21:35 -08:00
bors
0be38e1ce3 Auto merge of #48600 - Mark-Simulacrum:rustbuild-updates-2, r=alexcrichton
Remove --host and --target arguments to configure in Dockerfiles

These arguments are passed to the relevant x.py invocation in all cases
anyway. As such, there is no need to separately configure them. x.py
will ignore the configuration when they are passed on the command line
anyway.

r? @alexcrichton
2018-03-04 01:32:57 +00:00
Mark Simulacrum
ab7ce1c3c4 Remove --host and --target arguments to configure
These arguments are passed to the relevant x.py invocation in all cases
anyway. As such, there is no need to separately configure them. x.py
will ignore the configuration when they are passed on the command line
anyway.
2018-03-02 11:28:29 -07:00
Alex Crichton
fdef6a881e rustbuild: Pass ccache to build scripts
This is a re-attempt at #48192 hopefully this time with 100% less randomly
[blocking builds for 20 minutes][block]. To work around #48192 the sccache
server is started in the `run.sh` script very early on in the compilation
process.

[block]: https://github.com/rust-lang/rust/issues/48192
2018-03-01 21:53:46 -08:00
kennytm
a9f940e320
Run the external doc tests in tools job. 2018-02-24 00:54:13 +08:00
kennytm
f6e4751ebe
Disallow toolstate regression at the last week of the 6-week cycle. 2018-02-23 07:09:45 +08:00
kennytm
18d7be3381
Rollup merge of #48120 - matthiaskrgr:typos_src_1, r=alexcrichton
fix typos in src/{bootstrap,ci,etc,lib{backtrace,core,fmt_macros}}

via codespell
2018-02-11 00:53:05 +08:00
Matthias Krüger
7ee3e39f64 fix typos in src/{bootstrap,ci,etc,lib{backtrace,core,fmt_macros}} 2018-02-10 12:22:57 +01:00
kennytm
22c5067ae1
Rollup merge of #48098 - alexcrichton:fix-i686-dist-bootstrap, r=Mark-Simulacrum
ci: Actually bootstrap on i686 dist

Right now the `--build` option was accidentally omitted, so we're bootstraping
from `x86_64` to `i686`. In addition to being slower (more compiles) that's not
actually bootstrapping!
2018-02-10 14:24:07 +08:00
Alex Crichton
6b7b6b63a9 rustc: Upgrade to LLVM 6
The following submodules have been updated for a new version of LLVM:

- `src/llvm`
- `src/libcompiler_builtins` - transitively contains compiler-rt
- `src/dlmalloc`

This also updates the docker container for dist-i686-freebsd as the old 16.04
container is no longer capable of building LLVM. The
compiler-rt/compiler-builtins and dlmalloc updates are pretty routine without
much interesting happening, but the LLVM update here is of particular note.
Unlike previous updates I haven't cherry-picked all existing patches we had on
top of our LLVM branch as we have a [huge amount][patches4] and have at this
point forgotten what most of them are for. Instead I started from the current
`release_60` branch in LLVM and only applied patches that were necessary to get
our tests working and building.

The current set of custom rustc-specific patches included in this LLVM update are:

* rust-lang/llvm@1187443 - this is how we actually implement
  `cfg(target_feature)` for now and continues to not be upstreamed. While a
  hazard for SIMD stabilization this commit is otherwise keeping the status
  quo of a small rustc-specific feature.
* rust-lang/llvm@013f2ec - this is a rustc-specific optimization that we haven't
  upstreamed, notably teaching LLVM about our allocation-related routines (which
  aren't malloc/free). Once we stabilize the global allocator routines we will
  likely want to upstream this patch, but for now it seems reasonable to keep it
  on our fork.
* rust-lang/llvm@a65bbfd - I found this necessary to fix compilation of LLVM in
  our 32-bit linux container. I'm not really sure why it's necessary but my
  guess is that it's because of the absolutely ancient glibc that we're using.
  In any case it's only updating pieces we're not actually using in LLVM so I'm
  hoping it'll turn out alright. This doesn't seem like something we'll want to
  upstream.c
* rust-lang/llvm@77ab1f0 - this is what's actually enabling LLVM to build in our
  i686-freebsd container, I'm not really sure what's going on but we for sure
  probably don't want to upstream this and otherwise it seems not too bad for
  now at least.
* rust-lang/llvm@9eb9267 - we currently suffer on MSVC from an [upstream bug]
  which although diagnosed to a particular revision isn't currently fixed
  upstream (and the bug itself doesn't seem too active). This commit is a
  partial revert of the suspected cause of this regression (found via a
  bisection). I'm sort of hoping that this eventually gets fixed upstream with a
  similar fix (which we can replace in our branch), but for now I'm also hoping
  it's a relatively harmless change to have.

After applying these patches (plus one [backport] which should be [backported
upstream][llvm-back]) I believe we should have all tests working on all
platforms in our current test suite. I'm like 99% sure that we'll need some more
backports as issues are reported for LLVM 6 when this propagates through
nightlies, but that's sort of just par for the course nowadays!

In any case though some extra scrutiny of the patches here would definitely be
welcome, along with scrutiny of the "missing patches" like a [change to pass
manager order](rust-lang/llvm@2717444753), [another change to pass manager
order](rust-lang/llvm@c782febb7b), some [compile fixes for
sparc](rust-lang/llvm@1a83de63c4), and some [fixes for
solaris](rust-lang/llvm@c2bfe0abb).

[patches4]: https://github.com/rust-lang/llvm/compare/5401fdf23...rust-llvm-release-4-0-1
[backport]: https://github.com/rust-lang/llvm/commit/5c54c252db
[llvm-back]: https://bugs.llvm.org/show_bug.cgi?id=36114
[upstream bug]: https://bugs.llvm.org/show_bug.cgi?id=36096

---

The update to LLVM 6 is desirable for a number of reasons, notably:

* This'll allow us to keep up with the upstream wasm backend, picking up new
  features as they start landing.
* Upstream LLVM has fixed a number of SIMD-related compilation errors,
  especially around AVX-512 and such.
* There's a few assorted known bugs which are fixed in LLVM 5 and aren't fixed
  in the LLVM 4 branch we're using.
* Overall it's not a great idea to stagnate with our codegen backend!

This update is mostly powered by #47730 which is allowing us to update LLVM
*independent* of the version of LLVM that Emscripten is locked to. This means
that when compiling code for Emscripten we'll still be using the old LLVM 4
backend, but when compiling code for any other target we'll be using the new
LLVM 6 target. Once Emscripten updates we may no longer need this distinction,
but we're not sure when that will happen!

Closes #43370
Closes #43418
Closes #47015
Closes #47683
Closes rust-lang-nursery/stdsimd#157
Closes rust-lang-nursery/rust-wasm#3
2018-02-09 17:13:14 -08:00
Alex Crichton
9c05babe25 ci: Actually bootstrap on i686 dist
Right now the `--build` option was accidentally omitted, so we're bootstraping
from `x86_64` to `i686`. In addition to being slower (more compiles) that's not
actually bootstrapping!
2018-02-09 10:12:32 -08:00
Oliver Schneider
70717f20f5
Update clippy and miri submodule 2018-02-05 11:36:51 +01:00
Mark Simulacrum
e1f04c04df Disable ThinLTO for dist builds.
Dist builds should always be as fast as we can make them, and since
those run on CI we don't care quite as much for the build being somewhat
slower. As such, we don't automatically enable ThinLTO on builds for the
dist builders.
2018-02-03 18:38:17 -07:00
Alex Crichton
c6daea7c9a rustc: Split Emscripten to a separate codegen backend
This commit introduces a separately compiled backend for Emscripten, avoiding
compiling the `JSBackend` target in the main LLVM codegen backend. This builds
on the foundation provided by #47671 to create a new codegen backend dedicated
solely to Emscripten, removing the `JSBackend` of the main codegen backend in
the process.

A new field was added to each target for this commit which specifies the backend
to use for translation, the default being `llvm` which is the main backend that
we use. The Emscripten targets specify an `emscripten` backend instead of the
main `llvm` one.

There's a whole bunch of consequences of this change, but I'll try to enumerate
them here:

* A *second* LLVM submodule was added in this commit. The main LLVM submodule
  will soon start to drift from the Emscripten submodule, but currently they're
  both at the same revision.
* Logic was added to rustbuild to *not* build the Emscripten backend by default.
  This is gated behind a `--enable-emscripten` flag to the configure script. By
  default users should neither check out the emscripten submodule nor compile
  it.
* The `init_repo.sh` script was updated to fetch the Emscripten submodule from
  GitHub the same way we do the main LLVM submodule (a tarball fetch).
* The Emscripten backend, turned off by default, is still turned on for a number
  of targets on CI. We'll only be shipping an Emscripten backend with Tier 1
  platforms, though. All cross-compiled platforms will not be receiving an
  Emscripten backend yet.

This commit means that when you download the `rustc` package in Rustup for Tier
1 platforms you'll be receiving two trans backends, one for Emscripten and one
that's the general LLVM backend. If you never compile for Emscripten you'll
never use the Emscripten backend, so we may update this one day to only download
the Emscripten backend when you add the Emscripten target. For now though it's
just an extra 10MB gzip'd.

Closes #46819
2018-01-28 18:32:45 -08:00
bors
385ef1514c Auto merge of #47663 - malbarbo:mips-crt-static, r=alexcrichton
Do not assume dynamic linking for musl/mips[el] targets

All musl targets except mips[el] assume static linking by default. This can be [confusing](https://users.rust-lang.org/t/static-cross-compiled-binaries-arent-really-static/6084).

When the musl/mips[el] targets was [added](https://github.com/rust-lang/rust/pull/31298), dynamic linking was chosen because of binary size concerns, and probably also because libunwind [didn't](https://users.rust-lang.org/t/static-cross-compiled-binaries-arent-really-static/6084/8) supported mips.

Now that we have `crt-static` target-feature (the user can choose dynamic link for musl targets), and libunwind [6.0](https://github.com/llvm-mirror/libunwind/commits/release_60) add support to mips, we do not need to assume dynamic linking.
2018-01-28 23:30:15 +00:00
Marco A L Barbosa
2875f825fd Remove musl/libunwind patch for i686
The i686 problem was fixed upstream:
aa805e415f
2018-01-27 20:57:10 -02:00
Alex Crichton
beb756f2aa Revert update of the i686-freebsd builder
Looks like llvm 4.0.0 isn't read for that either.
2018-01-25 14:53:30 -08:00
Alex Crichton
8a9381db01 Rollup merge of #47710 - alexcrichton:llvm-6-compat, r=nikomatsakis
First round of LLVM 6.0.0 compatibility

This includes a number of commits for the first round of upgrading to LLVM 6. There are still [lingering bugs](https://github.com/rust-lang/rust/issues/47683) but I believe all of this will nonetheless be necessary!
2018-01-25 13:49:54 -08:00
Alex Crichton
e9a64996be llvm6: Update FreeBSD images to Ubuntu 18.04
Looks like the clang with 16.04 fails to compile LLVM 6, but it looks like clang
in 18.04 can indeed compile LLVM 6.
2018-01-24 15:34:41 -08:00
Ed Schouten
583b382deb Remove workarounds for cc 1.0.3.
Now that the Rust codebase depends on cc 1.0.4, there is no longer any
need to specify a compiler for CloudABI manually. Cargo will
automatically call into the right compiler executable.
2018-01-24 08:30:17 +01:00
Marco A L Barbosa
9b13e40376 Build musl with -fPIC for all targets 2018-01-23 21:39:17 -02:00
Marco A L Barbosa
abc1061a65 Use libuwind 6.0 for all musl targets 2018-01-23 14:05:31 -02:00
Marco A L Barbosa
68db72d8cd Do not assume dynamic linking for musl/mips[el] targets
All musl targets except mips[el] assume static linking by default. This
can be confusing
https://users.rust-lang.org/t/static-cross-compiled-binaries-arent-really-static/6084

When the musl/mips[el] targets was
[added](https://github.com/rust-lang/rust/pull/31298), dynamic linking
was chosen because of binary size concerns, and probably also because
libunwind
[didn't](https://users.rust-lang.org/t/static-cross-compiled-binaries-arent-really-static/6084/8)
supported mips.

Now that we have `crt-static` target-feature (the user can choose
dynamic link for musl targets), and libunwind
[6.0](https://github.com/llvm-mirror/libunwind/commits/release_60) add
support to mips, we do not need to assume dynamic linking.
2018-01-22 16:14:51 -02:00
Alex Crichton
06913a5b1d Automaticaly calculate beta prerelease numbers
This is a forward-port of:

* 9426dda83d7a928d6ced377345e14b84b0f11c21
* cbfb9858951da7aee22d82178405306fca9decb1

from the beta branch which is used to automatically calculate the beta number
based on the number of merges to the beta branch so far.
2018-01-19 08:57:01 -08:00
Ed Schouten
dcf0cd0ac0 Only enable CloudABI builds for x86-64 for now.
We'll turn on other architectures if it turns out we have enough
capacity.
2018-01-16 23:21:51 +01:00
Ed Schouten
ae4288f9ff Integrate dist-cloudabi into dist-various-2.
As discussed in #47427, let's not have a separate container for doing
CloudABI builds. It's a lot faster if we integrate it into an existing
container, so there's less duplication of what's being built.

Upgrade the existing container to Ubuntu 17.10, which is required for
CloudABI builds. The version of Clang shipped with 16.04 is not recent
enough to support CloudABI properly.
2018-01-16 22:23:18 +01:00
Ed Schouten
5801f95cbc Move dist-cloudabi/ into disabled/.
There is not enough capacity to do automated builds for CloudABI at this
time.
2018-01-16 20:31:09 +01:00
Ed Schouten
24c3a6db9b Add a Docker container for doing automated builds for CloudABI.
Setting up a cross compilation toolchain for CloudABI is relatively
easy. It's just a matter of installing a somewhat recent version of
Clang (5.0 preferred) and installing the corresponding
${target}-cxx-runtime package, containing a set of core C/C++ libraries
(libc, libc++, libunwind, etc).

Eventually it would be nice if we could also run 'x.py test'. That,
however still requires some more work. Both libtest and compiletest
would need to be adjusted to deal with CloudABI's requirement of having
all of an application's dependencies injected. Let's settle for just
doing 'x.py dist' for now.
2018-01-16 20:31:09 +01:00
kennytm
0d199d56ea Rollup merge of #47283 - malbarbo:musl-1.1.18, r=alexcrichton
Update musl to 1.1.18

According to http://www.musl-libc.org/download.html:

This release corrects regressions in glob() and armv4t build failure
introduced in the previous release, and includes an important bug fix
for posix_spawnp in the presence of a large PATH environment variable.
2018-01-13 02:26:27 +08:00
Marco A L Barbosa
882cd3cf0b Add i586-unknown-linux-musl target 2018-01-11 15:57:28 -02:00
Marco A L Barbosa
b5a3f56f75 Update musl to 1.1.18
According to http://www.musl-libc.org/download.html:

This release corrects regressions in glob() and armv4t build failure
introduced in the previous release, and includes an important bug fix
for posix_spawnp in the presence of a large PATH environment variable.
2018-01-08 21:26:47 -02:00
bors
2148bcd5fa Auto merge of #47157 - malbarbo:shared-build-musl, r=alexcrichton
ci: use a shared script to build musl

The dist-x86_64-musl, dist-various-1 and dist-i586-gnu-i686-musl builders had different scripts to build musl. This PR creates an unified script, which makes it easier to add new musl targets and update musl and libunwind (used in the musl targets).

The libunwind is update from 3.7 to 3.9 for dist-x86_64-musl and dist-i586-gnu-i686-musl (dist-various-1 already used 3.9 version).
2018-01-07 03:41:00 +00:00
Alex Crichton
53fd0c50d8 Bump to 1.25.0
* Bump the release version to 1.25
* Bump the bootstrap compiler to the recent beta
* Allow using unstable rustdoc features on beta - this fix has been applied to
  the beta branch but needed to go to the master branch as well.
2018-01-04 07:21:22 -08:00
Marco A L Barbosa
00b5413087 ci: use musl shared script in dist-i586-gnu-i686-musl 2018-01-03 13:49:13 -02:00
Marco A L Barbosa
10a759130e ci: use musl shared script in dist-various-1 (for arm targets)
Update libunwind to 39. This is necessary to build the arm targets
2018-01-03 13:49:13 -02:00
Marco A L Barbosa
ca0499d736 ci: use musl shared script in dist-x86_64-musl 2018-01-03 13:49:13 -02:00
Malo Jaffré
2449230cce Enable compiler docs testing in x86_64-gnu 2018-01-01 14:44:13 +01:00
kennytm
8ed16fe47a
Requires tools to test-pass if the corresponding submodule is updated.
If a PR intends to update a tool but its test has failed, abort the merge
regardless of current channel. This should help the tool maintainers if the
update turns out to be failing due to changes in latest master.
2017-12-30 17:15:40 +08:00
bors
e687205fd0 Auto merge of #47016 - malbarbo:dist-armv5te, r=alexcrichton
Add dist builder for armv5te-unknown-linux-gnueabi (again)

The dist builder was first add in https://github.com/rust-lang/rust/pull/46498 and later remove in https://github.com/rust-lang/rust/pull/46498 because of https://github.com/rust-lang/rust/issues/46822.

https://github.com/rust-lang/rust/issues/46822 seems to be fixed now (I and @green-s have [tested](https://github.com/rust-lang/rust/pull/46498#issuecomment-353901216) it).
2017-12-27 22:39:00 +00:00
bors
503153e950 Auto merge of #46554 - kennytm:45861-step-4-5-6-7-upload-test-result-and-remove-toolstate-toml, r=alexcrichton
[auto-toolstate] Upload the toolstate result to an external git repository, and removes BuildExpectation

This PR consists of 3 commits.

1. (Steps 4–6) The `toolstate.json` output previously collected is now pushed to the https://github.com/rust-lang-nursery/rust-toolstate repository.
2. (Step 7) Revert commit ab018c7, thus removing all traces of `BuildExpectation` and `toolstate.toml`.
3. (Step 8) Adjust CONTRIBUTION.md for the new procedure.

These are the last steps of #45861. After this PR, the toolstate will be automatically computed and published to https://rust-lang-nursery.github.io/rust-toolstate/. There is no need to manage toolstate.toml again.

Closes #45861.
2017-12-26 18:03:00 +00:00
kennytm
44954ab52d
Clarify toolstate names. Move publish.py to a more convenient location. 2017-12-27 00:00:46 +08:00
kennytm
519f92f2aa
Upload the toolstate to the remote repository. 2017-12-27 00:00:45 +08:00
Marco A L Barbosa
606a0a5da0 Add dist builder for armv5te-unknown-linux-gnueabi 2017-12-26 09:21:09 -02:00
kennytm
ec302a2714
Fixup some previous configuration errors. 2017-12-26 02:14:54 +08:00
Jed Davis
f7a0dffc78 Re-do the FreeBSD cross-builds to use Clang and libc++. Fixes #44433.
The main goal here is to use FreeBSD's normal libc++, instead of
statically linking the libstdc++ packaged with GCC, because that
libstdc++ has bugs that cause rustc to deadlock inside LLVM.

But the easiest way to use libc++ is to switch the build from GCC to
Clang, and the Clang package in the Ubuntu image already knows how to
cross-compile (given a sysroot and preferably cross-binutils), so the
toolchain script now uses that instead of building a custom compiler.

This also de-duplicates the `build-toolchain.sh` script.
2017-12-22 02:34:09 -06:00
kennytm
bf0653ea8e
Download the crosstool from GitHub instead of crosstool-ng.org
Temporary workaround since crosstool-ng.org was down. Consider mirroring
the release tarball as a more permanent solution.
2017-12-14 04:50:13 +08:00
Marco A L Barbosa
73e5334df3 Remove armv5te target from dist-various-1
Reverts https://github.com/rust-lang/rust/pull/46498

I must have made some mistake when I tested that commit and thought
armv5te target worked. but testing it now the produced binaries
segfaults
(https://github.com/rust-lang/rust/pull/46498#issuecomment-350599233).

I tried using crosstool-ng and buildroot toolchain (for armv5te) but
the produced binaries also segfaults. Maybe there is a issue with the
target, but I cannot investigate it any further.

I think the best for now is not to distribute the armv5te target.

I'm sorry for what happened.
2017-12-11 22:00:25 -02:00
Marco A L Barbosa
bea65d4041 Add armv5te-unknown-linux-gnueabi to cross builder 2017-12-05 07:46:24 -02:00
Marco A L Barbosa
79b47a1f26 Rename cross(2) builder to dist-various-{1,2}
Follows the convention of the other builders.
2017-12-04 16:38:19 -02:00
kennytm
183964505b
Update the tools CI to use --no-fail-fast and --save-toolstates. 2017-12-03 18:36:56 +08:00
kennytm
128199e39c
Move the swap the tools test and cargotest within check-aux.
The cargotest job is renamed to tools for clarification.
2017-12-03 18:36:56 +08:00
Alex Crichton
73970bf6f2 ci: Start running wasm32 tests on Travis
This commit allocates a builder to running wasm32 tests on Travis. Not all test
suites pass right now so this is starting out with just the run-pass and the
libcore test suites. This'll hopefully give us a pretty broad set of coverage
for integration in rustc itself as well as a somewhat broad coverage of the llvm
backend itself through integration/unit tests.
2017-11-28 09:27:35 -08:00
Alex Crichton
48996f9e75 rustbuild: Enable WebAssembly backend by default
This commit alters how we compile LLVM by default enabling the WebAssembly
backend. This then also adds the wasm32-unknown-unknown target to get compiled
on the `cross` builder and distributed through rustup. Tests are not yet enabled
for this target but that should hopefully be coming soon!
2017-11-25 06:44:35 -08:00
Martin Lindhe
ece9a57d1b fix some typos 2017-11-21 15:33:45 +01:00
bors
3bcb00dbc2 Auto merge of #45991 - gnzlbg:fix_i586, r=alexcrichton
fix linking error on i586

Try to fix this linking error on i586 in cross:

https://travis-ci.org/japaric/cross/builds/302095949#L8670

The problem is that `std` is built in Ubuntu 16.04 and `cross` uses a linker from 12.04.

Currently this fix solves the problem for `i686-musl`  making it "supercompatible", this PR applies the fix to `i586` as well.

The cross PR is here: https://github.com/japaric/cross/pull/157
2017-11-17 05:18:45 +00:00
gnzlbg
5146663602 fix linking error on i586 2017-11-14 21:35:05 +01:00
Guillaume Gomez
529bb320f7 Rollup merge of #45961 - dereckson:unix-agnosticity-ci, r=Mark-Simulacrum
Use #!/usr/bin/env as shebang for Bash scripts

On some systems, the bash command could be available in another
directory than /bin. As such, to offer an env shebang is more
convenient.

This make sense even for docker scripts, as you can use Docker
on FreeBSD or SmartOS for example.
2017-11-14 16:52:11 +01:00
Sébastien Santoro
de8c57cb24 Use #!/usr/bin/env as shebang for Bash scripts
On some systems, the bash command could be available in another
directory than /bin. As such, to offer an env shebang is more
convenient.

This make sense even for docker scripts, as you can use Docker
on FreeBSD or SmartOS for example.
2017-11-13 14:33:12 +00:00
bors
b7ccb0a5a7 Auto merge of #45810 - SimonSapin:ac-dc, r=aturon
Disable LLVM assertions on Nightly, enable them in "alt" builds.

Per IRC discussion https://mozilla.logbot.info/rust-infra/20171106#c13812170-c13812204

Background: https://internals.rust-lang.org/t/disabling-llvm-assertions-in-nightly-builds/5388/14
2017-11-13 11:46:55 +00:00
bors
7f43981ebf Auto merge of #45623 - mneumann:dragonfly-ci, r=alexcrichton
Add ci for DragonFly
2017-11-12 23:31:58 +00:00
Michael Neumann
949abfe7d5 Add ci for DragonFly (disabled for now) 2017-11-12 19:00:35 +01:00
Simon Sapin
325a769a21 Disable LLVM assertions on Nightly, enable them in "alt" builds. 2017-11-11 23:31:37 +01:00
kennytm
33400fbbcd
Modify the script to allow for running docker images on Windows 7. 2017-11-06 03:53:40 +08:00
kennytm
1b5aaf22e8
dist-powerpc64le-linux: Use links from vault.centos.org instead.
This commit reverts #45734 and applies #45744. We expect the vault links
to be more stable than mirror.centos.org.
2017-11-06 03:53:40 +08:00
kennytm
1f95c6de3c
Fix broken link in build-powerpc64le-toolchain.sh 2017-11-03 15:53:36 +08:00
kennytm
c46b04cbdd
ci: Upgrade Android SDK/NDK and refactor to use sdkmanager/avdmanager.
* SDK tools is upgraded to 27.0.0.
   - Refactored to use `sdkmanager`/`avdmanager` instead of the deprecated
     `android` tool.

* The Java version used by Android SDK is downgraded to OpenJDK-8, in order
  to download the SDK through HTTPS.

* NDK is upgrade to r15c.
   - Dropped support for android-9 (2.3 / Gingerbread), the minimal
     supported version is now android-14 (4.0 / Ice Cream Sandwich).
   - Changed the default Android compiler from GCC to clang.
   - For details of change introduced by NDK r15, see
     https://github.com/android-ndk/ndk/wiki/Changelog-r15.
2017-10-29 05:58:00 +08:00
Alex Crichton
36822ba39a ci: Test more asmjs again
This was accidentally added in #45352, forgot to back it out!
2017-10-25 09:58:48 -07:00