These commits build on [some great work on reddit](http://www.reddit.com/r/rust/comments/33boew/weekend_experiment_link_rust_programs_against/) for adding MUSL support to the compiler. This goal of this PR is to enable a `--target x86_64-unknown-linux-musl` argument to the compiler to work A-OK. The outcome here is that there are 0 compile-time dependencies for a MUSL-targeting build *except for a linker*. Currently this also assumes that MUSL is being used for statically linked binaries so there is no support for dynamically linked binaries with MUSL.
MUSL support largely just entailed munging around with the linker and where libs are located, and the major highlights are:
* The entirety of `libc.a` is included in `liblibc.rlib` (statically included as an archive).
* The entirety of `libunwind.a` is included in `libstd.rlib` (like with liblibc).
* The target specification for MUSL passes a number of ... flavorful options! Each option is documented in the relevant commit.
* The entire test suite currently passes with MUSL as a target, except for:
* Dynamic linking tests are all ignored as it's not supported with MUSL
* Stack overflow detection is not working MUSL yet (I'm not sure why)
* There is a language change included in this PR to add a `target_env` `#[cfg]` directive. This is used to conditionally build code for only MUSL (or for linux distros not MUSL). I highly suspect that this will also be used by Windows to target MSVC instead of a MinGW-based toolchain.
To build a compiler targeting MUSL you need to follow these steps:
1. Clone the current MUSL repo from `git://git.musl-libc.org/musl`. Build this as usual and install it.
2. Clone and build LLVM's [libcxxabi](http://libcxxabi.llvm.org/) library. Only the `libunwind.a` artifact is needed. I have tried using upstream libunwind's source repo but I have not gotten unwinding to work with it unfortunately. Move `libunwind.a` adjacent to MUSL's `libc.a`
3. Configure a Rust checkout with `--target=x86_64-unknown-linux-musl --musl-root=$MUSL_ROOT` where `MUSL_ROOT` is where you installed MUSL in step 1.
I hope to improve building a copy of libunwind as it's still a little sketchy and difficult to do today, but other than that everything should "just work"! This PR is not intended to include 100% comprehensive support for MUSL, as future modifications will probably be necessary.
There were a few test cases to fix:
* Dynamic libraries are not supported with MUSL right now, so all of those
related test which force or require dylibs are ignored.
* Looks like the default stack for MUSL is smaller than glibc, so a few stack
allocations in benchmarks were boxed up (shouldn't have a perf impact).
* Some small linkage tweaks here and there
* Out-of-stack detection does not currently work with MUSL
This commit adds support to the makefiles, configuration script, and build
system to understand MUSL. This is broken up into a few parts:
* Any target of the form `*-musl` requires the `--musl-root` option to
`./configure` which will indicate the root of the MUSL installation. It is
also expected that there is a libunwind build inside of that installation
built against that MUSL.
* Objects from MUSL are copied into the build tree for Rust to be statically
linked into the appropriate Rust library.
* Objects for binary startup and shutdown are included in each Rust installation
by default for MUSL. This requires MUSL to only be installed on the machine
compiling rust. Only a linker will be necessary for compiling against MUSL on
a target machine.
Eventually a MUSL and/or libunwind build may be integrated by default into the
build but for now they are just always assumed to exist externally.
In most places in mk/tests.mk, it's positioned after rpass-full and
before cfail-full (because rfail comes before cfail). The order of tests
seems a little inconsistent, but reordering everywhere would obscure this
commit.
This required fixing the `pretty-rpass-full` tests to have the same `$$(CSREQ$(1)_T_$(2)_H_$(3))` dependencies as the `rpass-full` and `cfail-full` tests. It also required fixing the `run-make/simd-ffi` test to use unique names for its output files.
The current code attempts to define the
PRETTY_DEPS$(1)_H_$(3)_pretty-rpass-full variable, which does not work,
because $(1) and $(3) are not inside a function. Moreover, there is a test
(run-pass-fulldeps/compiler-calls.rs) that uses rustc_driver, which is not
an indirect dependency of librustc or libsyntax. Listing all the
dependencies will be hard to maintain, but there's a better way to do
this...
As with the rpass-full and cfail-full tests, add dependencies using the
$$(CSREQ$(1)_T_$(3)_H_$(3)) variable, which includes the complete set of
host and target crates, built for a particular stage and host. We use
T_$(3), not T_$(2), because we only build LLVM for host triples (not
target triples), so we can only build rustc_llvm for host triples. The
fulldeps tests that use plugins need host rustc crates, whereas fulldeps
tests that link against rustc and run should be skipped for
cross-compilation (such as Android).
Fixes#22021
Instead of rustc-1.0.0-beta-$triple.tar.gz, betas will be named
rustc-beta-$triple.tar.gz. This will give betas a stable download
URL, prevent old artifacts from accumulating in the dist server's
root directory, and not require the website to be updated every
beta.
As a tradeoff, it will be harder to download previous betas because
they will need to be located in the archives.
This patch
1. renames libunicode to librustc_unicode,
2. deprecates several pieces of libunicode (see below), and
3. removes references to deprecated functions from
librustc_driver and libsyntax. This may change pretty-printed
output from these modules in cases involving wide or combining
characters used in filenames, identifiers, etc.
The following functions are marked deprecated:
1. char.width() and str.width():
--> use unicode-width crate
2. str.graphemes() and str.grapheme_indices():
--> use unicode-segmentation crate
3. str.nfd_chars(), str.nfkd_chars(), str.nfc_chars(), str.nfkc_chars(),
char.compose(), char.decompose_canonical(), char.decompose_compatible(),
char.canonical_combining_class():
--> use unicode-normalization crate
This patch
1. renames libunicode to librustc_unicode,
2. deprecates several pieces of libunicode (see below), and
3. removes references to deprecated functions from
librustc_driver and libsyntax. This may change pretty-printed
output from these modules in cases involving wide or combining
characters used in filenames, identifiers, etc.
The following functions are marked deprecated:
1. char.width() and str.width():
--> use unicode-width crate
2. str.graphemes() and str.grapheme_indices():
--> use unicode-segmentation crate
3. str.nfd_chars(), str.nfkd_chars(), str.nfc_chars(), str.nfkc_chars(),
char.compose(), char.decompose_canonical(), char.decompose_compatible(),
char.canonical_combining_class():
--> use unicode-normalization crate
This commit series starts out with more official test harness support for rustdoc tests, and then each commit afterwards adds a test (where appropriate). Each commit should also test and finish independently of all others (they're all pretty separable).
I've uploaded a [copy of the documentation](http://people.mozilla.org/~acrichton/doc/std/) generated after all these commits were applied, and a double check on issues being closed would be greatly appreciated! I'll also browse the docs a bit and make sure nothing regressed too horribly.
This commit ceases documentation-by-default of crates such as `term`,
`serialize`, and `alloc`. Crates like `term` and `rand` have duplicates on
`crates.io` and the search index entries generated in the local tree end up
only leading to confusion. Crates like the entire compiler infrastructure,
`flate`, or `rbml` don't need to be documented in such a prominent location.
This change also means that doc tests will no longer be run for crates beyond
the facade (e.g. `serialize` or `term`), but there were very few doc tests in
there to begin with.
Closes#22168
Add a new test directory called 'rustdoc' where all files inside are documented
and run against the `htmldocck` script to have assertions about the output.
Rationale for this, is that I lurked `ulimit -c unlimited` into my .profile to debug an unrelated crash, that I kept forgetting to set before hand. I then ran the test suite and discovered that I had 150 gigs of core dumps in `/cores`.
Very open to another approach, or to setting the limit to something higher than 0, but I think it would be nice if the build system tried to save you from yourself here.
This saves a bunch of a time and will make distributions smaller, as well as
avoiding filling the implementors page with internal garbage. Turn it back on
with `--enable-compiler-docs` if you want them.
(Crates behind the facade are not documented at all)
[breaking-change]
This saves a bunch of a time and will make distributions smaller, as well as
avoiding filling the implementors page with internal garbage. Turn it back on
with `--enable-compiler-docs` if you want compiler docs during development.
Crates behind the facade are only documented on nightly/dev builds (where they
can be used).
[breaking-change]
Closes#23772Closes#21297
The RUST_TARGET_STAGE_N rule uses LLVM_LIBDIR_RUSTFLAGS_<target-triple>,
which expands to -L "$(llvm-config --libdir)" when the target-triple is
also a host-triple. Rather than expand to -L "" if llvm-config has not yet
been built, add a dependency on the target llvm-config.
When the target-triple is not a host-triple, the new LLVM_CONFIG_$(2)
dependency should expand to nothing.
r? alexcrichton
The RUST_TARGET_STAGE_N rule uses LLVM_LIBDIR_RUSTFLAGS_<target-triple>,
which expands to -L "$(llvm-config --libdir)" when the target-triple is
also a host-triple. Rather than expand to -L "" if llvm-config has not yet
been built, add a dependency on the target llvm-config.
When the target-triple is not a host-triple, the new LLVM_CONFIG_$(2)
dependency should expand to nothing.
LLVM_LIBDIR_<triple> is only defined for host triples, not target triples.
FWIW, the same is true for LLVM_STDCPP_RUSTFLAGS_<triple>, where we
explicitly define it as empty when --enable-llvm-static-stdcpp is not
specified, but it's still undefined for cross-compiled triples.
This commit is an implementation of [RFC 563][rfc] which adds a new
`cfg(debug_assertions)` directive which is specially recognized and calculated
by the compiler. The flag is turned off at any optimization level greater than 1
and may also be explicitly controlled through the `-C debug-assertions`
flag.
[rfc]: https://github.com/rust-lang/rfcs/pull/563
The `debug_assert!` and `debug_assert_eq!` macros now respect this instead of
the `ndebug` variable and `ndebug` no longer holds any meaning to the standard
library.
Code which was previously relying on `not(ndebug)` to gate expensive code should
be updated to rely on `debug_assertions` instead.
Closes#22492
[breaking-change]
This commit deprecates the majority of std::old_io::fs in favor of std::fs and
its new functionality. Some functions remain non-deprecated but are now behind a
feature gate called `old_fs`. These functions will be deprecated once
suitable replacements have been implemented.
The compiler has been migrated to new `std::fs` and `std::path` APIs where
appropriate as part of this change.
Strategy: If the end goal is to touch e.g. `stamp.std`, then we first
touch `stamp.std.start_time` before doing anything else. Then when
the receipe finishes, we touch `stamp.std` using the timestamp from
`stamp.std.start_time` as the reference time, and remove
`stamp.std.start_time`.
Fix#6518.
This pulls out the implementations of most built-in lints into a
separate crate, to reduce edit-compile-test iteration times with
librustc_lint and increase parallelism. This should enable lints to be
refactored, added and deleted much more easily as it slashes the
edit-compile cycle to get a minimal working compiler to test with (`make
rustc-stage1`) from
librustc -> librustc_typeck -> ... -> librustc_driver ->
libcore -> ... -> libstd
to
librustc_lint -> librustc_driver -> libcore -> ... libstd
which is significantly faster, mainly due to avoiding the librustc build
itself.
The intention would be to move as much as possible of the infrastructure
into the crate too, but the plumbing is deeply intertwined with librustc
itself at the moment. Also, there are lints for which diagnostics are
registered directly in the compiler code, not in their own crate
traversal, and their definitions have to remain in librustc.
This is a [breaking-change] for direct users of the compiler APIs:
callers of `rustc::session::build_session` or
`rustc::session::build_session_` need to manually call
`rustc_lint::register_builtins` on their return value.
This should make #22206 easier.
After making `rustc` fail on errors at a stop point, like `-Z parse-only`, in #22117, the files in this PR also fail during the parse stage and should be moved as well. Sorry for spliting this move up in two PRs.
- Now "make check-stage2-T-aarch64-linux-android-H-x86_64-unknown-linux-gnu" works (#21773)
- Fix & enable debuginfo tests for android (#10381)
- Fix & enable more tests for android (both for arm/aarch64)
- Enable many already-pass tests on android (both for arm/aarch64)
`cp -a` is a GNU extension. Use an alternate combinaison of POSIX options
(`-PRp`) that do nearly the same.
The difference is `-a` will preserve context, links and xattr attributes,
whereas `-p` not. But as we use it only for copy a file, there is no
difference in the current context.
This is a patch for #22291.
PLEASE_BENCH=1 adds --bench to the arguments passed to the executable to be tested. At the moment, compiletest does not accept a --bench argument, because it is not needed for any test in src/test/, even the tests in src/test/bench do not use #[bench].
I have updated the makefile to only add the --bench flag for crate tests. I do not think that changing compiletest add --bench to the run arguments of all compile tests makes sense, because it would mess up tests which check command line arguments. Also the bench option can be added as comment in a compile test as well.
Highlights:
* Adds an 'uninstall.sh' script to `/usr/local/lib/rustlib/uninstall.sh`, the path to which is printed during installation.
* Components can be deselected during install, like `install.sh --without=rust-docs`.
* Components can be listed with `install.sh --list-components`.
* Vastly reduces spew during install (but supporting a `--verbose` option).
Typicall install run looks like:
```
brian@brianX1:~/dev/multirust⟫ sudo ./install.sh
[sudo] password for brian:
install: creating uninstall script at /usr/local/lib/rustlib/uninstall.sh
install: installing component 'rustc'
install: installing component 'cargo'
install: installing component 'rust-docs'
Rust is ready to roll.
```
Needs to be merged right before corresponding PRs to cargo and rust-packaging.
Fixes https://github.com/rust-lang/rust/issues/21117
Fixes https://github.com/rust-lang/rust/issues/20283
`cp -a` is a GNU extension. Use an alternate combinaison of POSIX options
(`-PRp`) that do nearly the same.
The difference is `-a` will preserve context, links and xattr attributes,
whereas `-p` not. But as we use it only for copy a file, there is no
difference in the current context.
This restructures tidy.py to walk the tree itself,
and improves performance considerably by not loading entire
files into buffers for licenseck.
Splits build rules into 'tidy', 'tidy-basic', 'tidy-binaries',
'tidy-errors', 'tidy-features'.
This adds a new lexer/parser combo for the entire Rust language can be generated with with flex and bison, taken from my project at https://github.com/bleibig/rust-grammar. There is also a testing script that runs the generated parser with all *.rs files in the repository (except for tests in compile-fail or ones that marked as "ignore-test" or "ignore-lexer-test"). If you have flex and bison installed, you can run these tests using the new "check-grammar" make target.
This does not depend on or interact with the existing testing code in the grammar, which only provides and tests a lexer specification.
OS X users should take note that the version of bison that comes with the Xcode toolchain (2.3) is too old to work with this grammar, they need to download and install version 3.0 or later.
The parser builds up an S-expression-based AST, which can be displayed by giving the "-v" argument to parser-lalr (normally it only gives output on error). It is only a rough approximation of what is parsed and doesn't capture every detail and nuance of the program.
Hopefully this should be sufficient for issue #2234, or at least a good starting point.
The regex library was largely used for non-critical aspects of the compiler and
various external tooling. The library at this point is duplicated with its
out-of-tree counterpart and as such imposes a bit of a maintenance overhead as
well as compile time hit for the compiler itself.
The last major user of the regex library is the libtest library, using regexes
for filters when running tests. This removal means that the filtering has gone
back to substring matching rather than using regexes.
Initial support for aarch64-linux-android (#18920)
- Add new configuration files
- Modify some options to compile & link succesfully.
(PIE, disable tls on jemalloc, modify some external function linkage, ..)
- To build, refer to https://github.com/rust-lang/rust/wiki/Doc-building-for-android.
(tested with platform=21 and toolchain=aarch64-linux-android-4.9)
This does the bare minimum to make registration of error codes work again. After this patch, every call to `span_err!` with an error code gets that error code validated against a list in that crate and a new tidy script `errorck.py` validates that no error codes are duplicated globally.
There are further improvements to be made yet, detailed in #19624.
r? @nikomatsakis
Removed use of unused LDPATH variable on Windows as is done for other platforms, and added GCC flag to ensure MINGW's ANSI compatible STDIO functions are used wherever available (required by jemalloc).
Without these changes it ends up setting the PATH twice, and the second time the PATH begins with `:` which is invalid. Also the regular msvcrt printf-like functions would be used which don't understand stuff like %hhd and %z which jemalloc uses.
This change ought not to make any difference to the output but it fixes the build process for me since at least my build environment couldn't handle that broken path caused by LDPATH being empty.
The script is intended as a tool for doing every sort of verifications amenable to Rustdoc's HTML output. For example, link checkers would go to this script. It already parses HTML into a document tree form (with a slight caveat), so future tests can make use of it.
As an example, relevant `rustdoc-*` run-make tests have been updated to use `htmldocck.py` and got their `verify.sh` removed. In the future they may go to a dedicated directory with htmldocck running by default. The detailed explanation of test scripts is provided as a docstring of htmldocck.
cc #19723
The script is intended as a tool for doing every sort of verifications
amenable to Rustdoc's HTML output. For example, link checkers would go
to this script. It already parses HTML into a document tree form (with
a slight caveat), so future tests can make use of it.
As an example, relevant `rustdoc-*` run-make tests have been updated
to use `htmldocck.py` and got their `verify.sh` removed. In the future
they may go to a dedicated directory with htmldocck running by default.
The detailed explanation of test scripts is provided as a docstring of
htmldocck.
cc #19723
In accordance with [collections reform part 2][rfc] this macro has been moved to
an external [bitflags crate][crate] which is [available though
crates.io][cratesio]. Inside the standard distribution the macro has been moved
to a crate called `rustc_bitflags` for current users to continue using.
[rfc]: https://github.com/rust-lang/rfcs/blob/master/text/0509-collections-reform-part-2.md
[crate]: https://github.com/rust-lang/bitflags
[cratesio]: http://crates.io/crates/bitflags
The major user of `bitflags!` in terms of a public-facing possibly-stable API
today is the `FilePermissions` structure inside of `std::io`. This user,
however, will likely no longer use `bitflags!` after I/O reform has landed. To
prevent breaking APIs today, this structure remains as-is.
Current users of the `bitflags!` macro should add this to their `Cargo.toml`:
bitflags = "0.1"
and this to their crate root:
#[macro_use] extern crate bitflags;
Due to the removal of a public macro, this is a:
[breaking-change]
Originally, this was going to be discussed and revisted, however I've been working on this for months, and a rebase on top of master was about 1 flight's worth of work so I just went ahead and did it.
This gets you as far as being able to target powerpc with, eg:
LD_LIBRARY_PATH=./x86_64-unknown-linux-gnu/stage2/lib/ x86_64-unknown-linux-gnu/stage2/bin/rustc -C linker=powerpc-linux-gnu-gcc --target powerpc-unknown-linux-gnu hello.rs
Would really love to get this out before 1.0. r? @alexcrichton
There is likely to be new users with the alpha release, and there are a lot of documents on the internet (StackOverflow, reddit, blogs) that refer to these guides, so emitting a more helpful error than "404" is nice. Hence, I've temporarily reinstated stub documents for each of the old guides, referring to as relevant a part of the book as possible.
Also, rustbook was silently ignoring some errors, which lead to an inconsistency with directory creation/file writing. This meant the CSS file was not being written if no `doc` directory existed in the users build dir (e.g. the buildbots). This should mean that the CSS will appear automatically in later builds.
Without this, rustbook was failing because it was expecting the
directory to exist. (Previously, rustbook was just silently failing to
install the CSS files due to this.)
There are hundreds of stackoverflow answers, reddit posts and blog
articles that link to these documents, so it's a nicer user experience
if they're not plain 404s.
The intention is to let these hang around only for relatively short
while. The alpha is likely to bring in many new users and they will be
reading the documents mentioned above.
This pulls all of our long-form documentation into a single document,
nicknamed "the book" and formally titled "The Rust Programming
Language."
A few things motivated this change:
* People knew of The Guide, but not the individual Guides. This merges
them together, helping discoverability.
* You can get all of Rust's longform documentation in one place, which
is nice.
* We now have rustbook in-tree, which can generate this kind of
documentation. While its style is basic, the general idea is much
better: a table of contents on the left-hand side.
* Rather than a almost 10,000-line guide.md, there are now smaller files
per section.
This partially implements the feature staging described in the
[release channel RFC][rc]. It does not yet fully conform to the RFC as
written, but does accomplish its goals sufficiently for the 1.0 alpha
release.
It has three primary user-visible effects:
* On the nightly channel, use of unstable APIs generates a warning.
* On the beta channel, use of unstable APIs generates a warning.
* On the beta channel, use of feature gates generates a warning.
Code that does not trigger these warnings is considered 'stable',
modulo pre-1.0 bugs.
Disabling the warnings for unstable APIs continues to be done in the
existing (i.e. old) style, via `#[allow(...)]`, not that specified in
the RFC. I deem this marginally acceptable since any code that must do
this is not using the stable dialect of Rust.
Use of feature gates is itself gated with the new 'unstable_features'
lint, on nightly set to 'allow', and on beta 'warn'.
The attribute scheme used here corresponds to an older version of the
RFC, with the `#[staged_api]` crate attribute toggling the staging
behavior of the stability attributes, but the user impact is only
in-tree so I'm not concerned about having to make design changes later
(and I may ultimately prefer the scheme here after all, with the
`#[staged_api]` crate attribute).
Since the Rust codebase itself makes use of unstable features the
compiler and build system to a midly elaborate dance to allow it to
bootstrap while disobeying these lints (which would otherwise be
errors because Rust builds with `-D warnings`).
This patch includes one significant hack that causes a
regression. Because the `format_args!` macro emits calls to unstable
APIs it would trigger the lint. I added a hack to the lint to make it
not trigger, but this in turn causes arguments to `println!` not to be
checked for feature gates. I don't presently understand macro
expansion well enough to fix. This is bug #20661.
Closes#16678
[rc]: https://github.com/rust-lang/rfcs/blob/master/text/0507-release-channels.md
After 8b3c67690c the `make install`
command fails if docs are not disabled through CFG_DISABLE_DOCS,
because now the `install` target uses
../../tmp/dist/$(DOC_PKG_NAME)-$(CFG_BUILD)/install.sh
Instead of explicitly depending on
dist/$(PKG_NAME)-$(CFG_BUILD).tar.gz, the `prepare_[un]install`
targets now depend on `dist-tar-bins`, which packages the appropriate
dist archives depending on the configuration.
This removes a large array of deprecated functionality, regardless of how
recently it was deprecated. The purpose of this commit is to clean out the
standard libraries and compiler for the upcoming alpha release.
Some notable compiler changes were to enable warnings for all now-deprecated
command line arguments (previously the deprecated versions were silently
accepted) as well as removing deriving(Zero) entirely (the trait was removed).
The distribution no longer contains the libtime or libregex_macros crates. Both
of these have been deprecated for some time and are available externally.
After 8b3c67690c the `make install`
command fails if docs are not disabled through CFG_DISABLE_DOCS,
because now the `install` target uses
../../tmp/dist/$(DOC_PKG_NAME)-$(CFG_BUILD)/install.sh
In 714a2c678c the `prepare_install`
target wwas changed to conditionally depend also on the doc archive,
but did not modify `prepare_uninstall`.
Instead of explicitly depending on
dist/$(PKG_NAME)-$(CFG_BUILD).tar.gz, the `prepare_[un]install`
targets now depend on `dist-tar-bins`, which packages the appropriate
dist archives depending on the configuration.
