This tells trans:🔙:write not to LLVM codegen to create .o
files but to put LLMV bitcode in .o files.
Emscripten's emcc supports .o in this format, and this is,
I think, slightly easier than making rlibs work without .o
files.
Backtraces, and the compilation of libbacktrace for asmjs, are disabled.
This port doesn't use jemalloc so, like pnacl, it disables jemalloc *for all targets*
in the configure file.
It disables stack protection.
This pull request adds support for [Illumos](http://illumos.org/)-based operating systems: SmartOS, OpenIndiana, and others. For now it's x86-64 only, as I'm not sure if 32-bit installations are widespread. This PR is based on #28589 by @potatosalad, and also closes#21000, #25845, and #25846.
Required changes in libc are already merged: https://github.com/rust-lang-nursery/libc/pull/138
Here's a snapshot required to build a stage0 compiler:
https://s3-eu-west-1.amazonaws.com/nbaksalyar/rustc-sunos-snapshot.tar.gz
It passes all checks from `make check`.
There are some changes I'm not quite sure about, e.g. macro usage in `src/libstd/num/f64.rs` and `DirEntry` structure in `src/libstd/sys/unix/fs.rs`, so any comments on how to rewrite it better would be greatly appreciated.
Also, LLVM configure script might need to be patched to build it successfully, or a pre-built libLLVM should be used. Some details can be found here: https://llvm.org/bugs/show_bug.cgi?id=25409
Thanks!
r? @brson
Currently the `mipsel-unknown-linux-gnu` target doesn't actually set the
`target_arch` value to `mipsel` but it rather uses `mips`. Alternatively the
`powerpc64le` target does indeed set the `target_arch` as `powerpc64le`,
causing a bit of inconsistency between theset two.
As these are just the same instance of one instruction set, let's use
`target_endian` to switch between them and only set the `target_arch` as one
value. This should cut down on the number of `#[cfg]` annotations necessary and
all around be a little more ergonomic.
Currently the `mipsel-unknown-linux-gnu` target doesn't actually set the
`target_arch` value to `mipsel` but it rather uses `mips`. Alternatively the
`powerpc64le` target does indeed set the `target_arch` as `powerpc64le`,
causing a bit of inconsistency between theset two.
As these are just the same instance of one instruction set, let's use
`target_endian` to switch between them and only set the `target_arch` as one
value. This should cut down on the number of `#[cfg]` annotations necessary and
all around be a little more ergonomic.
Currently any compilation to MIPS spits out the warning:
'generic' is not a recognized processor for this target (ignoring processor)
Doesn't make for a great user experience! We don't encounter this in the normal
bootstrap because the cpu/feature set are set by the makefiles. Instead let's
just propagate these to the defaults for the entire target all the time (still
overridable from the command line) and prevent warnings from being emitted by
default.
This target covers MIPS devices that run the trunk version of OpenWRT.
The x86_64-unknown-linux-musl target always links statically to C libraries. For
the mips(el)-unknown-linux-musl target, we opt for dynamic linking (like most of
other targets do) to keep binary size down.
As for the C compiler flags used in the build system, we use the same flags used
for the mips(el)-unknown-linux-gnu target.
r? @alexcrichton
Currently any compilation to MIPS spits out the warning:
'generic' is not a recognized processor for this target (ignoring processor)
Doesn't make for a great user experience! We don't encounter this in the normal
bootstrap because the cpu/feature set are set by the makefiles. Instead let's
just propagate these to the defaults for the entire target all the time (still
overridable from the command line) and prevent warnings from being emitted by
default.
These commits perform a few high-level changes with the goal of enabling i686 MSVC unwinding:
* LLVM is upgraded to pick up the new exception handling instructions and intrinsics for MSVC. This puts us somewhere along the 3.8 branch, but we should still be compatible with LLVM 3.7 for non-MSVC targets.
* All unwinding for MSVC targets (both 32 and 64-bit) are implemented in terms of this new LLVM support. I would like to also extend this to Windows GNU targets to drop the runtime dependencies we have on MinGW, but I'd like to land this first.
* Some tests were fixed up for i686 MSVC here and there where necessary. The full test suite should be passing now for that target.
In terms of landing this I plan to have this go through first, then verify that i686 MSVC works, then I'll enable `make check` on the bots for that target instead of just `make` as-is today.
Closes#25869
This commit transitions the compiler to using the new exception handling
instructions in LLVM for implementing unwinding for MSVC. This affects both 32
and 64-bit MSVC as they're both now using SEH-based strategies. In terms of
standard library support, lots more details about how SEH unwinding is
implemented can be found in the commits.
In terms of trans, this change necessitated a few modifications:
* Branches were added to detect when the old landingpad instruction is used or
the new cleanuppad instruction is used to `trans::cleanup`.
* The return value from `cleanuppad` is not stored in an `alloca` (because it
cannot be).
* Each block in trans now has an `Option<LandingPad>` instead of `is_lpad: bool`
for indicating whether it's in a landing pad or not. The new exception
handling intrinsics require that on MSVC each `call` inside of a landing pad
is annotated with which landing pad that it's in. This change to the basic
block means that whenever a `call` or `invoke` instruction is generated we
know whether to annotate it as part of a cleanuppad or not.
* Lots of modifications were made to the instruction builders to construct the
new instructions as well as pass the tagging information for the call/invoke
instructions.
* The translation of the `try` intrinsics for MSVC has been overhauled to use
the new `catchpad` instruction. The filter function is now also a
rustc-generated function instead of a purely libstd-defined function. The
libstd definition still exists, it just has a stable ABI across architectures
and leaves some of the really weird implementation details to the compiler
(e.g. the `localescape` and `localrecover` intrinsics).
This target covers MIPS devices that run the trunk version of OpenWRT.
The x86_64-unknown-linux-musl target always links statically to C libraries. For
the mips(el)-unknown-linux-musl target, we opt for dynamic linking (like most of
other targets do) to keep binary size down.
As for the C compiler flags used in the build system, we use the same flags used
for the mips(el)-unknown-linux-gnu target.
This explicitly adds an option telling the linker on these platforms to make the stack and heap non-executable (should already be the case for Windows, and likely OS X).
Without this option there is some risk of accidentally losing NX protection, as the linker will not enable NX if any of the binary's constituent objects don't contain the .note.GNU-stack header.
We're not aware of any users who would want a binary with executable stack or heap, but in future this could be made possible by passing a flag to disable the protection, which would also help document the fact to the crate's users.
Edit: older discussion of previous quickfix to add a .note.GNU-stack header to libunwind's assembly:
Short term solution for issue #30824 to ensure that object files generated from assembler contain the .note.GNU-stack header.
When this header is not present in any constituent object files, the linker refrains from making the stack NX in the final executable.
Further actions:
I'll try to get this change merged in with upstream too, and then update these instructions to just compile the fixed version.
It seems a good idea to use issue #30824 or some other issue to add a test that similar security regressions can be automatically caught in future.
This commit removes the `-D warnings` flag being passed through the makefiles to
all crates to instead be a crate attribute. We want these attributes always
applied for all our standard builds, and this is more amenable to Cargo-based
builds as well.
Note that all `deny(warnings)` attributes are gated with a `cfg(stage0)`
attribute currently to match the same semantics we have today
We currently pass generic as the CPU to LLVM. This results in worse
than required code generation. On little endian, which is only POWER8,
we avoid many POWER4 and newer instructions.
Pass ppc64 and ppc64le instead.
This commit stabilizes and deprecates the FCP (final comment period) APIs for
the upcoming 1.7 beta release. The specific APIs which changed were:
Stabilized
* `Path::strip_prefix` (renamed from `relative_from`)
* `path::StripPrefixError` (new error type returned from `strip_prefix`)
* `Ipv4Addr::is_loopback`
* `Ipv4Addr::is_private`
* `Ipv4Addr::is_link_local`
* `Ipv4Addr::is_multicast`
* `Ipv4Addr::is_broadcast`
* `Ipv4Addr::is_documentation`
* `Ipv6Addr::is_unspecified`
* `Ipv6Addr::is_loopback`
* `Ipv6Addr::is_unique_local`
* `Ipv6Addr::is_multicast`
* `Vec::as_slice`
* `Vec::as_mut_slice`
* `String::as_str`
* `String::as_mut_str`
* `<[T]>::clone_from_slice` - the `usize` return value is removed
* `<[T]>::sort_by_key`
* `i32::checked_rem` (and other signed types)
* `i32::checked_neg` (and other signed types)
* `i32::checked_shl` (and other signed types)
* `i32::checked_shr` (and other signed types)
* `i32::saturating_mul` (and other signed types)
* `i32::overflowing_add` (and other signed types)
* `i32::overflowing_sub` (and other signed types)
* `i32::overflowing_mul` (and other signed types)
* `i32::overflowing_div` (and other signed types)
* `i32::overflowing_rem` (and other signed types)
* `i32::overflowing_neg` (and other signed types)
* `i32::overflowing_shl` (and other signed types)
* `i32::overflowing_shr` (and other signed types)
* `u32::checked_rem` (and other unsigned types)
* `u32::checked_neg` (and other unsigned types)
* `u32::checked_shl` (and other unsigned types)
* `u32::saturating_mul` (and other unsigned types)
* `u32::overflowing_add` (and other unsigned types)
* `u32::overflowing_sub` (and other unsigned types)
* `u32::overflowing_mul` (and other unsigned types)
* `u32::overflowing_div` (and other unsigned types)
* `u32::overflowing_rem` (and other unsigned types)
* `u32::overflowing_neg` (and other unsigned types)
* `u32::overflowing_shl` (and other unsigned types)
* `u32::overflowing_shr` (and other unsigned types)
* `ffi::IntoStringError`
* `CString::into_string`
* `CString::into_bytes`
* `CString::into_bytes_with_nul`
* `From<CString> for Vec<u8>`
* `From<CString> for Vec<u8>`
* `IntoStringError::into_cstring`
* `IntoStringError::utf8_error`
* `Error for IntoStringError`
Deprecated
* `Path::relative_from` - renamed to `strip_prefix`
* `Path::prefix` - use `components().next()` instead
* `os::unix::fs` constants - moved to the `libc` crate
* `fmt::{radix, Radix, RadixFmt}` - not used enough to stabilize
* `IntoCow` - conflicts with `Into` and may come back later
* `i32::{BITS, BYTES}` (and other integers) - not pulling their weight
* `DebugTuple::formatter` - will be removed
* `sync::Semaphore` - not used enough and confused with system semaphores
Closes#23284
cc #27709 (still lots more methods though)
Closes#27712Closes#27722Closes#27728Closes#27735Closes#27729Closes#27755Closes#27782Closes#27798
Currently a compiler can be built with the `--disable-elf-tls` option for compatibility with OSX 10.6 which doesn't have ELF TLS. This is unfortunate, however, as a whole new compiler must be generated which can take some time. These commits add a new (feature gated) `cfg(target_thread_local)` annotation set by the compiler which indicates whether `#[thread_local]` is available for use. The compiler now interprets `MACOSX_DEPLOYMENT_TARGET` (a standard environment variable) to set this flag on OSX. With this we may want to start compiling our OSX nightlies with `MACOSX_DEPLOYMENT_TARGET` set to 10.6 which would allow the compiler out-of-the-box to generate 10.6-compatible binaries.
For now the compiler still by default targets OSX 10.7 by allowing ELF TLS by default (e.g. if `MACOSX_DEPLOYMENT_TARGET` isn't set).
Currently the standard library has some pretty complicated logic to detect
whether #[thread_local] should be used or whether it's supported. This is also
unfortunately not quite true for OSX where not all versions support
the #[thread_local] attribute (only 10.7+ does). Compiling code for OSX 10.6 is
typically requested via the MACOSX_DEPLOYMENT_TARGET environment variable (e.g.
the linker recognizes this), but the standard library unfortunately does not
respect this.
This commit updates the compiler to add a `target_thread_local` cfg annotation
if the platform being targeted supports the `#[thread_local]` attribute. This is
feature gated for now, and it is only true on non-aarch64 Linux and 10.7+ OSX
(e.g. what the module already does today). Logic has also been added to parse
the deployment target environment variable.
This PR is a rebase of the original PR by @eddyb https://github.com/rust-lang/rust/pull/21836 with some unrebasable parts manually reapplied, feature gate added + type equality restriction added as described below.
This implementation is partial because the type equality restriction is applied to all type ascription expressions and not only those in lvalue contexts. Thus, all difficulties with detection of these contexts and translation of coercions having effect in runtime are avoided.
So, you can't write things with coercions like `let slice = &[1, 2, 3]: &[u8];`. It obviously makes type ascription less useful than it should be, but it's still much more useful than not having type ascription at all.
In particular, things like `let v = something.iter().collect(): Vec<_>;` and `let u = t.into(): U;` work as expected and I'm pretty happy with these improvements alone.
Part of https://github.com/rust-lang/rust/issues/23416
This commit is the standard API stabilization commit for the 1.6 release cycle.
The list of issues and APIs below have all been through their cycle-long FCP and
the libs team decisions are listed below
Stabilized APIs
* `Read::read_exact`
* `ErrorKind::UnexpectedEof` (renamed from `UnexpectedEOF`)
* libcore -- this was a bit of a nuanced stabilization, the crate itself is now
marked as `#[stable]` and the methods appearing via traits for primitives like
`char` and `str` are now also marked as stable. Note that the extension traits
themeselves are marked as unstable as they're imported via the prelude. The
`try!` macro was also moved from the standard library into libcore to have the
same interface. Otherwise the functions all have copied stability from the
standard library now.
* The `#![no_std]` attribute
* `fs::DirBuilder`
* `fs::DirBuilder::new`
* `fs::DirBuilder::recursive`
* `fs::DirBuilder::create`
* `os::unix::fs::DirBuilderExt`
* `os::unix::fs::DirBuilderExt::mode`
* `vec::Drain`
* `vec::Vec::drain`
* `string::Drain`
* `string::String::drain`
* `vec_deque::Drain`
* `vec_deque::VecDeque::drain`
* `collections::hash_map::Drain`
* `collections::hash_map::HashMap::drain`
* `collections::hash_set::Drain`
* `collections::hash_set::HashSet::drain`
* `collections::binary_heap::Drain`
* `collections::binary_heap::BinaryHeap::drain`
* `Vec::extend_from_slice` (renamed from `push_all`)
* `Mutex::get_mut`
* `Mutex::into_inner`
* `RwLock::get_mut`
* `RwLock::into_inner`
* `Iterator::min_by_key` (renamed from `min_by`)
* `Iterator::max_by_key` (renamed from `max_by`)
Deprecated APIs
* `ErrorKind::UnexpectedEOF` (renamed to `UnexpectedEof`)
* `OsString::from_bytes`
* `OsStr::to_cstring`
* `OsStr::to_bytes`
* `fs::walk_dir` and `fs::WalkDir`
* `path::Components::peek`
* `slice::bytes::MutableByteVector`
* `slice::bytes::copy_memory`
* `Vec::push_all` (renamed to `extend_from_slice`)
* `Duration::span`
* `IpAddr`
* `SocketAddr::ip`
* `Read::tee`
* `io::Tee`
* `Write::broadcast`
* `io::Broadcast`
* `Iterator::min_by` (renamed to `min_by_key`)
* `Iterator::max_by` (renamed to `max_by_key`)
* `net::lookup_addr`
New APIs (still unstable)
* `<[T]>::sort_by_key` (added to mirror `min_by_key`)
Closes#27585Closes#27704Closes#27707Closes#27710Closes#27711Closes#27727Closes#27740Closes#27744Closes#27799Closes#27801
cc #27801 (doesn't close as `Chars` is still unstable)
Closes#28968
Note: for now, this change only affects `-windows-gnu` builds.
So why was this `libgcc` dylib dependency needed in the first place?
The stack unwinder needs to know about locations of unwind tables of all the modules loaded in the current process. The easiest portable way of achieving this is to have each module register itself with the unwinder when loaded into the process. All modules compiled by GCC do this by calling the __register_frame_info() in their startup code (that's `crtbegin.o` and `crtend.o`, which are automatically linked into any gcc output).
Another important piece is that there should be only one copy of the unwinder (and thus unwind tables registry) in the process. This pretty much means that the unwinder must be in a shared library (unless everything is statically linked).
Now, Rust compiler tries very hard to make sure that any given Rust crate appears in the final output just once. So if we link the unwinder statically to one of Rust's crates, everything should be fine.
Unfortunately, GCC startup objects are built under assumption that `libgcc` is the one true place for the unwind info registry, so I couldn't find any better way than to replace them. So out go `crtbegin`/`crtend`, in come `rsbegin`/`rsend`!
A side benefit of this change is that rustc is now more in control of the command line that goes to the linker, so we could stop using `gcc` as the linker driver and just invoke `ld` directly.
This commit stabilizes and deprecates library APIs whose FCP has closed in the
last cycle, specifically:
Stabilized APIs:
* `fs::canonicalize`
* `Path::{metadata, symlink_metadata, canonicalize, read_link, read_dir, exists,
is_file, is_dir}` - all moved to inherent methods from the `PathExt` trait.
* `Formatter::fill`
* `Formatter::width`
* `Formatter::precision`
* `Formatter::sign_plus`
* `Formatter::sign_minus`
* `Formatter::alternate`
* `Formatter::sign_aware_zero_pad`
* `string::ParseError`
* `Utf8Error::valid_up_to`
* `Iterator::{cmp, partial_cmp, eq, ne, lt, le, gt, ge}`
* `<[T]>::split_{first,last}{,_mut}`
* `Condvar::wait_timeout` - note that `wait_timeout_ms` is not yet deprecated
but will be once 1.5 is released.
* `str::{R,}MatchIndices`
* `str::{r,}match_indices`
* `char::from_u32_unchecked`
* `VecDeque::insert`
* `VecDeque::shrink_to_fit`
* `VecDeque::as_slices`
* `VecDeque::as_mut_slices`
* `VecDeque::swap_remove_front` - (renamed from `swap_front_remove`)
* `VecDeque::swap_remove_back` - (renamed from `swap_back_remove`)
* `Vec::resize`
* `str::slice_mut_unchecked`
* `FileTypeExt`
* `FileTypeExt::{is_block_device, is_char_device, is_fifo, is_socket}`
* `BinaryHeap::from` - `from_vec` deprecated in favor of this
* `BinaryHeap::into_vec` - plus a `Into` impl
* `BinaryHeap::into_sorted_vec`
Deprecated APIs
* `slice::ref_slice`
* `slice::mut_ref_slice`
* `iter::{range_inclusive, RangeInclusive}`
* `std::dynamic_lib`
Closes#27706Closes#27725
cc #27726 (align not stabilized yet)
Closes#27734Closes#27737Closes#27742Closes#27743Closes#27772Closes#27774Closes#27777Closes#27781
cc #27788 (a few remaining methods though)
Closes#27790Closes#27793Closes#27796Closes#27810
cc #28147 (not all parts stabilized)
Since it isn't possible to disable linkage of just GCC startup objects, we now need logic for finding libc installation directory and copying the required startup files (e.g. crt2.o) to rustlib directory.
Bonus change: use the `-nodefaultlibs` flag on Windows, thus paving the way to direct linker invocation.
This commit updates the compiler to not attempt to use jemalloc for platforms
where jemalloc is never enabled. Currently the compiler attempts to link in
jemalloc based on whether `--disable-jemalloc` was specified at build time for
the compiler itself, but this is only the right decision for the host target,
not for other targets.
This still leaves a hole open where a set of target libraries are downloaded
which were built with `--disable-jemalloc` and the compiler is unaware of that,
but this is a pretty rare case so it can always be fixed later.
For most parts, rumprun currently looks like NetBSD, as they share the same
libc and drivers. However, being a unikernel, rumprun does not support
process management, signals or virtual memory, so related functions
might fail at runtime. Stack guards are disabled exactly for this reason.
Code for rumprun is always cross-compiled, it uses always static
linking and needs a custom linker.
This commit updates the compiler to not attempt to use jemalloc for platforms
where jemalloc is never enabled. Currently the compiler attempts to link in
jemalloc based on whether `--disable-jemalloc` was specified at build time for
the compiler itself, but this is only the right decision for the host target,
not for other targets.
This still leaves a hole open where a set of target libraries are downloaded
which were built with `--disable-jemalloc` and the compiler is unaware of that,
but this is a pretty rare case so it can always be fixed later.
This adds a new target property, `target_vendor` which can be used as a
matcher for conditional compilation. The vendor is part of the autoconf
target triple: <arch><sub>-<vendor>-<os>-<env>
The default value for `target_vendor` is "unknown".
Matching against the `target_vendor` with `#[cfg]` is currently feature
gated as `cfg_target_vendor`.
This patch basically adds a target option for omitting the `-nodefaultlibs` flag when invoking the linker. I am not sure if this is the correct or only way to approach this problem, so any feedback is welcome.
Motivation: I'm currently working on a Rust target specification for the [rumprun](/rumpkernel/rumprun) unikernel. rumprun is based on rump kernels and uses NetBSDs libc and drivers to provide a POSIXy environment. It provides its own linker wrapper that generates binaries which can be "baked" into a unikernel after configuration. Using `-nodefaultlibs` on the rumprun linker will prevent it from selecting the search paths for the rumprun libraries. My current target implementation for rumprun is here: gandro/rust@295744b2ee
Currently, only a target that `is_like_windows` will omit the `-nodefaultlibs` flag, but since rumprun is not like Windows otherwise, I think a separate flag makes more sense. This might be a breaking change for target specifications that have the `is_like_windows` option set to true. Such targets need to set `no_default_libraries` to false in order to restore the old behavior.
If set to false, `-nodefaultlibs` is not passed to the linker. This
was the default behavior on Windows, but it should be configurable
per target.
This is a [breaking-change] for target specifications that have
the `is_like_windows` option set to true. Such targets need to
set `no_default_libraries` to false in order to restore the old
behavior.
This patch transforms functions of the form
```
fn f<Generic: AsRef<Concrete>>(arg: Generic) {
let arg: &Concrete = arg.as_ref();
// Code using arg
}
```
to the next form:
```
#[inline]
fn f<Generic: AsRef<Concrete>>(arg: Generic) {
fn f_inner(arg: &Concrete) {
// Code using arg
}
f_inner(arg.as_ref());
}
```
Therefore, most of the code is concrete and not duplicated during monomorphisation (unless inlined)
and only the tiny bit of conversion code is duplicated. This method was mentioned by @aturon in the
Conversion Traits RFC (https://github.com/rust-lang/rfcs/blame/master/text/0529-conversion-traits.md#L249) and similar techniques are not uncommon in C++ template libraries.
This patch goes to the extremes and applies the transformation even to smaller functions<sup>1</sup>
for purity of the experiment. *Some of them can be rolled back* if considered too ridiculous.
<sup>1</sup> However who knows how small are these functions are after inlining and everything.
The functions in question are mostly `fs`/`os` functions and not used especially often with variety
of argument types, so the code size reduction is rather small (but consistent). Here are the sizes
of stage2 artifacts before and after the patch:
https://gist.github.com/petrochenkov/e76a6b280f382da13c5dhttps://gist.github.com/petrochenkov/6cc28727d5256dbdfed0
Note:
All the `inner` functions are concrete and unavailable for cross-crate inlining, some of them may
need `#[inline]` annotations in the future.
r? @aturon
- All the libstd tests are passing in the optimized build against
a Zenfone2 and the x86 Android emulator.
I haven't tested the other libraries though.
This commit is an implementation of [RFC 1183][rfc] which allows swapping out
the default allocator on nightly Rust. No new stable surface area should be
added as a part of this commit.
[rfc]: https://github.com/rust-lang/rfcs/pull/1183
Two new attributes have been added to the compiler:
* `#![needs_allocator]` - this is used by liballoc (and likely only liballoc) to
indicate that it requires an allocator crate to be in scope.
* `#![allocator]` - this is a indicator that the crate is an allocator which can
satisfy the `needs_allocator` attribute above.
The ABI of the allocator crate is defined to be a set of symbols that implement
the standard Rust allocation/deallocation functions. The symbols are not
currently checked for exhaustiveness or typechecked. There are also a number of
restrictions on these crates:
* An allocator crate cannot transitively depend on a crate that is flagged as
needing an allocator (e.g. allocator crates can't depend on liballoc).
* There can only be one explicitly linked allocator in a final image.
* If no allocator is explicitly requested one will be injected on behalf of the
compiler. Binaries and Rust dylibs will use jemalloc by default where
available and staticlibs/other dylibs will use the system allocator by
default.
Two allocators are provided by the distribution by default, `alloc_system` and
`alloc_jemalloc` which operate as advertised.
Closes#27389
This commit removes all unstable and deprecated functions in the standard
library. A release was recently cut (1.3) which makes this a good time for some
spring cleaning of the deprecated functions.
This commit leverages the runtime support for DWARF exception info added
in #27210 to enable unwinding by default on 64-bit MSVC. This also additionally
adds a few minor fixes here and there in the test harness and such to get
`make check` entirely passing on 64-bit MSVC:
* The invocation of `maketest.py` now works with spaces/quotes in CC
* debuginfo tests are disabled on MSVC
* A link error for librustc was hacked around (see #27438)
This commit leverages the runtime support for DWARF exception info added
in #27210 to enable unwinding by default on 64-bit MSVC. This also additionally
adds a few minor fixes here and there in the test harness and such to get
`make check` entirely passing on 64-bit MSVC:
* The invocation of `maketest.py` now works with spaces/quotes in CC
* debuginfo tests are disabled on MSVC
* A link error for librustc was hacked around (see #27438)
This means that we no longer need to ship the `llvm-ar.exe` binary in the MSVC
distribution, and after a snapshot we can remove a good bit of logic from the
makefiles!
This means that we no longer need to ship the `llvm-ar.exe` binary in the MSVC
distribution, and after a snapshot we can remove a good bit of logic from the
makefiles!
This commit removes all morestack support from the compiler which entails:
* Segmented stacks are no longer emitted in codegen.
* We no longer build or distribute libmorestack.a
* The `stack_exhausted` lang item is no longer required
The only current use of the segmented stack support in LLVM is to detect stack
overflow. This is no longer really required, however, because we already have
guard pages for all threads and registered signal handlers watching for a
segfault on those pages (to print out a stack overflow message). Additionally,
major platforms (aka Windows) already don't use morestack.
This means that Rust is by default less likely to catch stack overflows because
if a function takes up more than one page of stack space it won't hit the guard
page. This is what the purpose of morestack was (to catch this case), but it's
better served with stack probes which have more cross platform support and no
runtime support necessary. Until LLVM supports this for all platform it looks
like morestack isn't really buying us much.
cc #16012 (still need stack probes)
Closes#26458 (a drive-by fix to help diagnostics on stack overflow)
Turns out for OSX our data layout was subtly wrong and the LLVM update must have
exposed this. Instead of fixing this I've removed all data layouts from the
compiler to just use the defaults that LLVM provides for all targets. All data
layouts (and a number of dead modules) are removed from the compiler here.
Custom target specifications can still provide a custom data layout, but it is
now an optional key as the default will be used if one isn't specified.
Updates our LLVM bindings to be able to write out multiple kinds of archives.
This commit also enables using LLVM instead of the system ar on all current
targets.
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
The common pattern `iter::repeat(elt).take(n).collect::<Vec<_>>()` is
exactly equivalent to `vec![elt; n]`, do this replacement in the whole
tree.
(Actually, vec![] is smart enough to only call clone n - 1 times, while
the former solution would call clone n times, and this fact is
virtually irrelevant in practice.)
This commit shards the broad `core` feature of the libcore library into finer
grained features. This split groups together similar APIs and enables tracking
each API separately, giving a better sense of where each feature is within the
stabilization process.
A few minor APIs were deprecated along the way:
* Iterator::reverse_in_place
* marker::NoCopy
Bug fixes for configure on FreeBSD:
- Don't ban using gcc; newer versions can be installed and other checks should enforce its suitability.
- Don't force Rust to link itself with /usr/local/lib{,gcc4[46]}, which causes builds to fail if Rust is already installed. I've not been able to find an use case where this is actually necessary.
This flag indicates that when files are being replaced or added to archives (the
`r` flag) that the new file should not be inserted if it is not newer than the
file that already exists in the archive. The compiler never actually has a use
case of *not* wanting to insert a file because it already exists, and this
causes rlibs to not be updated in some cases when the compiler was re-run too
quickly.
Closes#18913
The first one in particular results in Rust not being able to build itself
if it is installed. The latter two shouldn't be necessary, and should only
be included if they are actually going to be used.
Special thanks to @retep998 for the [excellent writeup](https://github.com/rust-lang/rfcs/issues/1061) of tasks to be done and @ricky26 for initially blazing the trail here!
# MSVC Support
This goal of this series of commits is to add MSVC support to the Rust compiler
and build system, allowing it more easily interoperate with Visual Studio
installations and native libraries compiled outside of MinGW.
The tl;dr; of this change is that there is a new target of the compiler,
`x86_64-pc-windows-msvc`, which will not interact with the MinGW toolchain at
all and will instead use `link.exe` to assemble output artifacts.
## Why try to use MSVC?
With today's Rust distribution, when you install a compiler on Windows you also
install `gcc.exe` and a number of supporting libraries by default (this can be
opted out of). This allows installations to remain independent of MinGW
installations, but it still generally requires native code to be linked with
MinGW instead of MSVC. Some more background can also be found in #1768 about the
incompatibilities between MinGW and MSVC.
Overall the current installation strategy is quite nice so long as you don't
interact with native code, but once you do the usage of a MinGW-based `gcc.exe`
starts to get quite painful.
Relying on a nonstandard Windows toolchain has also been a long-standing "code
smell" of Rust and has been slated for remedy for quite some time now. Using a
standard toolchain is a great motivational factor for improving the
interoperability of Rust code with the native system.
## What does it mean to use MSVC?
"Using MSVC" can be a bit of a nebulous concept, but this PR defines it as:
* The build system for Rust will build as much code as possible with the MSVC
compiler, `cl.exe`.
* The build system will use native MSVC tools for managing archives.
* The compiler will link all output with `link.exe` instead of `gcc.exe`.
None of these are currently implemented today, but all are required for the
compiler to fluently interoperate with MSVC.
## How does this all work?
At the highest level, this PR adds a new target triple to the Rust compiler:
x86_64-pc-windows-msvc
All logic for using MSVC or not is scoped within this triple and code can
conditionally build for MSVC or MinGW via:
#[cfg(target_env = "msvc")]
It is expected that auto builders will be set up for MSVC-based compiles in
addition to the existing MinGW-based compiles, and we will likely soon start
shipping MSVC nightlies where `x86_64-pc-windows-msvc` is the host target triple
of the compiler.
# Summary of changes
Here I'll explain at a high level what many of the changes made were targeted
at, but many more details can be found in the commits themselves. Many thanks to
@retep998 for the excellent writeup in rust-lang/rfcs#1061 and @rick26 for a lot
of the initial proof-of-concept work!
## Build system changes
As is probably expected, a large chunk of this PR is changes to Rust's build
system to build with MSVC. At a high level **it is an explicit non goal** to
enable building outside of a MinGW shell, instead all Makefile infrastructure we
have today is retrofitted with support to use MSVC instead of the standard MSVC
toolchain. Some of the high-level changes are:
* The configure script now detects when MSVC is being targeted and adds a number
of additional requirements about the build environment:
* The `--msvc-root` option must be specified or `cl.exe` must be in PATH to
discover where MSVC is installed. The compiler in use is also required to
target x86_64.
* Once the MSVC root is known, the INCLUDE/LIB environment variables are
scraped so they can be reexported by the build system.
* CMake is required to build LLVM with MSVC (and LLVM is also configured with
CMake instead of the normal configure script).
* jemalloc is currently unconditionally disabled for MSVC targets as jemalloc
isn't a hard requirement and I don't know how to build it with MSVC.
* Invocations of a C and/or C++ compiler are now abstracted behind macros to
appropriately call the underlying compiler with the correct format of
arguments, for example there is now a macro for "assemble an archive from
objects" instead of hard-coded invocations of `$(AR) crus liboutput.a ...`
* The output filenames for standard libraries such as morestack/compiler-rt are
now "more correct" on windows as they are shipped as `foo.lib` instead of
`libfoo.a`.
* Rust targets can now depend on native tools provided by LLVM, and as you'll
see in the commits the entire MSVC target depends on `llvm-ar.exe`.
* Support for custom arbitrary makefile dependencies of Rust targets has been
added. The MSVC target for `rustc_llvm` currently requires a custom `.DEF`
file to be passed to the linker to get further linkages to complete.
## Compiler changes
The modifications made to the compiler have so far largely been minor tweaks
here and there, mostly just adding a layer of abstraction over whether MSVC or a
GNU-like linker is being used. At a high-level these changes are:
* The section name for metadata storage in dynamic libraries is called `.rustc`
for MSVC-based platorms as section names cannot contain more than 8
characters.
* The implementation of `rustc_back::Archive` was refactored, but the
functionality has remained the same.
* Targets can now specify the default `ar` utility to use, and for MSVC this
defaults to `llvm-ar.exe`
* The building of the linker command in `rustc_trans:🔙:link` has been
abstracted behind a trait for the same code path to be used between GNU and
MSVC linkers.
## Standard library changes
Only a few small changes were required to the stadnard library itself, and only
for minor differences between the C runtime of msvcrt.dll and MinGW's libc.a
* Some function names for floating point functions have leading underscores, and
some are not present at all.
* Linkage to the `advapi32` library for crypto-related functions is now
explicit.
* Some small bits of C code here and there were fixed for compatibility with
MSVC's cl.exe compiler.
# Future Work
This commit is not yet a 100% complete port to using MSVC as there are still
some key components missing as well as some unimplemented optimizations. This PR
is already getting large enough that I wanted to draw the line here, but here's
a list of what is not implemented in this PR, on purpose:
## Unwinding
The revision of our LLVM submodule [does not seem to implement][llvm] does not
support lowering SEH exception handling on the Windows MSVC targets, so
unwinding support is not currently implemented for the standard library (it's
lowered to an abort).
[llvm]: https://github.com/rust-lang/llvm/blob/rust-llvm-2015-02-19/lib/CodeGen/Passes.cpp#L454-L461
It looks like, however, that upstream LLVM has quite a bit more support for SEH
unwinding and landing pads than the current revision we have, so adding support
will likely just involve updating LLVM and then adding some shims of our own
here and there.
## dllimport and dllexport
An interesting part of Windows which MSVC forces our hand on (and apparently
MinGW didn't) is the usage of `dllimport` and `dllexport` attributes in LLVM IR
as well as native dependencies (in C these correspond to
`__declspec(dllimport)`).
Whenever a dynamic library is built by MSVC it must have its public interface
specified by functions tagged with `dllexport` or otherwise they're not
available to be linked against. This poses a few problems for the compiler, some
of which are somewhat fundamental, but this commit alters the compiler to attach
the `dllexport` attribute to all LLVM functions that are reachable (e.g. they're
already tagged with external linkage). This is suboptimal for a few reasons:
* If an object file will never be included in a dynamic library, there's no need
to attach the dllexport attribute. Most object files in Rust are not destined
to become part of a dll as binaries are statically linked by default.
* If the compiler is emitting both an rlib and a dylib, the same source object
file is currently used but with MSVC this may be less feasible. The compiler
may be able to get around this, but it may involve some invasive changes to
deal with this.
The flipside of this situation is that whenever you link to a dll and you import
a function from it, the import should be tagged with `dllimport`. At this time,
however, the compiler does not emit `dllimport` for any declarations other than
constants (where it is required), which is again suboptimal for even more
reasons!
* Calling a function imported from another dll without using `dllimport` causes
the linker/compiler to have extra overhead (one `jmp` instruction on x86) when
calling the function.
* The same object file may be used in different circumstances, so a function may
be imported from a dll if the object is linked into a dll, but it may be
just linked against if linked into an rlib.
* The compiler has no knowledge about whether native functions should be tagged
dllimport or not.
For now the compiler takes the perf hit (I do not have any numbers to this
effect) by marking very little as `dllimport` and praying the linker will take
care of everything. Fixing this problem will likely require adding a few
attributes to Rust itself (feature gated at the start) and then strongly
recommending static linkage on Windows! This may also involve shipping a
statically linked compiler on Windows instead of a dynamically linked compiler,
but these sorts of changes are pretty invasive and aren't part of this PR.
## CI integration
Thankfully we don't need to set up a new snapshot bot for the changes made here as our snapshots are freestanding already, we should be able to use the same snapshot to bootstrap both MinGW and MSVC compilers (once a new snapshot is made from these changes).
I plan on setting up a new suite of auto bots which are testing MSVC configurations for now as well, for now they'll just be bootstrapping and not running tests, but once unwinding is implemented they'll start running all tests as well and we'll eventually start gating on them as well.
---
I'd love as many eyes on this as we've got as this was one of my first interactions with MSVC and Visual Studio, so there may be glaring holes that I'm missing here and there!
cc @retep998, @ricky26, @vadimcn, @klutzy
r? @brson
This commit was initially written to target either `ar` or `lib.exe` for MSVC,
but it ended up not needing `lib.exe` support after all. I would personally like
to refactor this to one day not invoke processes at all (but rather use the
`llvm-ar.cpp` file in LLVM as alibrary) so I chose to preserve this refactoring
to allow it to be easily done in the future.
This allows compiling entire crates from memory or preprocessing source files before they are tokenized.
Minor API refactoring included, which is a [breaking-change] for libsyntax users:
* `ParseSess::{next_node_id, reserve_node_ids}` moved to rustc's `Session`
* `new_parse_sess` -> `ParseSess::new`
* `new_parse_sess_special_handler` -> `ParseSess::with_span_handler`
* `mk_span_handler` -> `SpanHandler::new`
* `default_handler` -> `Handler::new`
* `mk_handler` -> `Handler::with_emitter`
* `string_to_filemap(sess source, path)` -> `sess.codemap().new_filemap(path, source)`
I've been working with some archives generated by MSVC's `lib.exe` tool lately,
and it looks like the embedded name of the members in those archives sometimes
have slahes in the name (e.g. `foo/bar/baz.obj`). Currently the compiler chokes
on these paths as it assumes that each file in the archive is only the filename
(which is what unix does).
This commit interprets the name of each file in all archives as a path and then
only uses the `file_name` portion of the path to extract the file to a separate
location and then reassemble it back into a new archive later. Note that
duplicate filenames are already handled, so this won't introduce any conflicts.
This is OK to do given:
- PIE is supported on Android starting with API 16.
- The bots are running API 18.
- API < 16 now has a 12.5% market share[0] as of 2015-04-29.
Closes#17437.
[0] https://developer.android.com/about/dashboards/index.html
r? @alexcrichton
This commit is an implementation of [RFC 1044][rfc] which adds additional
surface area to the `std::fs` module. All new APIs are `#[unstable]` behind
assorted feature names for each one.
[rfc]: https://github.com/rust-lang/rfcs/pull/1044
The new APIs added are:
* `fs::canonicalize` - bindings to `realpath` on unix and
`GetFinalPathNameByHandle` on windows.
* `fs::symlink_metadata` - similar to `lstat` on unix
* `fs::FileType` and accessor methods as `is_{file,dir,symlink}`
* `fs::Metadata::file_type` - accessor for the raw file type
* `fs::DirEntry::metadata` - acquisition of metadata which is free on Windows
but requires a syscall on unix.
* `fs::DirEntry::file_type` - access the file type which may not require a
syscall on most platforms.
* `fs::DirEntry::file_name` - access just the file name without leading
components.
* `fs::PathExt::symlink_metadata` - convenience method for the top-level
function.
* `fs::PathExt::canonicalize` - convenience method for the top-level
function.
* `fs::PathExt::read_link` - convenience method for the top-level
function.
* `fs::PathExt::read_dir` - convenience method for the top-level
function.
* `std::os::raw` - type definitions for raw OS/C types available on all
platforms.
* `std::os::$platform` - new modules have been added for all currently supported
platforms (e.g. those more specific than just `unix`).
* `std::os::$platform::raw` - platform-specific type definitions. These modules
are populated with the bare essentials necessary for lowing I/O types into
their raw representations, and currently largely consist of the `stat`
definition for unix platforms.
This commit also deprecates `Metadata::{modified, accessed}` in favor of
inspecting the raw representations via the lowering methods of `Metadata`.
Closes https://github.com/rust-lang/rust/issues/24796
This is OK to do given:
- PIE is supported on Android starting with API 16.
- The bots are running API 18.
- API < 16 now has a 12.5% market share[0] as of 2015-04-29.
Unfortunately, this breaks backtrace support. See #17520.
Closes#17437.
[0] https://developer.android.com/about/dashboards/index.html
This commit is an implementation of [RFC 1044][rfc] which adds additional
surface area to the `std::fs` module. All new APIs are `#[unstable]` behind
assorted feature names for each one.
[rfc]: https://github.com/rust-lang/rfcs/pull/1044
The new APIs added are:
* `fs::canonicalize` - bindings to `realpath` on unix and
`GetFinalPathNameByHandle` on windows.
* `fs::symlink_metadata` - similar to `lstat` on unix
* `fs::FileType` and accessor methods as `is_{file,dir,symlink}`
* `fs::Metadata::file_type` - accessor for the raw file type
* `fs::DirEntry::metadata` - acquisition of metadata which is free on Windows
but requires a syscall on unix.
* `fs::DirEntry::file_type` - access the file type which may not require a
syscall on most platforms.
* `fs::DirEntry::file_name` - access just the file name without leading
components.
* `fs::PathExt::symlink_metadata` - convenience method for the top-level
function.
* `fs::PathExt::canonicalize` - convenience method for the top-level
function.
* `fs::PathExt::read_link` - convenience method for the top-level
function.
* `fs::PathExt::read_dir` - convenience method for the top-level
function.
* `std::os::raw` - type definitions for raw OS/C types available on all
platforms.
* `std::os::$platform` - new modules have been added for all currently supported
platforms (e.g. those more specific than just `unix`).
* `std::os::$platform::raw` - platform-specific type definitions. These modules
are populated with the bare essentials necessary for lowing I/O types into
their raw representations, and currently largely consist of the `stat`
definition for unix platforms.
This commit also deprecates `Metadata::{modified, accessed}` in favor of
inspecting the raw representations via the lowering methods of `Metadata`.
This commit adds support for x86_64-unknown-linux-musl as a target of the
compiler. There's some comments in the commit about some of the more flavorful
flags passed to the linker as it's not quite as trivial as the normal specs.
This adds a new `#[cfg]` matcher against the `target_env` property of the
destination target triple. For example all windows triples today end with `-gnu`
but we will also hopefully support non-`gnu` targets for Windows, at which point
we'll need to differentiate between the two. This new `target_env` matches is
provided and filled in with the target's environment name.
Currently the only non-empty value of this name is `gnu`, but `musl` will be
shortly added for the linux triples.
Changes the style guidelines regarding unit tests to recommend using a
sub-module named "tests" instead of "test" for unit tests as "test"
might clash with imports of libtest.
This commit removes all the old casting/generic traits from `std::num` that are
no longer in use by the standard library. This additionally removes the old
`strconv` module which has not seen much use in quite a long time. All generic
functionality has been supplanted with traits in the `num` crate and the
`strconv` module is supplanted with the [rust-strconv crate][rust-strconv].
[rust-strconv]: https://github.com/lifthrasiir/rust-strconv
This is a breaking change due to the removal of these deprecated crates, and the
alternative crates are listed above.
[breaking-change]
When linking an archive statically to an rlib, the compiler will extract all
contents of the archive and add them all to the rlib being generated. The
current method of extraction is to run `ar x`, dumping all files into a
temporary directory. Object archives, however, are allowed to have multiple
entries with the same file name, so there is no method for them to extract their
contents into a directory in a lossless fashion.
This commit adds iterator support to the `ArchiveRO` structure which hooks into
LLVM's support for reading object archives. This iterator is then used to
inspect each object in turn and extract it to a unique location for later
assembly.
