Makes the lint a bit more accurate, and improves the quality of the diagnostic
messages by explicitly returning an error message.
The new lint is also a little more aggressive: specifically, it now
rejects tuples, and it recurses into function pointers.
This commit moves the IR files in the distribution, rust_try.ll,
rust_try_msvc_64.ll, and rust_try_msvc_32.ll into the compiler from the main
distribution. There's a few reasons for this change:
* LLVM changes its IR syntax from time to time, so it's very difficult to
have these files build across many LLVM versions simultaneously. We'll likely
want to retain this ability for quite some time into the future.
* The implementation of these files is closely tied to the compiler and runtime
itself, so it makes sense to fold it into a location which can do more
platform-specific checks for various implementation details (such as MSVC 32
vs 64-bit).
* This removes LLVM as a build-time dependency of the standard library. This may
end up becoming very useful if we move towards building the standard library
with Cargo.
In the immediate future, however, this commit should restore compatibility with
LLVM 3.5 and 3.6.
Turns out for OSX our data layout was subtly wrong and the LLVM update must have
exposed this. Instead of fixing this I've removed all data layouts from the
compiler to just use the defaults that LLVM provides for all targets. All data
layouts (and a number of dead modules) are removed from the compiler here.
Custom target specifications can still provide a custom data layout, but it is
now an optional key as the default will be used if one isn't specified.
Updates our LLVM bindings to be able to write out multiple kinds of archives.
This commit also enables using LLVM instead of the system ar on all current
targets.
The C API of this function changed so it no longer takes a personality function.
A shim was introduced to call the right LLVM function (depending on which
version we're compiled against) to set the personality function on the outer
function.
The compiler only ever sets one personality function for all generated
functions, so this should be equivalent.
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
This commit shards the all-encompassing `core`, `std_misc`, `collections`, and `alloc` features into finer-grained components that are much more easily opted into and tracked. This reflects the effort to push forward current unstable APIs to either stabilization or removal. Keeping track of unstable features on a much more fine-grained basis will enable the library subteam to quickly analyze a feature and help prioritize internally about what APIs should be stabilized.
A few assorted APIs were deprecated along the way, but otherwise this change is just changing the feature name associated with each API. Soon we will have a dashboard for keeping track of all the unstable APIs in the standard library, and I'll also start making issues for each unstable API after performing a first-pass for stabilization.
This commit updates the LLVM submodule in use to the current HEAD of the LLVM
repository. This is primarily being done to start picking up unwinding support
for MSVC, which is currently unimplemented in the revision of LLVM we are using.
Along the way a few changes had to be made:
* As usual, lots of C++ debuginfo bindings in LLVM changed, so there were some
significant changes to our RustWrapper.cpp
* As usual, some pass management changed in LLVM, so clang was re-scrutinized to
ensure that we're doing the same thing as clang.
* Some optimization options are now passed directly into the
`PassManagerBuilder` instead of through CLI switches to LLVM.
* The `NoFramePointerElim` option was removed from LLVM, favoring instead the
`no-frame-pointer-elim` function attribute instead.
Additionally, LLVM has picked up some new optimizations which required fixing an
existing soundness hole in the IR we generate. It appears that the current LLVM
we use does not expose this hole. When an enum is moved, the previous slot in
memory is overwritten with a bit pattern corresponding to "dropped". When the
drop glue for this slot is run, however, the switch on the discriminant can
often start executing the `unreachable` block of the switch due to the
discriminant now being outside the normal range. This was patched over locally
for now by having the `unreachable` block just change to a `ret void`.
* Removes `RustJITMemoryManager` from public API.
This was really sort of an implementation detail to begin with.
* `__morestack` is linked to C++ wrapper code and this pointer
is used when resolving the symbol for `ExecutionEngine` code.
* `__morestack_addr` is also resolved for `ExecutionEngine` code.
This function is sometimes referenced in LLVM-generated code,
but was not able to be resolved on Mac OS systems.
* Added Windows support to `ExecutionEngine` API.
* Added a test for basic `ExecutionEngine` functionality.
* Removes `RustJITMemoryManager` from public API.
This was really sort of an implementation detail to begin with.
* `__morestack` is linked to C++ wrapper code and this pointer
is used when resolving the symbol for `ExecutionEngine` code.
* `__morestack_addr` is also resolved for `ExecutionEngine` code.
This function is sometimes referenced in LLVM-generated code,
but was not able to be resolved on Mac OS systems.
* Added Windows support to `ExecutionEngine` API.
* Added a test for basic `ExecutionEngine` functionality.
This commit introduce a third parameter for compatible_ifn!, as new
intrinsics are being added in recent LLVM releases and there is no
need to hardcode a specific case.
Signed-off-by: Luca Bruno <lucab@debian.org>
Special thanks to @retep998 for the [excellent writeup](https://github.com/rust-lang/rfcs/issues/1061) of tasks to be done and @ricky26 for initially blazing the trail here!
# MSVC Support
This goal of this series of commits is to add MSVC support to the Rust compiler
and build system, allowing it more easily interoperate with Visual Studio
installations and native libraries compiled outside of MinGW.
The tl;dr; of this change is that there is a new target of the compiler,
`x86_64-pc-windows-msvc`, which will not interact with the MinGW toolchain at
all and will instead use `link.exe` to assemble output artifacts.
## Why try to use MSVC?
With today's Rust distribution, when you install a compiler on Windows you also
install `gcc.exe` and a number of supporting libraries by default (this can be
opted out of). This allows installations to remain independent of MinGW
installations, but it still generally requires native code to be linked with
MinGW instead of MSVC. Some more background can also be found in #1768 about the
incompatibilities between MinGW and MSVC.
Overall the current installation strategy is quite nice so long as you don't
interact with native code, but once you do the usage of a MinGW-based `gcc.exe`
starts to get quite painful.
Relying on a nonstandard Windows toolchain has also been a long-standing "code
smell" of Rust and has been slated for remedy for quite some time now. Using a
standard toolchain is a great motivational factor for improving the
interoperability of Rust code with the native system.
## What does it mean to use MSVC?
"Using MSVC" can be a bit of a nebulous concept, but this PR defines it as:
* The build system for Rust will build as much code as possible with the MSVC
compiler, `cl.exe`.
* The build system will use native MSVC tools for managing archives.
* The compiler will link all output with `link.exe` instead of `gcc.exe`.
None of these are currently implemented today, but all are required for the
compiler to fluently interoperate with MSVC.
## How does this all work?
At the highest level, this PR adds a new target triple to the Rust compiler:
x86_64-pc-windows-msvc
All logic for using MSVC or not is scoped within this triple and code can
conditionally build for MSVC or MinGW via:
#[cfg(target_env = "msvc")]
It is expected that auto builders will be set up for MSVC-based compiles in
addition to the existing MinGW-based compiles, and we will likely soon start
shipping MSVC nightlies where `x86_64-pc-windows-msvc` is the host target triple
of the compiler.
# Summary of changes
Here I'll explain at a high level what many of the changes made were targeted
at, but many more details can be found in the commits themselves. Many thanks to
@retep998 for the excellent writeup in rust-lang/rfcs#1061 and @rick26 for a lot
of the initial proof-of-concept work!
## Build system changes
As is probably expected, a large chunk of this PR is changes to Rust's build
system to build with MSVC. At a high level **it is an explicit non goal** to
enable building outside of a MinGW shell, instead all Makefile infrastructure we
have today is retrofitted with support to use MSVC instead of the standard MSVC
toolchain. Some of the high-level changes are:
* The configure script now detects when MSVC is being targeted and adds a number
of additional requirements about the build environment:
* The `--msvc-root` option must be specified or `cl.exe` must be in PATH to
discover where MSVC is installed. The compiler in use is also required to
target x86_64.
* Once the MSVC root is known, the INCLUDE/LIB environment variables are
scraped so they can be reexported by the build system.
* CMake is required to build LLVM with MSVC (and LLVM is also configured with
CMake instead of the normal configure script).
* jemalloc is currently unconditionally disabled for MSVC targets as jemalloc
isn't a hard requirement and I don't know how to build it with MSVC.
* Invocations of a C and/or C++ compiler are now abstracted behind macros to
appropriately call the underlying compiler with the correct format of
arguments, for example there is now a macro for "assemble an archive from
objects" instead of hard-coded invocations of `$(AR) crus liboutput.a ...`
* The output filenames for standard libraries such as morestack/compiler-rt are
now "more correct" on windows as they are shipped as `foo.lib` instead of
`libfoo.a`.
* Rust targets can now depend on native tools provided by LLVM, and as you'll
see in the commits the entire MSVC target depends on `llvm-ar.exe`.
* Support for custom arbitrary makefile dependencies of Rust targets has been
added. The MSVC target for `rustc_llvm` currently requires a custom `.DEF`
file to be passed to the linker to get further linkages to complete.
## Compiler changes
The modifications made to the compiler have so far largely been minor tweaks
here and there, mostly just adding a layer of abstraction over whether MSVC or a
GNU-like linker is being used. At a high-level these changes are:
* The section name for metadata storage in dynamic libraries is called `.rustc`
for MSVC-based platorms as section names cannot contain more than 8
characters.
* The implementation of `rustc_back::Archive` was refactored, but the
functionality has remained the same.
* Targets can now specify the default `ar` utility to use, and for MSVC this
defaults to `llvm-ar.exe`
* The building of the linker command in `rustc_trans:🔙:link` has been
abstracted behind a trait for the same code path to be used between GNU and
MSVC linkers.
## Standard library changes
Only a few small changes were required to the stadnard library itself, and only
for minor differences between the C runtime of msvcrt.dll and MinGW's libc.a
* Some function names for floating point functions have leading underscores, and
some are not present at all.
* Linkage to the `advapi32` library for crypto-related functions is now
explicit.
* Some small bits of C code here and there were fixed for compatibility with
MSVC's cl.exe compiler.
# Future Work
This commit is not yet a 100% complete port to using MSVC as there are still
some key components missing as well as some unimplemented optimizations. This PR
is already getting large enough that I wanted to draw the line here, but here's
a list of what is not implemented in this PR, on purpose:
## Unwinding
The revision of our LLVM submodule [does not seem to implement][llvm] does not
support lowering SEH exception handling on the Windows MSVC targets, so
unwinding support is not currently implemented for the standard library (it's
lowered to an abort).
[llvm]: https://github.com/rust-lang/llvm/blob/rust-llvm-2015-02-19/lib/CodeGen/Passes.cpp#L454-L461
It looks like, however, that upstream LLVM has quite a bit more support for SEH
unwinding and landing pads than the current revision we have, so adding support
will likely just involve updating LLVM and then adding some shims of our own
here and there.
## dllimport and dllexport
An interesting part of Windows which MSVC forces our hand on (and apparently
MinGW didn't) is the usage of `dllimport` and `dllexport` attributes in LLVM IR
as well as native dependencies (in C these correspond to
`__declspec(dllimport)`).
Whenever a dynamic library is built by MSVC it must have its public interface
specified by functions tagged with `dllexport` or otherwise they're not
available to be linked against. This poses a few problems for the compiler, some
of which are somewhat fundamental, but this commit alters the compiler to attach
the `dllexport` attribute to all LLVM functions that are reachable (e.g. they're
already tagged with external linkage). This is suboptimal for a few reasons:
* If an object file will never be included in a dynamic library, there's no need
to attach the dllexport attribute. Most object files in Rust are not destined
to become part of a dll as binaries are statically linked by default.
* If the compiler is emitting both an rlib and a dylib, the same source object
file is currently used but with MSVC this may be less feasible. The compiler
may be able to get around this, but it may involve some invasive changes to
deal with this.
The flipside of this situation is that whenever you link to a dll and you import
a function from it, the import should be tagged with `dllimport`. At this time,
however, the compiler does not emit `dllimport` for any declarations other than
constants (where it is required), which is again suboptimal for even more
reasons!
* Calling a function imported from another dll without using `dllimport` causes
the linker/compiler to have extra overhead (one `jmp` instruction on x86) when
calling the function.
* The same object file may be used in different circumstances, so a function may
be imported from a dll if the object is linked into a dll, but it may be
just linked against if linked into an rlib.
* The compiler has no knowledge about whether native functions should be tagged
dllimport or not.
For now the compiler takes the perf hit (I do not have any numbers to this
effect) by marking very little as `dllimport` and praying the linker will take
care of everything. Fixing this problem will likely require adding a few
attributes to Rust itself (feature gated at the start) and then strongly
recommending static linkage on Windows! This may also involve shipping a
statically linked compiler on Windows instead of a dynamically linked compiler,
but these sorts of changes are pretty invasive and aren't part of this PR.
## CI integration
Thankfully we don't need to set up a new snapshot bot for the changes made here as our snapshots are freestanding already, we should be able to use the same snapshot to bootstrap both MinGW and MSVC compilers (once a new snapshot is made from these changes).
I plan on setting up a new suite of auto bots which are testing MSVC configurations for now as well, for now they'll just be bootstrapping and not running tests, but once unwinding is implemented they'll start running all tests as well and we'll eventually start gating on them as well.
---
I'd love as many eyes on this as we've got as this was one of my first interactions with MSVC and Visual Studio, so there may be glaring holes that I'm missing here and there!
cc @retep998, @ricky26, @vadimcn, @klutzy
r? @brson
For imports of constants across DLLs to work on Windows it *requires* that the
import be marked with `dllimport` (unlike functions where the marker is
optional, but strongly recommended). This currently isn't working for importing
FFI constants across boundaries, however, so the one constant exported from
`rustc_llvm.dll` is now a function to be called instead.
These new intrinsics are comparable to `atomic_signal_fence` in C++,
ensuring the compiler will not reorder memory accesses across the
barrier, nor will it emit any machine instructions for it.
Closes#24118, implementing RFC 888.
When linking an archive statically to an rlib, the compiler will extract all
contents of the archive and add them all to the rlib being generated. The
current method of extraction is to run `ar x`, dumping all files into a
temporary directory. Object archives, however, are allowed to have multiple
entries with the same file name, so there is no method for them to extract their
contents into a directory in a lossless fashion.
This commit adds iterator support to the `ArchiveRO` structure which hooks into
LLVM's support for reading object archives. This iterator is then used to
inspect each object in turn and extract it to a unique location for later
assembly.
const_eval : add overflow-checking for {`+`, `-`, `*`, `/`, `<<`, `>>`}.
One tricky detail here: There is some duplication of labor between `rustc::middle::const_eval` and `rustc_trans::trans::consts`. It might be good to explore ways to try to factor out the common structure to the two passes (by abstracting over the particular value-representation used in the compile-time interpreter).
----
Update: Rebased atop #23841Fix#22531Fix#23030Fix#23221Fix#23235
Add option-returning variants to `const_to_int`/`const_to_uint` that
never assert fail. (These will be used for overflow checking from
rustc_trans::trans::consts.)
This commit cleans out a large amount of deprecated APIs from the standard
library and some of the facade crates as well, updating all users in the
compiler and in tests as it goes along.
This permits all coercions to be performed in casts, but adds lints to warn in those cases.
Part of this patch moves cast checking to a later stage of type checking. We acquire obligations to check casts as part of type checking where we previously checked them. Once we have type checked a function or module, then we check any cast obligations which have been acquired. That means we have more type information available to check casts (this was crucial to making coercions work properly in place of some casts), but it means that casts cannot feed input into type inference.
[breaking change]
* Adds two new lints for trivial casts and trivial numeric casts, these are warn by default, but can cause errors if you build with warnings as errors. Previously, trivial numeric casts and casts to trait objects were allowed.
* The unused casts lint has gone.
* Interactions between casting and type inference have changed in subtle ways. Two ways this might manifest are:
- You may need to 'direct' casts more with extra type information, for example, in some cases where `foo as _ as T` succeeded, you may now need to specify the type for `_`
- Casts do not influence inference of integer types. E.g., the following used to type check:
```
let x = 42;
let y = &x as *const u32;
```
Because the cast would inform inference that `x` must have type `u32`. This no longer applies and the compiler will fallback to `i32` for `x` and thus there will be a type error in the cast. The solution is to add more type information:
```
let x: u32 = 42;
let y = &x as *const u32;
```
at least that's what the docs say: http://doc.rust-lang.org/std/slice/fn.from_raw_parts.html
A few situations got prettier. In some situations the mutability of the resulting and source pointers differed (and was cast away by transmute), the mutability matches now.
This commit stabilizes essentially all of the new `std::path` API. The
API itself is changed in a couple of ways (which brings it in closer
alignment with the RFC):
* `.` components are now normalized away, unless they appear at the
start of a path. This in turn effects the semantics of e.g. asking for
the file name of `foo/` or `foo/.`, both of which yield `Some("foo")`
now. This semantics is what the original RFC specified, and is also
desirable given early experience rolling out the new API.
* The `parent` function now succeeds if, and only if, the path has at
least one non-root/prefix component. This change affects `pop` as
well.
* The `Prefix` component now involves a separate `PrefixComponent`
struct, to better allow for keeping both parsed and unparsed prefix data.
In addition, the `old_path` module is now deprecated.
Closes#23264
[breaking-change]
This commit deprecates the majority of std::old_io::fs in favor of std::fs and
its new functionality. Some functions remain non-deprecated but are now behind a
feature gate called `old_fs`. These functions will be deprecated once
suitable replacements have been implemented.
The compiler has been migrated to new `std::fs` and `std::path` APIs where
appropriate as part of this change.
This commit is an implementation of [RFC 592][r592] and [RFC 840][r840]. These
two RFCs tweak the behavior of `CString` and add a new `CStr` unsized slice type
to the module.
[r592]: https://github.com/rust-lang/rfcs/blob/master/text/0592-c-str-deref.md
[r840]: https://github.com/rust-lang/rfcs/blob/master/text/0840-no-panic-in-c-string.md
The new `CStr` type is only constructable via two methods:
1. By `deref`'ing from a `CString`
2. Unsafely via `CStr::from_ptr`
The purpose of `CStr` is to be an unsized type which is a thin pointer to a
`libc::c_char` (currently it is a fat pointer slice due to implementation
limitations). Strings from C can be safely represented with a `CStr` and an
appropriate lifetime as well. Consumers of `&CString` should now consume `&CStr`
instead to allow producers to pass in C-originating strings instead of just
Rust-allocated strings.
A new constructor was added to `CString`, `new`, which takes `T: IntoBytes`
instead of separate `from_slice` and `from_vec` methods (both have been
deprecated in favor of `new`). The `new` method returns a `Result` instead of
panicking. The error variant contains the relevant information about where the
error happened and bytes (if present). Conversions are provided to the
`io::Error` and `old_io::IoError` types via the `FromError` trait which
translate to `InvalidInput`.
This is a breaking change due to the modification of existing `#[unstable]` APIs
and new deprecation, and more detailed information can be found in the two RFCs.
Notable breakage includes:
* All construction of `CString` now needs to use `new` and handle the outgoing
`Result`.
* Usage of `CString` as a byte slice now explicitly needs a `.as_bytes()` call.
* The `as_slice*` methods have been removed in favor of just having the
`as_bytes*` methods.
Closes#22469Closes#22470
[breaking-change]
