When performing LTO, the rust compiler has an opportunity to completely strip
all landing pads in all dependent libraries. I've modified the LTO pass to
recognize the -Z no-landing-pads option when also running an LTO pass to flag
everything in LLVM as nothrow. I've verified that this prevents any and all
invoke instructions from being emitted.
I believe that this is one of our best options for moving forward with
accomodating use-cases where unwinding doesn't really make sense. This will
allow libraries to be built with landing pads by default but allow usage of them
in contexts where landing pads aren't necessary.
When performing LTO, the rust compiler has an opportunity to completely strip
all landing pads in all dependent libraries. I've modified the LTO pass to
recognize the -Z no-landing-pads option when also running an LTO pass to flag
everything in LLVM as nothrow. I've verified that this prevents any and all
invoke instructions from being emitted.
I believe that this is one of our best options for moving forward with
accomodating use-cases where unwinding doesn't really make sense. This will
allow libraries to be built with landing pads by default but allow usage of them
in contexts where landing pads aren't necessary.
cc #10780
This replaces the link meta attributes with a pkgid attribute and uses a hash
of this as the crate hash. This makes the crate hash computable by things
other than the Rust compiler. It also switches the hash function ot SHA1 since
that is much more likely to be available in shell, Python, etc than SipHash.
Fixes#10188, #8523.
This commit implements LTO for rust leveraging LLVM's passes. What this means
is:
* When compiling an rlib, in addition to insdering foo.o into the archive, also
insert foo.bc (the LLVM bytecode) of the optimized module.
* When the compiler detects the -Z lto option, it will attempt to perform LTO on
a staticlib or binary output. The compiler will emit an error if a dylib or
rlib output is being generated.
* The actual act of performing LTO is as follows:
1. Force all upstream libraries to have an rlib version available.
2. Load the bytecode of each upstream library from the rlib.
3. Link all this bytecode into the current LLVM module (just using llvm
apis)
4. Run an internalization pass which internalizes all symbols except those
found reachable for the local crate of compilation.
5. Run the LLVM LTO pass manager over this entire module
6a. If assembling an archive, then add all upstream rlibs into the output
archive. This ignores all of the object/bitcode/metadata files rust
generated and placed inside the rlibs.
6b. If linking a binary, create copies of all upstream rlibs, remove the
rust-generated object-file, and then link everything as usual.
As I have explained in #10741, this process is excruciatingly slow, so this is
*not* turned on by default, and it is also why I have decided to hide it behind
a -Z flag for now. The good news is that the binary sizes are about as small as
they can be as a result of LTO, so it's definitely working.
Closes#10741Closes#10740
Right now whenever an rlib file is linked against, all of the metadata from the
rlib is pulled in to the final staticlib or binary. The reason for this is that
the metadata is currently stored in a section of the object file. Note that this
is intentional for dynamic libraries in order to distribute metadata bundled
with static libraries.
This commit alters the situation for rlib libraries to instead store the
metadata in a separate file in the archive. In doing so, when the archive is
passed to the linker, none of the metadata will get pulled into the result
executable. Furthermore, the metadata file is skipped when assembling rlibs into
an archive.
The snag in this implementation comes with multiple output formats. When
generating a dylib, the metadata needs to be in the object file, but when
generating an rlib this needs to be separate. In order to accomplish this, the
metadata variable is inserted into an entirely separate LLVM Module which is
then codegen'd into a different location (foo.metadata.o). This is then linked
into dynamic libraries and silently ignored for rlib files.
While changing how metadata is inserted into archives, I have also stopped
compressing metadata when inserted into rlib files. We have wanted to stop
compressing metadata, but the sections it creates in object file sections are
apparently too large. Thankfully if it's just an arbitrary file it doesn't
matter how large it is.
I have seen massive reductions in executable sizes, as well as staticlib output
sizes (to confirm that this is all working).
The main one removed is rust_upcall_reset_stack_limit (continuation of #10156),
and this also removes the upcall_trace function. The was hidden behind a
`-Z trace` flag, but if you attempt to use this now you'll get a linker error
because there is no implementation of the 'upcall_trace' function. Due to this
no longer working, I decided to remove it entirely from the compiler (I'm also a
little unsure on what it did in the first place).
* Don't flag any address_insignificant statics as reachable because the whole
point of the address_insignificant optimization is that the static is not
reachable. Additionally, there's no need for it to be reachable because LLVM
optimizes it away.
* Be sure to not leak external node ids into our reachable set, this can
spuriously cause local items to be considered reachable if the node ids just
happen to line up
**Note**: I only tested on top of my #10670 PR, size reductions come from both change sets.
With this, [more enums are shrinked](https://gist.github.com/eddyb/08fef0dfc6ff54e890bc), the most significant one being `ast_node`, from 104 bytes (master) to 96 (#10670) and now to 32 bytes.
My own testcase requires **200MB** less when compiling (not including the other **200MB** gained in #10670), and rustc-stage2 is down by about **130MB**.
I believe there is more to gain by fiddling with the enums' layouts.
In this series of commits, I've implemented static linking for rust. The scheme I implemented was the same as my [mailing list post](https://mail.mozilla.org/pipermail/rust-dev/2013-November/006686.html).
The commits have more details to the nitty gritty of what went on. I've rebased this on top of my native mutex pull request (#10479), but I imagine that it will land before this lands, I just wanted to pre-emptively get all the rebase conflicts out of the way (becuase this is reorganizing building librustrt as well).
Some contentious points I want to make sure are all good:
* I've added more "compiler chooses a default" behavior than I would like, I want to make sure that this is all very clearly outlined in the code, and if not I would like to remove behavior or make it clearer.
* I want to make sure that the new "fancy suite" tests are ok (using make/python instead of another rust crate)
If we do indeed pursue this, I would be more than willing to write up a document describing how linking in rust works. I believe that this behavior should be very understandable, and the compiler should never hinder someone just because linking is a little fuzzy.
In #10422, I didn't actually test to make sure that the '-Z gen-crate-map'
option was usable before I implemented it. The crate map was indeed generated
when '-Z gen-crate-map' was specified, but the I/O factory slot was empty
because of an extra check in trans about filling in that location.
This commit both fixes that location, and checks in a "fancy test" which does
lots of fun stuff. The test will use the rustc library to compile a rust crate,
and then compile a C program to link against that crate and run the C program.
To my knowledge this is the first test of its kind, so it's a little ad-hoc, but
it seems to get the job done. We could perhaps generalize running tests like
this, but for now I think it's fine to have this sort of functionality tucked
away in a test.
This commit implements the support necessary for generating both intermediate
and result static rust libraries. This is an implementation of my thoughts in
https://mail.mozilla.org/pipermail/rust-dev/2013-November/006686.html.
When compiling a library, we still retain the "lib" option, although now there
are "rlib", "staticlib", and "dylib" as options for crate_type (and these are
stackable). The idea of "lib" is to generate the "compiler default" instead of
having too choose (although all are interchangeable). For now I have left the
"complier default" to be a dynamic library for size reasons.
Of the rust libraries, lib{std,extra,rustuv} will bootstrap with an
rlib/dylib pair, but lib{rustc,syntax,rustdoc,rustpkg} will only be built as a
dynamic object. I chose this for size reasons, but also because you're probably
not going to be embedding the rustc compiler anywhere any time soon.
Other than the options outlined above, there are a few defaults/preferences that
are now opinionated in the compiler:
* If both a .dylib and .rlib are found for a rust library, the compiler will
prefer the .rlib variant. This is overridable via the -Z prefer-dynamic option
* If generating a "lib", the compiler will generate a dynamic library. This is
overridable by explicitly saying what flavor you'd like (rlib, staticlib,
dylib).
* If no options are passed to the command line, and no crate_type is found in
the destination crate, then an executable is generated
With this change, you can successfully build a rust program with 0 dynamic
dependencies on rust libraries. There is still a dynamic dependency on
librustrt, but I plan on removing that in a subsequent commit.
This change includes no tests just yet. Our current testing
infrastructure/harnesses aren't very amenable to doing flavorful things with
linking, so I'm planning on adding a new mode of testing which I believe belongs
as a separate commit.
Closes#552
This is needed so that the FFI works as expected on platforms that don't
flatten aggregates the way the AMD64 ABI does, especially for `#[repr(C)]`.
This moves more of `type_of` into `trans::adt`, because the type might
or might not be an LLVM struct.
This was needed to access UEFI boot services in my new Boot2Rust experiment.
I also realized that Rust functions declared as extern always use the C calling convention regardless of how they were declared, so this pull request fixes that as well.
This replaces `*` with `..` in enums, `_` with `..` in structs, and `.._` with `..` in vectors. It adds obsolete syntax warnings for the old forms but doesn't turn them on yet because we need a snapshot.
#5830
This PR improves the single-stepping experience for if-expression (no more jumping into the *else* branch before entering the *then* branch, no more jumping to the end of the *else* branch after finishing the *then* branch). Unfortunately I don't know of a straight-forward way of writing automated tests for this. Suggestions welcome!
If a function is marked as external, then it's likely desired for use with some
native library, so we're not really accomplishing a whole lot by internalizing
all of these symbols.
As we start to move runtime components into the crate map, it's becoming harder
and harder to start the runtime from a C function as rust is embedded in another
application. Right now if you compile a rust crate as a dynamic library which is
then linked to another application, when using std::rt::start there are no I/O
local services, even though rustuv was linked against and requested. The reason
for this is that there is no top level crate map available specifying where to
find libuv I/O.
This option is not meant to be used regularly, but rather whenever compiling a
final library crate and linking it into another application. This lifts the
requirement that to get a crate map you must have the final destination be an
executable.
These two attributes are no longer useful now that Rust has decided to leave
segmented stacks behind. It is assumed that the rust task's stack is always
large enough to make an FFI call (due to the stack being very large).
There's always the case of stack overflow, however, to consider. This does not
change the behavior of stack overflow in Rust. This is still normally triggered
by the __morestack function and aborts the whole process.
C stack overflow will continue to corrupt the stack, however (as it did before
this commit as well). The future improvement of a guard page at the end of every
rust stack is still unimplemented and is intended to be the mechanism through
which we attempt to detect C stack overflow.
Closes#8822Closes#10155
As we start to move runtime components into the crate map, it's becoming harder
and harder to start the runtime from a C function as rust is embedded in another
application. Right now if you compile a rust crate as a dynamic library which is
then linked to another application, when using std::rt::start there are no I/O
local services, even though rustuv was linked against and requested. The reason
for this is that there is no top level crate map available specifying where to
find libuv I/O.
This option is not meant to be used regularly, but rather whenever compiling a
final library crate and linking it into another application. This lifts the
requirement that to get a crate map you must have the final destination be an
executable.
This adds an other ABI option which allows a custom selection over the target
architecture and OS. The only current candidate for this change is that kernel32
on win32 uses stdcall, but on win64 it uses the cdecl calling convention.
Otherwise everywhere else this is defined as using the Cdecl calling convention.
cc #10049Closes#8774
