When a syntax extension is loaded by the compiler, the dylib that is opened may
have other dylibs that it depends on. The dynamic linker must be able to find
these libraries on the system or else the library will fail to load.
Currently, unix gets by with the use of rpaths. This relies on the dylib not
moving around too drastically relative to its dependencies. For windows,
however, this is no rpath available, and in theory unix should work without
rpaths as well.
This modifies the compiler to add all -L search directories to the dynamic
linker's set of load paths. This is currently managed through environment
variables for each platform.
Closes#13848
When a syntax extension is loaded by the compiler, the dylib that is opened may
have other dylibs that it depends on. The dynamic linker must be able to find
these libraries on the system or else the library will fail to load.
Currently, unix gets by with the use of rpaths. This relies on the dylib not
moving around too drastically relative to its dependencies. For windows,
however, this is no rpath available, and in theory unix should work without
rpaths as well.
This modifies the compiler to add all -L search directories to the dynamic
linker's set of load paths. This is currently managed through environment
variables for each platform.
Closes#13848
Similar to my recent changes to ~[T]/&[T], these changes remove the vstore abstraction and represent str types as ~(str) and &(str). The Option<uint> in ty_str is the length of the string, None if the string is dynamically sized.
This allows the use of syntax extensions when cross-compiling (fixing #12102). It does this by encoding the target triple in the crate metadata and checking it when searching for files. Currently the crate triple must match the host triple when there is a macro_registrar_fn, it must match the target triple when linking, and can match either when only macro_rules! macros are used.
due to carelessness, this is pretty much a duplicate of https://github.com/mozilla/rust/pull/13450.
This adds the target triple to the crate metadata.
When searching for a crate the phase (link, syntax) is taken into account.
During link phase only crates matching the target triple are considered.
During syntax phase, either the target or host triple will be accepted, unless
the crate defines a macro_registrar, in which case only the host triple will
match.
This alters the borrow checker's requirements on invoking closures from
requiring an immutable borrow to requiring a unique immutable borrow. This means
that it is illegal to invoke a closure through a `&` pointer because there is no
guarantee that is not aliased. This does not mean that a closure is required to
be in a mutable location, but rather a location which can be proven to be
unique (often through a mutable pointer).
For example, the following code is unsound and is no longer allowed:
type Fn<'a> = ||:'a;
fn call(f: |Fn|) {
f(|| {
f(|| {})
});
}
fn main() {
call(|a| {
a();
});
}
There is no replacement for this pattern. For all closures which are stored in
structures, it was previously allowed to invoke the closure through `&self` but
it now requires invocation through `&mut self`.
The standard library has a good number of violations of this new rule, but the
fixes will be separated into multiple breaking change commits.
Closes#12224
Now with proper checking of enums and allows unsized fields as the last field in a struct or variant. This PR only checks passing of unsized types and distinguishing them from sized ones. To be safe we also need to control storage.
Closes issues #12969 and #13121, supersedes #13375 (all the discussion there is valid here too).
This currently requires linking against a library like libquadmath (or
libgcc), because compiler-rt barely has any support for this and most
hardware does not yet have 128-bit precision floating point. For this
reason, it's currently hidden behind a feature gate.
When compiler-rt is updated to trunk, some tests can be added for
constant evaluation since there will be support for the comparison
operators.
Closes#13381
Refactors all uses of ty_vec and associated things to remove the vstore abstraction (still used for strings, for now). Pointers to vectors are stored as ty_rptr or ty_uniq wrapped around a ty_vec. There are no user-facing changes. Existing behaviour is preserved by special-casing many instances of pointers containing vectors. Hopefully with DST most of these hacks will go away. For now it is useful to leave them hanging around rather than abstracting them into a method or something.
Closes#13554.
Refactores all uses of ty_vec and associated things to remove the vstore abstraction (still used for strings, for now). Pointers to vectors are stored as ty_rptr or ty_uniq wrapped around a ty_vec. There are no user-facing changes. Existing behaviour is preserved by special-casing many instances of pointers containing vectors. Hopefully with DST most of these hacks will go away. For now it is useful to leave them hanging around rather than abstracting them into a method or something.
Closes#13554.
Syntax-only crates are no longer registered with the cstore, so there's no need
to allocate crate numbers to them. This ends up leaving gaps in the crate
numbering scheme which is not expected in the rest of the compiler.
Closes#13560
Syntax-only crates are no longer registered with the cstore, so there's no need
to allocate crate numbers to them. This ends up leaving gaps in the crate
numbering scheme which is not expected in the rest of the compiler.
Closes#13560
This removes the `priv` keyword from the language and removes private enum
variants as a result. The remaining use cases of private enum variants were all
updated to be a struct with one private field that is a private enum.
RFC: 0006-remove-priv
Closes#13535
This bug was introduced in #13384 by accident, and this commit continues the
work of #13384 by finishing support for loading a syntax extension crate without
registering it with the local cstore.
Closes#13495
Cleans up some remnants of the old mutability system and only allows vector/trait mutability in `VstoreSlice` (`&mut [T]`) and `RegionTraitStore` (`&mut Trait`).
libstd: Implement `StrBuf`, a new string buffer type like `Vec`, and port all code over to use it.
Rebased & tests-fixed version of https://github.com/mozilla/rust/pull/13269
When calculating the sysroot, it's more accurate to use realpath() rather than
just one readlink() to account for any intermediate symlinks that the rustc
binary resolves itself to.
For rpath, realpath() is necessary because the rpath must dictate a relative
rpath from the destination back to the originally linked library, which works
more robustly if there are no symlinks involved.
Concretely, any binary generated on OSX into $TMPDIR requires an absolute rpath
because the temporary directory is behind a symlink with one layer of
indirection. This symlink causes all relative rpaths to fail to resolve.
cc #11734
cc #11857
This is an optimization which is quite impactful for compiling small crates.
Reading libstd's metadata takes about 50ms, and a hello world before this change
took about 100ms (this change halves that time).
Recent changes made it such that this optimization wasn't performed, but I think
it's a better idea do to this for now. See #10786 for tracking this issue.
When linking, all crates in the local CStore are used to link the final product.
With #[phase(syntax)], crates want to be omitted from this linkage phase, and
this was achieved by dumping the entire CStore after loading crates. This causes
crates like the standard library to get loaded twice. This loading process is a
fairly expensive operation when dealing with decompressing metadata.
This commit alters the loading process to never register syntax crates in
CStore. Instead, only phase(link) crates ever make their way into the map of
crates. The CrateLoader trait was altered to return everything in one method
instead of having separate methods for finding information.
This separate crate cache is one factor which is causing libstd to be loaded
twice during normal compilation. The crates loaded for syntax extensions have a
separate cache than the crates loaded for linking, so all crates are loaded once
per #[phase] they're tagged with.
This removes the cache and instead uses the CStore structure itself as the cache
for loaded crates. This should allow crates loaded during the syntax phase to be
shared with the crates loaded during the link phase.
Fix#13266.
There is a little bit of acrobatics in the definition of `crate_paths`
to avoid calling `clone()` on the dylib/rlib unless we actually are
going to need them.
The other oddity is that I have replaced the `root_ident: Option<&str>`
parameter with a `root: &Option<CratePaths>`, which may surprise one
who was expecting to see something like: `root: Option<&CratePaths>`.
I went with the approach here because I could not come up with code for
the alternative that was acceptable to the borrow checker.
(i.e. semi-generalized version of prior errorinfo gathering.)
Also revised presentation to put each path on its own line, prefixed
by file:linenum information.
`RefCell::get` can be a bit surprising, because it actually clones the wrapped value. This removes `RefCell::get` and replaces all the users with `RefCell::borrow()` when it can, and `RefCell::borrow().clone()` when it can't. It removes `RefCell::set` for consistency. This closes#13182.
It also fixes an infinite loop in a test when debugging is on.
It's surprising that `RefCell::get()` is implicitly doing a clone
on a value. This patch removes it and replaces all users with
either `.borrow()` when we can autoderef, or `.borrow().clone()`
when we cannot.
This change removes the AbiSet from the AST, converting all usage to have just
one Abi value. The current scheme selects a relevant ABI given a list of ABIs
based on the target architecture and how relevant each ABI is to that
architecture.
Instead of this mildly complicated scheme, only one ABI will be allowed in abi
strings, and pseudo-abis will be created for special cases as necessary. For
example the "system" abi exists for stdcall on win32 and C on win64.
Closes#10049
The recent pull request to remove libc from libstd has hit a wall in compiling
on windows, and I've been trying to investigate on the try bots as to why (it
compiles locally just fine). To the best of my knowledge, the LLVM section
iterator is behaving badly when iterating over the sections of the libc DLL.
Upon investigating the LLVMGetSectionName function in LLVM, I discovered that
this function doesn't always return a null-terminated string. It returns the
data pointer of a StringRef instance (LLVM's equivalent of &str essentially),
but it has no method of returning the length of the name of the section.
This commit modifies the section iteration when loading libraries to invoke a
custom LLVMRustGetSectionName which will correctly return both the length and
the data pointer.
I have not yet verified that this will fix landing liblibc, as it will require a
snapshot before doing a full test. Regardless, this is a worrisome situation
regarding the LLVM API, and should likely be fixed anyway.
Fix#13266.
There is a little bit of acrobatics in the definition of `crate_paths`
to avoid calling `clone()` on the dylib/rlib unless we actually are
going to need them.
The other oddity is that I have replaced the `root_ident: Option<&str>`
parameter with a `root: &Option<CratePaths>`, which may surprise one
who was expecting to see something like: `root: Option<&CratePaths>`.
I went with the approach here because I could not come up with code for
the alternative that was acceptable to the borrow checker.
This is a continuation of the work done in #13184 to make struct fields private
by default. This commit finishes RFC 4 by making all tuple structs have private
fields by default. Note that enum variants are not affected.
A tuple struct having a private field means that it cannot be matched on in a
pattern match (both refutable and irrefutable), and it also cannot have a value
specified to be constructed. Similarly to private fields, switching the type of
a private field in a tuple struct should be able to be done in a backwards
compatible way.
The one snag that I ran into which wasn't mentioned in the RFC is that this
commit also forbids taking the value of a tuple struct constructor. For example,
this code now fails to compile:
mod a {
pub struct A(int);
}
let a: fn(int) -> a::A = a::A; //~ ERROR: first field is private
Although no fields are bound in this example, it exposes implementation details
through the type itself. For this reason, taking the value of a struct
constructor with private fields is forbidden (outside the containing module).
RFC: 0004-private-fields
Summary:
So far, we've used the term POD "Plain Old Data" to refer to types that
can be safely copied. However, this term is not consistent with the
other built-in bounds that use verbs instead. This patch renames the Pod
kind into Copy.
RFC: 0003-opt-in-builtin-traits
Test Plan: make check
Reviewers: cmr
Differential Revision: http://phabricator.octayn.net/D3
A variety of stuff here, mostly aimed at making `make install` work correctly with `--libdir` and `--mandir`. `make install` again goes through `install.sh`.
This change is in preparation for #8122. Nothing is currently done with these
visibility qualifiers, they are just parsed and accepted by the compiler.
RFC: 0004-private-fields
This is not sufficient for finding the library directory for binary
installs, but it does make the build more complex by requiring
env vars be set to build rustc.
This needs to be removed as part of removing `~[T]`. Partial type hints
are now allowed, and will remove the need to add a version of this
method for `Vec<T>`. For now, this involves a few workarounds for
partial type hints not completely working.
These are superfluous now that we have fixed rvalue lifetimes and Deref.
I'd also like to kill off `get` and `set`, but that'll be a large change so I want to make sure that we actually want to do that first.
This is the first step to replacing OptVec with a new representation:
remove all mutability. Any mutations have to go via `Vec` and then make
to `OptVec`.
Many of the uses of OptVec are unnecessary now that Vec has no-alloc
emptiness (and have been converted to Vec): the only ones that really
need it are the AST and sty's (and so on) where there are a *lot* of
instances of them, and they're (mostly) immutable.
Previously, any library of the pattern `lib<name>-<hash>-<version>.so` was
>considered a candidate (rightly so) for loading a crate. Sets are generated for
each unique `<hash>`, and then from these sets a candidate is selected. If a set
contained more than one element, then it immediately generated an error saying
that multiple copies of the same dylib were found.
This is incorrect because each candidate needs to be validated to actually
contain a rust library (valid metadata). This commit alters the logic to filter
each set of candidates for a hash to only libraries which are actually rust
libraries. This means that if multiple false positives are found with the right
name pattern, they're all ignored.
Closes#13010
`Share` implies that all *reachable* content is *threadsafe*.
Threadsafe is defined as "exposing no operation that permits a data race if multiple threads have access to a &T pointer simultaneously". (NB: the type system should guarantee that if you have access to memory via a &T pointer, the only other way to gain access to that memory is through another &T pointer)...
Fixes#11781
cc #12577
What this PR will do
================
- [x] Add Share kind and
- [x] Replace usages of Freeze with Share in bounds.
- [x] Add Unsafe<T> #12577
- [x] Forbid taking the address of a immutable static item with `Unsafe<T>` interior
What's left to do in a separate PR (after the snapshot)?
===========================================
- Remove `Freeze` completely
Previously, any library of the pattern `lib<name>-<hash>-<version>.so` was
>considered a candidate (rightly so) for loading a crate. Sets are generated for
each unique `<hash>`, and then from these sets a candidate is selected. If a set
contained more than one element, then it immediately generated an error saying
that multiple copies of the same dylib were found.
This is incorrect because each candidate needs to be validated to actually
contain a rust library (valid metadata). This commit alters the logic to filter
each set of candidates for a hash to only libraries which are actually rust
libraries. This means that if multiple false positives are found with the right
name pattern, they're all ignored.
Closes#13010
This reduces the size of sty from 112 to 96; like with the ty_trait
variant, this variant of sty occurs rarely (~1%) so the benefits are
large and the costs small.
This reduces ty::sty from 160 bytes to just 112, and some measurements
eddyb made suggest that the ty_trait variant occurs very
rarely (e.g. ~1% of all sty instances) hence this will result in a large
memory saving, and the cost of the indirection is unlikely to be an
issue.
This commit switches over the backtrace infrastructure from piggy-backing off
the RUST_LOG environment variable to using the RUST_BACKTRACE environment
variable (logging is now disabled in libstd).
This commit starts to topographically sort rust dependencies on the linker
command line. The reason for this is that linkers use right-hand libraries to
resolve left-hand libraries symbols, which is especially crucial for us because
we're using --as-needed on linux.
This commit shreds all remnants of libextra from the compiler and standard
distribution. Two modules, c_vec/tempfile, were moved into libstd after some
cleanup, and the other modules were moved to separate crates as seen fit.
Closes#8784Closes#12413Closes#12576
Refactored get_metadata_section to return a Result<MetadataBlob,~str> instead of a Option<MetadataBlob>. This provides more clarity to the user through the debug output when using --ls.
This is kind of a continuation of my original closed pull request 2 months ago (#11544), but I think the time-span constitutes a new pull request.
This leverages the new hashing framework and hashmap implementation to provide a
much speedier hashing algorithm for node ids and def ids. The hash algorithm
used is currentl FNV hashing, but it's quite easy to swap out.
I originally implemented hashing as the identity function, but this actually
ended up in slowing down rustc compiling libstd from 8s to 13s. I would suspect
that this is a result of a large number of collisions.
With FNV hashing, we get these timings (compiling with --no-trans, in seconds):
| | before | after |
|-----------|---------:|--------:|
| libstd | 8.324 | 6.703 |
| stdtest | 47.674 | 46.857 |
| libsyntax | 9.918 | 8.400 |
Linker argument order with respect to libraries is important enough that we
shouldn't be attempting to filter out libraries getting passed through to the
linker. When linking with a native library that has multiple dependant native
libraries, it's useful to have control over the link argument order.
When the metadata format changes, old libraries often cause librustc to abort
when reading their metadata. This should all change with the introduction of SVH
markers, but the loader for crates should gracefully handle libraries without
SVH markers still.
This commit adds support for tripping fewer assertions when loading libraries by
using maybe_get_doc when initially parsing metadata. It's still possible for
some libraries to fall through the cracks, but this should deal with a fairly
large number of them up front.
This new SVH is used to uniquely identify all crates as a snapshot in time of
their ABI/API/publicly reachable state. This current calculation is just a hash
of the entire crate's AST. This is obviously incorrect, but it is currently the
reality for today.
This change threads through the new Svh structure which originates from crate
dependencies. The concept of crate id hash is preserved to provide efficient
matching on filenames for crate loading. The inspected hash once crate metadata
is opened has been changed to use the new Svh.
The goal of this hash is to identify when upstream crates have changed but
downstream crates have not been recompiled. This will prevent the def-id drift
problem where upstream crates were recompiled, thereby changing their metadata,
but downstream crates were not recompiled.
In the future this hash can be expanded to exclude contents of the AST like doc
comments, but limitations in the compiler prevent this change from being made at
this time.
Closes#10207
The previous code passed around a {name,version} pair everywhere, but this is
better expressed as a CrateId. This patch changes these paths to store and pass
around crate ids instead of these pairs of name/version. This also prepares the
code to change the type of hash that is stored in crates.
This commit implements a layman's version of realpath() for metadata::loader to
use in order to not error on symlinks pointing to the same file.
Closes#12459
Commits for details. Highlights:
- `flate` returns `CVec<u8>` to save reallocating a whole new `&[u8]`
- a lot of `transmute`s removed outright or replaced with `as` (etc.)
This trades an O(n) allocation + memcpy for a O(1) proc allocation (for
the destructor). Most users only need &[u8] anyway (all of the users in
the main repo), and so this offers large gains.
These two containers are indeed collections, so their place is in
libcollections, not in libstd. There will always be a hash map as part of the
standard distribution of Rust, but by moving it out of the standard library it
makes libstd that much more portable to more platforms and environments.
This conveniently also removes the stuttering of 'std::hashmap::HashMap',
although 'collections::HashMap' is only one character shorter.
This PR merges `IterBytes` and `Hash` into a trait that allows for generic non-stream-based hashing. It makes use of @eddyb's default type parameter support in order to have a similar usage to the old `Hash` framework.
Fixes#8038.
Todo:
- [x] Better documentation
- [ ] Benchmark
- [ ] Parameterize `HashMap` on a `Hasher`.
Closes#12366.
Parentheses around assignment statements such as
let mut a = (0);
a = (1);
a += (2);
are not necessary and therefore an unnecessary_parens warning is raised when
statements like this occur.
The warning mechanism was refactored along the way to allow for code reuse
between the routines for checking expressions and statements.
Code had to be adopted throughout the compiler and standard libraries to comply
with this modification of the lint.
This patch merges IterBytes and Hash traits, which clears up the
confusion of using `#[deriving(IterBytes)]` to support hashing.
Instead, it now is much easier to use the new `#[deriving(Hash)]`
for making a type hashable with a stream hash.
Furthermore, it supports custom non-stream-based hashers, such as
if a value's hash was cached in a database.
This does not yet replace the old IterBytes-hash with this new
version.
Added allow(non_camel_case_types) to librustc where necesary
Tried to fix problems with non_camel_case_types outside rustc
fixed failing tests
Docs updated
Moved #[allow(non_camel_case_types)] a level higher.
markdown.rs reverted
Fixed timer that was failing tests
Fixed another timer
The first commit improves error messages during linking, and the second commit improves error messages during crate-loading time.
Closes#12297Closes#12377
This commit rewrites crate loading internally in attempt to look at less
metadata and provide nicer errors. The loading is now split up into a few
stages:
1. Collect a mapping of (hash => ~[Path]) for a set of candidate libraries for a
given search. The hash is the hash in the filename and the Path is the
location of the library in question. All candidates are filtered based on
their prefix/suffix (dylib/rlib appropriate) and then the hash/version are
split up and are compared (if necessary).
This means that if you're looking for an exact hash of library you don't have
to open up the metadata of all libraries named the same, but also in your
path.
2. Once this mapping is constructed, each (hash, ~[Path]) pair is filtered down
to just a Path. This is necessary because the same rlib could show up twice
in the path in multiple locations. Right now the filenames are based on just
the crate id, so this could be indicative of multiple version of a crate
during one crate_id lifetime in the path. If multiple duplicate crates are
found, an error is generated.
3. Now that we have a mapping of (hash => Path), we error on multiple versions
saying that multiple versions were found. Only if there's one (hash => Path)
pair do we actually return that Path and its metadata.
With this restructuring, it restructures code so errors which were assertions
previously are now first-class errors. Additionally, this should read much less
metadata with lots of crates of the same name or same version in a path.
Closes#11908
This "bubble up an error" macro was originally named if_ok! in order to get it
landed, but after the fact it was discovered that this name is not exactly
desirable.
The name `if_ok!` isn't immediately clear that is has much to do with error
handling, and it doesn't look fantastic in all contexts (if if_ok!(...) {}). In
general, the agreed opinion about `if_ok!` is that is came in as subpar.
The name `try!` is more invocative of error handling, it's shorter by 2 letters,
and it looks fitting in almost all circumstances. One concern about the word
`try!` is that it's too invocative of exceptions, but the belief is that this
will be overcome with documentation and examples.
Close#12037
The first setp for #9880 is to add a new `crate` keyword. This PR does exactly that. I took a chance to refactor `parse_item_foreign_mod` and I broke it down into 2 separate methods to isolate each feature.
The next step will be to push a new stage0 snapshot and then get rid of all `extern mod` around the code.
This patch replaces all `crate` usage with `krate` before introducing the
new keyword. This ensures that after introducing the keyword, there
won't be any compilation errors.
krate might not be the most expressive substitution for crate but it's a
very close abbreviation for it. `module` was already used in several
places already.
Error messages cleaned in librustc/middle
Error messages cleaned in libsyntax
Error messages cleaned in libsyntax more agressively
Error messages cleaned in librustc more aggressively
Fixed affected tests
Fixed other failing tests
Last failing tests fixed
- `extra::json` didn't make the cut, because of `extra::json` required
dep on `extra::TreeMap`. If/when `extra::TreeMap` moves out of `extra`,
then `extra::json` could move into `serialize`
- `libextra`, `libsyntax` and `librustc` depend on the newly created
`libserialize`
- The extensions to various `extra` types like `DList`, `RingBuf`, `TreeMap`
and `TreeSet` for `Encodable`/`Decodable` were moved into the respective
modules in `extra`
- There is some trickery, evident in `src/libextra/lib.rs` where a stub
of `extra::serialize` is set up (in `src/libextra/serialize.rs`) for
use in the stage0 build, where the snapshot rustc is still making
deriving for `Encodable` and `Decodable` point at extra. Big props to
@huonw for help working out the re-export solution for this
extra: inline extra::serialize stub
fix stuff clobbered in rebase + don't reexport serialize::serialize
no more globs in libserialize
syntax: fix import of libserialize traits
librustc: fix bad imports in encoder/decoder
add serialize dep to librustdoc
fix failing run-pass tests w/ serialize dep
adjust uuid dep
more rebase de-clobbering for libserialize
fixing tests, pushing libextra dep into cfg(test)
fix doc code in extra::json
adjust index.md links to serialize and uuid library
This removes @[] from the parser as well as much of the handling of it (and `@str`) from the compiler as I can find.
I've just rebased @pcwalton's (already reviewed) `@str` removal (and fixed the problems in a separate commit); the only new work is the trailing commits with my authorship.
Closes#11967
