This commit switches over the backtrace infrastructure from piggy-backing off
the RUST_LOG environment variable to using the RUST_BACKTRACE environment
variable (logging is now disabled in libstd).
This commit removes all internal support for the previously used __log_level()
expression. The logging subsystem was previously modified to not rely on this
magical expression. This also removes the only other function to use the
module_data map in trans, decl_gc_metadata. It appears that this is an ancient
function from a GC only used long ago.
This does not remove the crate map entirely, as libgreen still uses it to hook
in to the event loop provided by libgreen.
This commit moves all logging out of the standard library into an external
crate. This crate is the new crate which is responsible for all logging macros
and logging implementation. A few reasons for this change are:
* The crate map has always been a bit of a code smell among rust programs. It
has difficulty being loaded on almost all platforms, and it's used almost
exclusively for logging and only logging. Removing the crate map is one of the
end goals of this movement.
* The compiler has a fair bit of special support for logging. It has the
__log_level() expression as well as generating a global word per module
specifying the log level. This is unfairly favoring the built-in logging
system, and is much better done purely in libraries instead of the compiler
itself.
* Initialization of logging is much easier to do if there is no reliance on a
magical crate map being available to set module log levels.
* If the logging library can be written outside of the standard library, there's
no reason that it shouldn't be. It's likely that we're not going to build the
highest quality logging library of all time, so third-party libraries should
be able to provide just as high-quality logging systems as the default one
provided in the rust distribution.
With a migration such as this, the change does not come for free. There are some
subtle changes in the behavior of liblog vs the previous logging macros:
* The core change of this migration is that there is no longer a physical
log-level per module. This concept is still emulated (it is quite useful), but
there is now only a global log level, not a local one. This global log level
is a reflection of the maximum of all log levels specified. The previously
generated logging code looked like:
if specified_level <= __module_log_level() {
println!(...)
}
The newly generated code looks like:
if specified_level <= ::log::LOG_LEVEL {
if ::log::module_enabled(module_path!()) {
println!(...)
}
}
Notably, the first layer of checking is still intended to be "super fast" in
that it's just a load of a global word and a compare. The second layer of
checking is executed to determine if the current module does indeed have
logging turned on.
This means that if any module has a debug log level turned on, all modules
with debug log levels get a little bit slower (they all do more expensive
dynamic checks to determine if they're turned on or not).
Semantically, this migration brings no change in this respect, but
runtime-wise, this will have a perf impact on some code.
* A `RUST_LOG=::help` directive will no longer print out a list of all modules
that can be logged. This is because the crate map will no longer specify the
log levels of all modules, so the list of modules is not known. Additionally,
warnings can no longer be provided if a malformed logging directive was
supplied.
The new "hello world" for logging looks like:
#[phase(syntax, link)]
extern crate log;
fn main() {
debug!("Hello, world!");
}
I ignored AtomicU64 methods on MIPS target
because libgcc doesn't implement MIPS32 64-bit atomic operations.
Otherwise it would cause link failure.
By the way, the patched LLVM doesn't have MIPS split stack anymore.
Should I file an issue about that?
On android, libgcc is missing the _Unwind_GetIP symbol because it's defined as a
macro. This is the same case for arm linux, so this commit adds the necessary
cfgs in place to use the "expanded macro" in rust for arm linux.
Add some more infrastructure support for demangling `$`-sequences, as well as fixing demangling of closure symbol names if there's more than one closure in a function.
This is something that is plausibly useful, and is provided by libuv. This is
not currently surfaced as part of the `TcpStream` type, but it may possibly
appear in the future. For now only the raw functionality is provided through the
Rtio objects.
The rust compiler not only outputs symbols in the form that C++ does, but it
also mangle symbols like '&' and '~' to special compiler-defined escape
sequences. For convenience, these symbols are demangled when printing
backtraces.
This is something that is plausibly useful, and is provided by libuv. This is
not currently surfaced as part of the `TcpStream` type, but it may possibly
appear in the future. For now only the raw functionality is provided through the
Rtio objects.
Whenever a failure happens, if a program is run with
`RUST_LOG=std::rt::backtrace` a backtrace will be printed to the task's stderr
handle. Stack traces are uncondtionally printed on double-failure and
rtabort!().
This ended up having a nontrivial implementation, and here's some highlights of
it:
* We're bundling libbacktrace for everything but OSX and Windows
* We use libgcc_s and its libunwind apis to get a backtrace of instruction
pointers
* On OSX we use dladdr() to go from an instruction pointer to a symbol
* On unix that isn't OSX, we use libbacktrace to get symbols
* Windows, as usual, has an entirely separate implementation
Lots more fun details and comments can be found in the source itself.
Closes#10128
It is often convenient to have forms of weak linkage or other various types of
linkage. Sadly, just using these flavors of linkage are not compatible with
Rust's typesystem and how it considers some pointers to be non-null.
As a compromise, this commit adds support for weak linkage to external symbols,
but it requires that this is only placed on extern statics of type `*T`.
Codegen-wise, we get translations like:
// rust code
extern {
#[linkage = "extern_weak"]
static foo: *i32;
}
// generated IR
@foo = extern_weak global i32
@_some_internal_symbol = internal global *i32 @foo
All references to the rust value of `foo` then reference `_some_internal_symbol`
instead of the symbol `_foo` itself. This allows us to guarantee that the
address of `foo` will never be null while the value may sometimes be null.
An example was implemented in `std::rt::thread` to determine if
`__pthread_get_minstack()` is available at runtime, and a test is checked in to
use it for a static value as well. Function pointers a little odd because you
still need to transmute the pointer value to a function pointer, but it's
thankfully better than not having this capability at all.
When using tasks in Rust, the expectation is that the runtime does not exit
before all tasks have exited. This is enforced in libgreen through the
`SchedPool` type, and it is enforced in libnative through a `bookkeeping` module
and a global count/mutex pair. Unfortunately, this means that a process which
originates with libgreen will not wait for spawned native tasks.
In order to fix this problem, the bookkeeping module was moved from libnative to
libstd so the runtime itself can wait for native tasks to exit. Green tasks do
not manage themselves through this bookkeeping module, but native tasks will
continue to manage themselves through this module.
Closes#12684
I've been playing around with code size when linking to libstd recently, and these were some findings I found that really helped code size. I started out by eliminating all I/O implementations from libnative and instead just return an unimplemented error.
In doing so, a `fn main() {}` executable was ~378K before this patch, and about 170K after the patch. These size wins are all pretty minor, but they all seemed pretty reasonable to me. With native I/O not stubbed out, this takes the size of an LTO executable from 675K to 400K.
Most of these are unnecessary because we're only looking at static strings. This
also moves to Vec in a few places instead of ~[T].
This didn't end up getting much of a code size win (update_log_settings is the
third largest function in the executables I'm looking at), but this seems like a
generally nice improvement regardless.
There's a lot of these types in the compiler libraries, and a few of the
older or private stdlib ones. Some types are obviously meant to be
public, others not so much.
The printing of the error message on stack overflow had two sometimes false
assumptions previously. The first is that a local task was always available (it
called Local::take) and the second is that it used `println!` instead of
manually writing.
The first assumption isn't necessarily true because while stack overflow will
likely only be detected in situations that a local task is available, it's not
guaranteed to always be in TLS. For example, during a `println!` call a task
may be blocking, causing it to be unavailable. By using Local::try_take(), we
can be resilient against these occurrences.
The second assumption could lead to odd behavior because the stdout logger can
be overwritten to run arbitrary code. Currently this should be possible, but the
utility is much diminished because a stack overflow translates to an abort()
instead of a failure.
The printing of the error message on stack overflow had two sometimes false
assumptions previously. The first is that a local task was always available (it
called Local::take) and the second is that it used println! instead of
manually writing.
The first assumption isn't necessarily true because while stack overflow will
likely only be detected in situations that a local task is available, it's not
guaranteed to always be in TLS. For example, during a println! call a task
may be blocking, causing it to be unavailable. By using Local::try_take(), we
can be resilient against these occurrences.
The second assumption could lead to odd behavior because the stdout logger can
be overwritten to run arbitrary code. Currently this should be possible, but the
utility is much diminished because a stack overflow translates to an abort()
instead of a failure.
Apparently weak linkage and dlopen aren't quite working out for applications
like servo on android. There appears to be a bug or two in how android loads
dynamic libraries and for some reason libservo.so isn't being found.
As a temporary solution, add an extern "C" function to libstd which can be
called if you have a handle to the crate map manually. When crawling the crate
map, we then check this manual symbol before falling back to the old solutions.
cc #11731
The std::run module is a relic from a standard library long since past, and
there's not much use to having two modules to execute processes with where one
is slightly more convenient. This commit merges the two modules, moving lots of
functionality from std::run into std::io::process and then deleting
std::run.
New things you can find in std::io::process are:
* Process::new() now only takes prog/args
* Process::configure() takes a ProcessConfig
* Process::status() is the same as run::process_status
* Process::output() is the same as run::process_output
* I/O for spawned tasks is now defaulted to captured in pipes instead of ignored
* Process::kill() was added (plus an associated green/native implementation)
* Process::wait_with_output() is the same as the old finish_with_output()
* destroy() is now signal_exit()
* force_destroy() is now signal_kill()
Closes#2625Closes#10016
With the stability attributes we can put public-but unstable modules next to others, so this moves `intrinsics` and `raw` out of the `unstable` module (and marks both as `#[experimental]`).
These two containers are indeed collections, so their place is in
libcollections, not in libstd. There will always be a hash map as part of the
standard distribution of Rust, but by moving it out of the standard library it
makes libstd that much more portable to more platforms and environments.
This conveniently also removes the stuttering of 'std::hashmap::HashMap',
although 'collections::HashMap' is only one character shorter.
- adds a `LockGuard` type returned by `.lock` and `.trylock` that unlocks the mutex in the destructor
- renames `mutex::Mutex` to `StaticNativeMutex`
- adds a `NativeMutex` type with a destructor
- removes `LittleLock`
- adds `#[must_use]` to `sync::mutex::Guard` to remind people to use it
Change `os::args()` and `os::env()` to use `str::from_utf8_lossy()`.
Add new functions `os::args_as_bytes()` and `os::env_as_bytes()` to retrieve the args/env as byte vectors instead.
The existing methods were left returning strings because I expect that the common use-case is to want string handling.
Fixes#7188.
os::args() was using str::raw::from_c_str(), which would assert if the
C-string wasn't valid UTF-8. Switch to using from_utf8_lossy() instead,
and add a separate function os::args_as_bytes() that returns the ~[u8]
byte-vectors instead.
This will hopefully bring us closer to #11937. We're still using gcc's idea of
"startup files", but this should prevent us from leaking in dependencies that we
don't quite want (libgcc for example once compiler-rt is what we use).
Any single-threaded task benchmark will spend a good chunk of time in `kqueue()` on osx and `epoll()` on linux, and the reason for this is that each time a task is terminated it will hit the syscall. When a task terminates, it context switches back to the scheduler thread, and the scheduler thread falls out of `run_sched_once` whenever it figures out that it did some work.
If we know that `epoll()` will return nothing, then we can continue to do work locally (only while there's work to be done). We must fall back to `epoll()` whenever there's active I/O in order to check whether it's ready or not, but without that (which is largely the case in benchmarks), we can prevent the costly syscall and can get a nice speedup.
I've separated the commits into preparation for this change and then the change itself, the last commit message has more details.
Instead, use an enum to allow running both a procedure and sending the task
result over a channel. I expect the common case to be sending on a channel (e.g.
task::try), so don't require an extra allocation in the common case.
cc #11389
The green scheduler can optimize its runtime based on this by deciding to not go
to sleep in epoll() if there is no active I/O and there is a task to be stolen.
This is implemented for librustuv by keeping a count of the number of tasks
which are currently homed. If a task is homed, and then performs a blocking I/O
operation, the count will be nonzero while the task is blocked. The homing count
is intentionally 0 when there are I/O handles, but no handles currently blocked.
The reason for this is that epoll() would only be used to wake up the scheduler
anyway.
The crux of this change was to have a `HomingMissile` contain a mutable borrowed
reference back to the `HomeHandle`. The rest of the change was just dealing with
this fallout. This reference is used to decrement the homed handle count in a
HomingMissile's destructor.
Also note that the count maintained is not atomic because all of its
increments/decrements/reads are all on the same I/O thread.
This, the Nth rewrite of channels, is not a rewrite of the core logic behind
channels, but rather their API usage. In the past, we had the distinction
between oneshot, stream, and shared channels, but the most recent rewrite
dropped oneshots in favor of streams and shared channels.
This distinction of stream vs shared has shown that it's not quite what we'd
like either, and this moves the `std::comm` module in the direction of "one
channel to rule them all". There now remains only one Chan and one Port.
This new channel is actually a hybrid oneshot/stream/shared channel under the
hood in order to optimize for the use cases in question. Additionally, this also
reduces the cognitive burden of having to choose between a Chan or a SharedChan
in an API.
My simple benchmarks show no reduction in efficiency over the existing channels
today, and a 3x improvement in the oneshot case. I sadly don't have a
pre-last-rewrite compiler to test out the old old oneshots, but I would imagine
that the performance is comparable, but slightly slower (due to atomic reference
counting).
This commit also brings the bonus bugfix to channels that the pending queue of
messages are all dropped when a Port disappears rather then when both the Port
and the Chan disappear.
