rust/library/std/src/backtrace.rs

505 lines
18 KiB
Rust
Raw Normal View History

//! Support for capturing a stack backtrace of an OS thread
//!
//! This module contains the support necessary to capture a stack backtrace of a
//! running OS thread from the OS thread itself. The `Backtrace` type supports
//! capturing a stack trace via the `Backtrace::capture` and
//! `Backtrace::force_capture` functions.
//!
//! A backtrace is typically quite handy to attach to errors (e.g. types
//! implementing `std::error::Error`) to get a causal chain of where an error
//! was generated.
//!
//! ## Accuracy
//!
//! Backtraces are attempted to be as accurate as possible, but no guarantees
//! are provided about the exact accuracy of a backtrace. Instruction pointers,
//! symbol names, filenames, line numbers, etc, may all be incorrect when
2022-10-04 00:29:29 +08:00
//! reported. Accuracy is attempted on a best-effort basis, however, any bug
//! reports are always welcome to indicate areas of improvement!
//!
//! For most platforms a backtrace with a filename/line number requires that
//! programs be compiled with debug information. Without debug information
//! filenames/line numbers will not be reported.
//!
//! ## Platform support
//!
//! Not all platforms that libstd compiles for support capturing backtraces.
//! Some platforms simply do nothing when capturing a backtrace. To check
//! whether the platform supports capturing backtraces you can consult the
//! `BacktraceStatus` enum as a result of `Backtrace::status`.
//!
//! Like above with accuracy platform support is done on a best effort basis.
//! Sometimes libraries might not be available at runtime or something may go
//! wrong which would cause a backtrace to not be captured. Please feel free to
//! report issues with platforms where a backtrace cannot be captured though!
//!
//! ## Environment Variables
//!
//! The `Backtrace::capture` function might not actually capture a backtrace by
//! default. Its behavior is governed by two environment variables:
//!
//! * `RUST_LIB_BACKTRACE` - if this is set to `0` then `Backtrace::capture`
2022-10-04 00:29:29 +08:00
//! will never capture a backtrace. Any other value set will enable
//! `Backtrace::capture`.
//!
//! * `RUST_BACKTRACE` - if `RUST_LIB_BACKTRACE` is not set, then this variable
//! is consulted with the same rules of `RUST_LIB_BACKTRACE`.
//!
//! * If neither of the above env vars are set, then `Backtrace::capture` will
//! be disabled.
//!
//! Capturing a backtrace can be a quite expensive runtime operation, so the
//! environment variables allow either forcibly disabling this runtime
//! performance hit or allow selectively enabling it in some programs.
//!
//! Note that the `Backtrace::force_capture` function can be used to ignore
//! these environment variables. Also note that the state of environment
//! variables is cached once the first backtrace is created, so altering
//! `RUST_LIB_BACKTRACE` or `RUST_BACKTRACE` at runtime might not actually change
//! how backtraces are captured.
2022-09-19 14:54:16 +02:00
#![stable(feature = "backtrace", since = "1.65.0")]
#[cfg(test)]
mod tests;
// NB: A note on resolution of a backtrace:
//
// Backtraces primarily happen in two steps, one is where we actually capture
// the stack backtrace, giving us a list of instruction pointers corresponding
// to stack frames. Next we take these instruction pointers and, one-by-one,
// turn them into a human readable name (like `main`).
//
// The first phase can be somewhat expensive (walking the stack), especially
// on MSVC where debug information is consulted to return inline frames each as
// their own frame. The second phase, however, is almost always extremely
// expensive (on the order of milliseconds sometimes) when it's consulting debug
// information.
//
// We attempt to amortize this cost as much as possible by delaying resolution
// of an address to a human readable name for as long as possible. When
// `Backtrace::create` is called to capture a backtrace it doesn't actually
// perform any symbol resolution, but rather we lazily resolve symbols only just
// before they're needed for printing. This way we can make capturing a
// backtrace and throwing it away much cheaper, but actually printing a
// backtrace is still basically the same cost.
//
// This strategy comes at the cost of some synchronization required inside of a
// `Backtrace`, but that's a relatively small price to pay relative to capturing
// a backtrace or actually symbolizing it.
std: Switch from libbacktrace to gimli This commit is a proof-of-concept for switching the standard library's backtrace symbolication mechanism on most platforms from libbacktrace to gimli. The standard library's support for `RUST_BACKTRACE=1` requires in-process parsing of object files and DWARF debug information to interpret it and print the filename/line number of stack frames as part of a backtrace. Historically this support in the standard library has come from a library called "libbacktrace". The libbacktrace library seems to have been extracted from gcc at some point and is written in C. We've had a lot of issues with libbacktrace over time, unfortunately, though. The library does not appear to be actively maintained since we've had patches sit for months-to-years without comments. We have discovered a good number of soundness issues with the library itself, both when parsing valid DWARF as well as invalid DWARF. This is enough of an issue that the libs team has previously decided that we cannot feed untrusted inputs to libbacktrace. This also doesn't take into account the portability of libbacktrace which has been difficult to manage and maintain over time. While possible there are lots of exceptions and it's the main C dependency of the standard library right now. For years it's been the desire to switch over to a Rust-based solution for symbolicating backtraces. It's been assumed that we'll be using the Gimli family of crates for this purpose, which are targeted at safely and efficiently parsing DWARF debug information. I've been working recently to shore up the Gimli support in the `backtrace` crate. As of a few weeks ago the `backtrace` crate, by default, uses Gimli when loaded from crates.io. This transition has gone well enough that I figured it was time to start talking seriously about this change to the standard library. This commit is a preview of what's probably the best way to integrate the `backtrace` crate into the standard library with the Gimli feature turned on. While today it's used as a crates.io dependency, this commit switches the `backtrace` crate to a submodule of this repository which will need to be updated manually. This is not done lightly, but is thought to be the best solution. The primary reason for this is that the `backtrace` crate needs to do some pretty nontrivial filesystem interactions to locate debug information. Working without `std::fs` is not an option, and while it might be possible to do some sort of trait-based solution when prototyped it was found to be too unergonomic. Using a submodule allows the `backtrace` crate to build as a submodule of the `std` crate itself, enabling it to use `std::fs` and such. Otherwise this adds new dependencies to the standard library. This step requires extra attention because this means that these crates are now going to be included with all Rust programs by default. It's important to note, however, that we're already shipping libbacktrace with all Rust programs by default and it has a bunch of C code implementing all of this internally anyway, so we're basically already switching already-shipping functionality to Rust from C. * `object` - this crate is used to parse object file headers and contents. Very low-level support is used from this crate and almost all of it is disabled. Largely we're just using struct definitions as well as convenience methods internally to read bytes and such. * `addr2line` - this is the main meat of the implementation for symbolication. This crate depends on `gimli` for DWARF parsing and then provides interfaces needed by the `backtrace` crate to turn an address into a filename / line number. This crate is actually pretty small (fits in a single file almost!) and mirrors most of what `dwarf.c` does for libbacktrace. * `miniz_oxide` - the libbacktrace crate transparently handles compressed debug information which is compressed with zlib. This crate is used to decompress compressed debug sections. * `gimli` - not actually used directly, but a dependency of `addr2line`. * `adler32`- not used directly either, but a dependency of `miniz_oxide`. The goal of this change is to improve the safety of backtrace symbolication in the standard library, especially in the face of possibly malformed DWARF debug information. Even to this day we're still seeing segfaults in libbacktrace which could possibly become security vulnerabilities. This change should almost entirely eliminate this possibility whilc also paving the way forward to adding more features like split debug information. Some references for those interested are: * Original addition of libbacktrace - #12602 * OOM with libbacktrace - #24231 * Backtrace failure due to use of uninitialized value - #28447 * Possibility to feed untrusted data to libbacktrace - #21889 * Soundness fix for libbacktrace - #33729 * Crash in libbacktrace - #39468 * Support for macOS, never merged - ianlancetaylor/libbacktrace#2 * Performance issues with libbacktrace - #29293, #37477 * Update procedure is quite complicated due to how many patches we need to carry - #50955 * Libbacktrace doesn't work on MinGW with dynamic libs - #71060 * Segfault in libbacktrace on macOS - #71397 Switching to Rust will not make us immune to all of these issues. The crashes are expected to go away, but correctness and performance may still have bugs arise. The gimli and `backtrace` crates, however, are actively maintained unlike libbacktrace, so this should enable us to at least efficiently apply fixes as situations come up.
2020-05-13 14:22:37 -07:00
use crate::backtrace_rs::{self, BytesOrWideString};
use crate::cell::UnsafeCell;
use crate::env;
use crate::ffi::c_void;
use crate::fmt;
use crate::sync::atomic::{AtomicUsize, Ordering::Relaxed};
use crate::sync::Once;
use crate::sys_common::backtrace::{lock, output_filename};
use crate::vec::Vec;
/// A captured OS thread stack backtrace.
///
/// This type represents a stack backtrace for an OS thread captured at a
/// previous point in time. In some instances the `Backtrace` type may
/// internally be empty due to configuration. For more information see
/// `Backtrace::capture`.
2022-09-19 14:54:16 +02:00
#[stable(feature = "backtrace", since = "1.65.0")]
#[must_use]
pub struct Backtrace {
inner: Inner,
}
/// The current status of a backtrace, indicating whether it was captured or
/// whether it is empty for some other reason.
2022-09-19 14:54:16 +02:00
#[stable(feature = "backtrace", since = "1.65.0")]
#[non_exhaustive]
#[derive(Debug, PartialEq, Eq)]
pub enum BacktraceStatus {
/// Capturing a backtrace is not supported, likely because it's not
/// implemented for the current platform.
2022-09-19 14:54:16 +02:00
#[stable(feature = "backtrace", since = "1.65.0")]
Unsupported,
/// Capturing a backtrace has been disabled through either the
/// `RUST_LIB_BACKTRACE` or `RUST_BACKTRACE` environment variables.
2022-09-19 14:54:16 +02:00
#[stable(feature = "backtrace", since = "1.65.0")]
Disabled,
/// A backtrace has been captured and the `Backtrace` should print
/// reasonable information when rendered.
2022-09-19 14:54:16 +02:00
#[stable(feature = "backtrace", since = "1.65.0")]
Captured,
}
enum Inner {
Unsupported,
Disabled,
Captured(LazilyResolvedCapture),
}
struct Capture {
actual_start: usize,
resolved: bool,
frames: Vec<BacktraceFrame>,
}
fn _assert_send_sync() {
fn _assert<T: Send + Sync>() {}
_assert::<Backtrace>();
}
2021-01-13 17:54:24 -06:00
/// A single frame of a backtrace.
#[unstable(feature = "backtrace_frames", issue = "79676")]
pub struct BacktraceFrame {
frame: RawFrame,
symbols: Vec<BacktraceSymbol>,
}
2021-01-13 17:54:24 -06:00
#[derive(Debug)]
enum RawFrame {
std: Switch from libbacktrace to gimli This commit is a proof-of-concept for switching the standard library's backtrace symbolication mechanism on most platforms from libbacktrace to gimli. The standard library's support for `RUST_BACKTRACE=1` requires in-process parsing of object files and DWARF debug information to interpret it and print the filename/line number of stack frames as part of a backtrace. Historically this support in the standard library has come from a library called "libbacktrace". The libbacktrace library seems to have been extracted from gcc at some point and is written in C. We've had a lot of issues with libbacktrace over time, unfortunately, though. The library does not appear to be actively maintained since we've had patches sit for months-to-years without comments. We have discovered a good number of soundness issues with the library itself, both when parsing valid DWARF as well as invalid DWARF. This is enough of an issue that the libs team has previously decided that we cannot feed untrusted inputs to libbacktrace. This also doesn't take into account the portability of libbacktrace which has been difficult to manage and maintain over time. While possible there are lots of exceptions and it's the main C dependency of the standard library right now. For years it's been the desire to switch over to a Rust-based solution for symbolicating backtraces. It's been assumed that we'll be using the Gimli family of crates for this purpose, which are targeted at safely and efficiently parsing DWARF debug information. I've been working recently to shore up the Gimli support in the `backtrace` crate. As of a few weeks ago the `backtrace` crate, by default, uses Gimli when loaded from crates.io. This transition has gone well enough that I figured it was time to start talking seriously about this change to the standard library. This commit is a preview of what's probably the best way to integrate the `backtrace` crate into the standard library with the Gimli feature turned on. While today it's used as a crates.io dependency, this commit switches the `backtrace` crate to a submodule of this repository which will need to be updated manually. This is not done lightly, but is thought to be the best solution. The primary reason for this is that the `backtrace` crate needs to do some pretty nontrivial filesystem interactions to locate debug information. Working without `std::fs` is not an option, and while it might be possible to do some sort of trait-based solution when prototyped it was found to be too unergonomic. Using a submodule allows the `backtrace` crate to build as a submodule of the `std` crate itself, enabling it to use `std::fs` and such. Otherwise this adds new dependencies to the standard library. This step requires extra attention because this means that these crates are now going to be included with all Rust programs by default. It's important to note, however, that we're already shipping libbacktrace with all Rust programs by default and it has a bunch of C code implementing all of this internally anyway, so we're basically already switching already-shipping functionality to Rust from C. * `object` - this crate is used to parse object file headers and contents. Very low-level support is used from this crate and almost all of it is disabled. Largely we're just using struct definitions as well as convenience methods internally to read bytes and such. * `addr2line` - this is the main meat of the implementation for symbolication. This crate depends on `gimli` for DWARF parsing and then provides interfaces needed by the `backtrace` crate to turn an address into a filename / line number. This crate is actually pretty small (fits in a single file almost!) and mirrors most of what `dwarf.c` does for libbacktrace. * `miniz_oxide` - the libbacktrace crate transparently handles compressed debug information which is compressed with zlib. This crate is used to decompress compressed debug sections. * `gimli` - not actually used directly, but a dependency of `addr2line`. * `adler32`- not used directly either, but a dependency of `miniz_oxide`. The goal of this change is to improve the safety of backtrace symbolication in the standard library, especially in the face of possibly malformed DWARF debug information. Even to this day we're still seeing segfaults in libbacktrace which could possibly become security vulnerabilities. This change should almost entirely eliminate this possibility whilc also paving the way forward to adding more features like split debug information. Some references for those interested are: * Original addition of libbacktrace - #12602 * OOM with libbacktrace - #24231 * Backtrace failure due to use of uninitialized value - #28447 * Possibility to feed untrusted data to libbacktrace - #21889 * Soundness fix for libbacktrace - #33729 * Crash in libbacktrace - #39468 * Support for macOS, never merged - ianlancetaylor/libbacktrace#2 * Performance issues with libbacktrace - #29293, #37477 * Update procedure is quite complicated due to how many patches we need to carry - #50955 * Libbacktrace doesn't work on MinGW with dynamic libs - #71060 * Segfault in libbacktrace on macOS - #71397 Switching to Rust will not make us immune to all of these issues. The crashes are expected to go away, but correctness and performance may still have bugs arise. The gimli and `backtrace` crates, however, are actively maintained unlike libbacktrace, so this should enable us to at least efficiently apply fixes as situations come up.
2020-05-13 14:22:37 -07:00
Actual(backtrace_rs::Frame),
#[cfg(test)]
Fake,
}
struct BacktraceSymbol {
name: Option<Vec<u8>>,
filename: Option<BytesOrWide>,
lineno: Option<u32>,
colno: Option<u32>,
}
enum BytesOrWide {
Bytes(Vec<u8>),
Wide(Vec<u16>),
}
2022-09-19 14:54:16 +02:00
#[stable(feature = "backtrace", since = "1.65.0")]
2020-02-10 15:15:13 -08:00
impl fmt::Debug for Backtrace {
fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {
let capture = match &self.inner {
Inner::Unsupported => return fmt.write_str("<unsupported>"),
Inner::Disabled => return fmt.write_str("<disabled>"),
Inner::Captured(c) => c.force(),
2020-02-10 15:15:13 -08:00
};
let frames = &capture.frames[capture.actual_start..];
2020-02-10 21:07:36 -08:00
2020-02-10 15:39:32 -08:00
write!(fmt, "Backtrace ")?;
2020-02-10 15:15:13 -08:00
let mut dbg = fmt.debug_list();
2020-02-10 21:07:36 -08:00
for frame in frames {
2020-02-10 17:28:04 -08:00
if frame.frame.ip().is_null() {
continue;
}
2020-02-10 15:15:13 -08:00
dbg.entries(&frame.symbols);
}
dbg.finish()
}
}
2022-07-08 14:21:33 -07:00
#[unstable(feature = "backtrace_frames", issue = "79676")]
2021-01-13 17:54:24 -06:00
impl fmt::Debug for BacktraceFrame {
fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {
let mut dbg = fmt.debug_list();
dbg.entries(&self.symbols);
dbg.finish()
}
}
impl fmt::Debug for BacktraceSymbol {
fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {
// FIXME: improve formatting: https://github.com/rust-lang/rust/issues/65280
// FIXME: Also, include column numbers into the debug format as Display already has them.
// Until there are stable per-frame accessors, the format shouldn't be changed:
// https://github.com/rust-lang/rust/issues/65280#issuecomment-638966585
2020-02-10 17:42:04 -08:00
write!(fmt, "{{ ")?;
2020-02-10 15:39:32 -08:00
std: Switch from libbacktrace to gimli This commit is a proof-of-concept for switching the standard library's backtrace symbolication mechanism on most platforms from libbacktrace to gimli. The standard library's support for `RUST_BACKTRACE=1` requires in-process parsing of object files and DWARF debug information to interpret it and print the filename/line number of stack frames as part of a backtrace. Historically this support in the standard library has come from a library called "libbacktrace". The libbacktrace library seems to have been extracted from gcc at some point and is written in C. We've had a lot of issues with libbacktrace over time, unfortunately, though. The library does not appear to be actively maintained since we've had patches sit for months-to-years without comments. We have discovered a good number of soundness issues with the library itself, both when parsing valid DWARF as well as invalid DWARF. This is enough of an issue that the libs team has previously decided that we cannot feed untrusted inputs to libbacktrace. This also doesn't take into account the portability of libbacktrace which has been difficult to manage and maintain over time. While possible there are lots of exceptions and it's the main C dependency of the standard library right now. For years it's been the desire to switch over to a Rust-based solution for symbolicating backtraces. It's been assumed that we'll be using the Gimli family of crates for this purpose, which are targeted at safely and efficiently parsing DWARF debug information. I've been working recently to shore up the Gimli support in the `backtrace` crate. As of a few weeks ago the `backtrace` crate, by default, uses Gimli when loaded from crates.io. This transition has gone well enough that I figured it was time to start talking seriously about this change to the standard library. This commit is a preview of what's probably the best way to integrate the `backtrace` crate into the standard library with the Gimli feature turned on. While today it's used as a crates.io dependency, this commit switches the `backtrace` crate to a submodule of this repository which will need to be updated manually. This is not done lightly, but is thought to be the best solution. The primary reason for this is that the `backtrace` crate needs to do some pretty nontrivial filesystem interactions to locate debug information. Working without `std::fs` is not an option, and while it might be possible to do some sort of trait-based solution when prototyped it was found to be too unergonomic. Using a submodule allows the `backtrace` crate to build as a submodule of the `std` crate itself, enabling it to use `std::fs` and such. Otherwise this adds new dependencies to the standard library. This step requires extra attention because this means that these crates are now going to be included with all Rust programs by default. It's important to note, however, that we're already shipping libbacktrace with all Rust programs by default and it has a bunch of C code implementing all of this internally anyway, so we're basically already switching already-shipping functionality to Rust from C. * `object` - this crate is used to parse object file headers and contents. Very low-level support is used from this crate and almost all of it is disabled. Largely we're just using struct definitions as well as convenience methods internally to read bytes and such. * `addr2line` - this is the main meat of the implementation for symbolication. This crate depends on `gimli` for DWARF parsing and then provides interfaces needed by the `backtrace` crate to turn an address into a filename / line number. This crate is actually pretty small (fits in a single file almost!) and mirrors most of what `dwarf.c` does for libbacktrace. * `miniz_oxide` - the libbacktrace crate transparently handles compressed debug information which is compressed with zlib. This crate is used to decompress compressed debug sections. * `gimli` - not actually used directly, but a dependency of `addr2line`. * `adler32`- not used directly either, but a dependency of `miniz_oxide`. The goal of this change is to improve the safety of backtrace symbolication in the standard library, especially in the face of possibly malformed DWARF debug information. Even to this day we're still seeing segfaults in libbacktrace which could possibly become security vulnerabilities. This change should almost entirely eliminate this possibility whilc also paving the way forward to adding more features like split debug information. Some references for those interested are: * Original addition of libbacktrace - #12602 * OOM with libbacktrace - #24231 * Backtrace failure due to use of uninitialized value - #28447 * Possibility to feed untrusted data to libbacktrace - #21889 * Soundness fix for libbacktrace - #33729 * Crash in libbacktrace - #39468 * Support for macOS, never merged - ianlancetaylor/libbacktrace#2 * Performance issues with libbacktrace - #29293, #37477 * Update procedure is quite complicated due to how many patches we need to carry - #50955 * Libbacktrace doesn't work on MinGW with dynamic libs - #71060 * Segfault in libbacktrace on macOS - #71397 Switching to Rust will not make us immune to all of these issues. The crashes are expected to go away, but correctness and performance may still have bugs arise. The gimli and `backtrace` crates, however, are actively maintained unlike libbacktrace, so this should enable us to at least efficiently apply fixes as situations come up.
2020-05-13 14:22:37 -07:00
if let Some(fn_name) = self.name.as_ref().map(|b| backtrace_rs::SymbolName::new(b)) {
2020-02-10 21:54:27 -08:00
write!(fmt, "fn: \"{:#}\"", fn_name)?;
2020-02-10 15:39:32 -08:00
} else {
write!(fmt, "fn: <unknown>")?;
2020-02-10 15:39:32 -08:00
}
2020-02-10 15:15:13 -08:00
if let Some(fname) = self.filename.as_ref() {
write!(fmt, ", file: \"{:?}\"", fname)?;
2020-02-10 15:15:13 -08:00
}
if let Some(line) = self.lineno {
2020-02-10 17:42:04 -08:00
write!(fmt, ", line: {:?}", line)?;
2020-02-10 15:15:13 -08:00
}
2020-02-10 17:42:04 -08:00
write!(fmt, " }}")
}
}
2020-02-10 11:47:26 -08:00
impl fmt::Debug for BytesOrWide {
fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {
output_filename(
fmt,
match self {
BytesOrWide::Bytes(w) => BytesOrWideString::Bytes(w),
BytesOrWide::Wide(w) => BytesOrWideString::Wide(w),
},
std: Switch from libbacktrace to gimli This commit is a proof-of-concept for switching the standard library's backtrace symbolication mechanism on most platforms from libbacktrace to gimli. The standard library's support for `RUST_BACKTRACE=1` requires in-process parsing of object files and DWARF debug information to interpret it and print the filename/line number of stack frames as part of a backtrace. Historically this support in the standard library has come from a library called "libbacktrace". The libbacktrace library seems to have been extracted from gcc at some point and is written in C. We've had a lot of issues with libbacktrace over time, unfortunately, though. The library does not appear to be actively maintained since we've had patches sit for months-to-years without comments. We have discovered a good number of soundness issues with the library itself, both when parsing valid DWARF as well as invalid DWARF. This is enough of an issue that the libs team has previously decided that we cannot feed untrusted inputs to libbacktrace. This also doesn't take into account the portability of libbacktrace which has been difficult to manage and maintain over time. While possible there are lots of exceptions and it's the main C dependency of the standard library right now. For years it's been the desire to switch over to a Rust-based solution for symbolicating backtraces. It's been assumed that we'll be using the Gimli family of crates for this purpose, which are targeted at safely and efficiently parsing DWARF debug information. I've been working recently to shore up the Gimli support in the `backtrace` crate. As of a few weeks ago the `backtrace` crate, by default, uses Gimli when loaded from crates.io. This transition has gone well enough that I figured it was time to start talking seriously about this change to the standard library. This commit is a preview of what's probably the best way to integrate the `backtrace` crate into the standard library with the Gimli feature turned on. While today it's used as a crates.io dependency, this commit switches the `backtrace` crate to a submodule of this repository which will need to be updated manually. This is not done lightly, but is thought to be the best solution. The primary reason for this is that the `backtrace` crate needs to do some pretty nontrivial filesystem interactions to locate debug information. Working without `std::fs` is not an option, and while it might be possible to do some sort of trait-based solution when prototyped it was found to be too unergonomic. Using a submodule allows the `backtrace` crate to build as a submodule of the `std` crate itself, enabling it to use `std::fs` and such. Otherwise this adds new dependencies to the standard library. This step requires extra attention because this means that these crates are now going to be included with all Rust programs by default. It's important to note, however, that we're already shipping libbacktrace with all Rust programs by default and it has a bunch of C code implementing all of this internally anyway, so we're basically already switching already-shipping functionality to Rust from C. * `object` - this crate is used to parse object file headers and contents. Very low-level support is used from this crate and almost all of it is disabled. Largely we're just using struct definitions as well as convenience methods internally to read bytes and such. * `addr2line` - this is the main meat of the implementation for symbolication. This crate depends on `gimli` for DWARF parsing and then provides interfaces needed by the `backtrace` crate to turn an address into a filename / line number. This crate is actually pretty small (fits in a single file almost!) and mirrors most of what `dwarf.c` does for libbacktrace. * `miniz_oxide` - the libbacktrace crate transparently handles compressed debug information which is compressed with zlib. This crate is used to decompress compressed debug sections. * `gimli` - not actually used directly, but a dependency of `addr2line`. * `adler32`- not used directly either, but a dependency of `miniz_oxide`. The goal of this change is to improve the safety of backtrace symbolication in the standard library, especially in the face of possibly malformed DWARF debug information. Even to this day we're still seeing segfaults in libbacktrace which could possibly become security vulnerabilities. This change should almost entirely eliminate this possibility whilc also paving the way forward to adding more features like split debug information. Some references for those interested are: * Original addition of libbacktrace - #12602 * OOM with libbacktrace - #24231 * Backtrace failure due to use of uninitialized value - #28447 * Possibility to feed untrusted data to libbacktrace - #21889 * Soundness fix for libbacktrace - #33729 * Crash in libbacktrace - #39468 * Support for macOS, never merged - ianlancetaylor/libbacktrace#2 * Performance issues with libbacktrace - #29293, #37477 * Update procedure is quite complicated due to how many patches we need to carry - #50955 * Libbacktrace doesn't work on MinGW with dynamic libs - #71060 * Segfault in libbacktrace on macOS - #71397 Switching to Rust will not make us immune to all of these issues. The crashes are expected to go away, but correctness and performance may still have bugs arise. The gimli and `backtrace` crates, however, are actively maintained unlike libbacktrace, so this should enable us to at least efficiently apply fixes as situations come up.
2020-05-13 14:22:37 -07:00
backtrace_rs::PrintFmt::Short,
2020-02-10 11:47:26 -08:00
crate::env::current_dir().as_ref().ok(),
)
}
}
impl Backtrace {
/// Returns whether backtrace captures are enabled through environment
/// variables.
fn enabled() -> bool {
// Cache the result of reading the environment variables to make
// backtrace captures speedy, because otherwise reading environment
// variables every time can be somewhat slow.
static ENABLED: AtomicUsize = AtomicUsize::new(0);
match ENABLED.load(Relaxed) {
0 => {}
1 => return false,
_ => return true,
}
let enabled = match env::var("RUST_LIB_BACKTRACE") {
Ok(s) => s != "0",
Err(_) => match env::var("RUST_BACKTRACE") {
Ok(s) => s != "0",
Err(_) => false,
},
};
ENABLED.store(enabled as usize + 1, Relaxed);
enabled
}
/// Capture a stack backtrace of the current thread.
///
/// This function will capture a stack backtrace of the current OS thread of
/// execution, returning a `Backtrace` type which can be later used to print
/// the entire stack trace or render it to a string.
///
/// This function will be a noop if the `RUST_BACKTRACE` or
/// `RUST_LIB_BACKTRACE` backtrace variables are both not set. If either
/// environment variable is set and enabled then this function will actually
/// capture a backtrace. Capturing a backtrace can be both memory intensive
/// and slow, so these environment variables allow liberally using
/// `Backtrace::capture` and only incurring a slowdown when the environment
/// variables are set.
///
/// To forcibly capture a backtrace regardless of environment variables, use
/// the `Backtrace::force_capture` function.
2022-09-19 14:54:16 +02:00
#[stable(feature = "backtrace", since = "1.65.0")]
#[inline(never)] // want to make sure there's a frame here to remove
pub fn capture() -> Backtrace {
if !Backtrace::enabled() {
return Backtrace { inner: Inner::Disabled };
}
Backtrace::create(Backtrace::capture as usize)
}
/// Forcibly captures a full backtrace, regardless of environment variable
/// configuration.
///
/// This function behaves the same as `capture` except that it ignores the
/// values of the `RUST_BACKTRACE` and `RUST_LIB_BACKTRACE` environment
/// variables, always capturing a backtrace.
///
/// Note that capturing a backtrace can be an expensive operation on some
/// platforms, so this should be used with caution in performance-sensitive
/// parts of code.
2022-09-19 14:54:16 +02:00
#[stable(feature = "backtrace", since = "1.65.0")]
#[inline(never)] // want to make sure there's a frame here to remove
pub fn force_capture() -> Backtrace {
Backtrace::create(Backtrace::force_capture as usize)
}
2020-07-18 15:51:44 -07:00
/// Forcibly captures a disabled backtrace, regardless of environment
/// variable configuration.
2022-09-19 14:54:16 +02:00
#[stable(feature = "backtrace", since = "1.65.0")]
#[rustc_const_stable(feature = "backtrace", since = "1.65.0")]
pub const fn disabled() -> Backtrace {
2020-07-18 15:57:57 -07:00
Backtrace { inner: Inner::Disabled }
2020-07-18 15:51:44 -07:00
}
// Capture a backtrace which start just before the function addressed by
// `ip`
fn create(ip: usize) -> Backtrace {
let _lock = lock();
let mut frames = Vec::new();
let mut actual_start = None;
unsafe {
std: Switch from libbacktrace to gimli This commit is a proof-of-concept for switching the standard library's backtrace symbolication mechanism on most platforms from libbacktrace to gimli. The standard library's support for `RUST_BACKTRACE=1` requires in-process parsing of object files and DWARF debug information to interpret it and print the filename/line number of stack frames as part of a backtrace. Historically this support in the standard library has come from a library called "libbacktrace". The libbacktrace library seems to have been extracted from gcc at some point and is written in C. We've had a lot of issues with libbacktrace over time, unfortunately, though. The library does not appear to be actively maintained since we've had patches sit for months-to-years without comments. We have discovered a good number of soundness issues with the library itself, both when parsing valid DWARF as well as invalid DWARF. This is enough of an issue that the libs team has previously decided that we cannot feed untrusted inputs to libbacktrace. This also doesn't take into account the portability of libbacktrace which has been difficult to manage and maintain over time. While possible there are lots of exceptions and it's the main C dependency of the standard library right now. For years it's been the desire to switch over to a Rust-based solution for symbolicating backtraces. It's been assumed that we'll be using the Gimli family of crates for this purpose, which are targeted at safely and efficiently parsing DWARF debug information. I've been working recently to shore up the Gimli support in the `backtrace` crate. As of a few weeks ago the `backtrace` crate, by default, uses Gimli when loaded from crates.io. This transition has gone well enough that I figured it was time to start talking seriously about this change to the standard library. This commit is a preview of what's probably the best way to integrate the `backtrace` crate into the standard library with the Gimli feature turned on. While today it's used as a crates.io dependency, this commit switches the `backtrace` crate to a submodule of this repository which will need to be updated manually. This is not done lightly, but is thought to be the best solution. The primary reason for this is that the `backtrace` crate needs to do some pretty nontrivial filesystem interactions to locate debug information. Working without `std::fs` is not an option, and while it might be possible to do some sort of trait-based solution when prototyped it was found to be too unergonomic. Using a submodule allows the `backtrace` crate to build as a submodule of the `std` crate itself, enabling it to use `std::fs` and such. Otherwise this adds new dependencies to the standard library. This step requires extra attention because this means that these crates are now going to be included with all Rust programs by default. It's important to note, however, that we're already shipping libbacktrace with all Rust programs by default and it has a bunch of C code implementing all of this internally anyway, so we're basically already switching already-shipping functionality to Rust from C. * `object` - this crate is used to parse object file headers and contents. Very low-level support is used from this crate and almost all of it is disabled. Largely we're just using struct definitions as well as convenience methods internally to read bytes and such. * `addr2line` - this is the main meat of the implementation for symbolication. This crate depends on `gimli` for DWARF parsing and then provides interfaces needed by the `backtrace` crate to turn an address into a filename / line number. This crate is actually pretty small (fits in a single file almost!) and mirrors most of what `dwarf.c` does for libbacktrace. * `miniz_oxide` - the libbacktrace crate transparently handles compressed debug information which is compressed with zlib. This crate is used to decompress compressed debug sections. * `gimli` - not actually used directly, but a dependency of `addr2line`. * `adler32`- not used directly either, but a dependency of `miniz_oxide`. The goal of this change is to improve the safety of backtrace symbolication in the standard library, especially in the face of possibly malformed DWARF debug information. Even to this day we're still seeing segfaults in libbacktrace which could possibly become security vulnerabilities. This change should almost entirely eliminate this possibility whilc also paving the way forward to adding more features like split debug information. Some references for those interested are: * Original addition of libbacktrace - #12602 * OOM with libbacktrace - #24231 * Backtrace failure due to use of uninitialized value - #28447 * Possibility to feed untrusted data to libbacktrace - #21889 * Soundness fix for libbacktrace - #33729 * Crash in libbacktrace - #39468 * Support for macOS, never merged - ianlancetaylor/libbacktrace#2 * Performance issues with libbacktrace - #29293, #37477 * Update procedure is quite complicated due to how many patches we need to carry - #50955 * Libbacktrace doesn't work on MinGW with dynamic libs - #71060 * Segfault in libbacktrace on macOS - #71397 Switching to Rust will not make us immune to all of these issues. The crashes are expected to go away, but correctness and performance may still have bugs arise. The gimli and `backtrace` crates, however, are actively maintained unlike libbacktrace, so this should enable us to at least efficiently apply fixes as situations come up.
2020-05-13 14:22:37 -07:00
backtrace_rs::trace_unsynchronized(|frame| {
frames.push(BacktraceFrame {
frame: RawFrame::Actual(frame.clone()),
symbols: Vec::new(),
});
if frame.symbol_address().addr() == ip && actual_start.is_none() {
actual_start = Some(frames.len());
}
true
});
}
// If no frames came out assume that this is an unsupported platform
// since `backtrace` doesn't provide a way of learning this right now,
// and this should be a good enough approximation.
let inner = if frames.is_empty() {
Inner::Unsupported
} else {
Inner::Captured(LazilyResolvedCapture::new(Capture {
actual_start: actual_start.unwrap_or(0),
frames,
resolved: false,
}))
};
Backtrace { inner }
}
/// Returns the status of this backtrace, indicating whether this backtrace
/// request was unsupported, disabled, or a stack trace was actually
/// captured.
2022-09-19 14:54:16 +02:00
#[stable(feature = "backtrace", since = "1.65.0")]
#[must_use]
pub fn status(&self) -> BacktraceStatus {
match self.inner {
Inner::Unsupported => BacktraceStatus::Unsupported,
Inner::Disabled => BacktraceStatus::Disabled,
Inner::Captured(_) => BacktraceStatus::Captured,
}
}
}
2021-01-13 17:54:24 -06:00
impl<'a> Backtrace {
/// Returns an iterator over the backtrace frames.
#[must_use]
2021-01-13 17:54:24 -06:00
#[unstable(feature = "backtrace_frames", issue = "79676")]
pub fn frames(&'a self) -> &'a [BacktraceFrame] {
if let Inner::Captured(c) = &self.inner { &c.force().frames } else { &[] }
}
}
2022-09-19 14:54:16 +02:00
#[stable(feature = "backtrace", since = "1.65.0")]
impl fmt::Display for Backtrace {
fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {
let capture = match &self.inner {
Inner::Unsupported => return fmt.write_str("unsupported backtrace"),
Inner::Disabled => return fmt.write_str("disabled backtrace"),
Inner::Captured(c) => c.force(),
};
let full = fmt.alternate();
let (frames, style) = if full {
std: Switch from libbacktrace to gimli This commit is a proof-of-concept for switching the standard library's backtrace symbolication mechanism on most platforms from libbacktrace to gimli. The standard library's support for `RUST_BACKTRACE=1` requires in-process parsing of object files and DWARF debug information to interpret it and print the filename/line number of stack frames as part of a backtrace. Historically this support in the standard library has come from a library called "libbacktrace". The libbacktrace library seems to have been extracted from gcc at some point and is written in C. We've had a lot of issues with libbacktrace over time, unfortunately, though. The library does not appear to be actively maintained since we've had patches sit for months-to-years without comments. We have discovered a good number of soundness issues with the library itself, both when parsing valid DWARF as well as invalid DWARF. This is enough of an issue that the libs team has previously decided that we cannot feed untrusted inputs to libbacktrace. This also doesn't take into account the portability of libbacktrace which has been difficult to manage and maintain over time. While possible there are lots of exceptions and it's the main C dependency of the standard library right now. For years it's been the desire to switch over to a Rust-based solution for symbolicating backtraces. It's been assumed that we'll be using the Gimli family of crates for this purpose, which are targeted at safely and efficiently parsing DWARF debug information. I've been working recently to shore up the Gimli support in the `backtrace` crate. As of a few weeks ago the `backtrace` crate, by default, uses Gimli when loaded from crates.io. This transition has gone well enough that I figured it was time to start talking seriously about this change to the standard library. This commit is a preview of what's probably the best way to integrate the `backtrace` crate into the standard library with the Gimli feature turned on. While today it's used as a crates.io dependency, this commit switches the `backtrace` crate to a submodule of this repository which will need to be updated manually. This is not done lightly, but is thought to be the best solution. The primary reason for this is that the `backtrace` crate needs to do some pretty nontrivial filesystem interactions to locate debug information. Working without `std::fs` is not an option, and while it might be possible to do some sort of trait-based solution when prototyped it was found to be too unergonomic. Using a submodule allows the `backtrace` crate to build as a submodule of the `std` crate itself, enabling it to use `std::fs` and such. Otherwise this adds new dependencies to the standard library. This step requires extra attention because this means that these crates are now going to be included with all Rust programs by default. It's important to note, however, that we're already shipping libbacktrace with all Rust programs by default and it has a bunch of C code implementing all of this internally anyway, so we're basically already switching already-shipping functionality to Rust from C. * `object` - this crate is used to parse object file headers and contents. Very low-level support is used from this crate and almost all of it is disabled. Largely we're just using struct definitions as well as convenience methods internally to read bytes and such. * `addr2line` - this is the main meat of the implementation for symbolication. This crate depends on `gimli` for DWARF parsing and then provides interfaces needed by the `backtrace` crate to turn an address into a filename / line number. This crate is actually pretty small (fits in a single file almost!) and mirrors most of what `dwarf.c` does for libbacktrace. * `miniz_oxide` - the libbacktrace crate transparently handles compressed debug information which is compressed with zlib. This crate is used to decompress compressed debug sections. * `gimli` - not actually used directly, but a dependency of `addr2line`. * `adler32`- not used directly either, but a dependency of `miniz_oxide`. The goal of this change is to improve the safety of backtrace symbolication in the standard library, especially in the face of possibly malformed DWARF debug information. Even to this day we're still seeing segfaults in libbacktrace which could possibly become security vulnerabilities. This change should almost entirely eliminate this possibility whilc also paving the way forward to adding more features like split debug information. Some references for those interested are: * Original addition of libbacktrace - #12602 * OOM with libbacktrace - #24231 * Backtrace failure due to use of uninitialized value - #28447 * Possibility to feed untrusted data to libbacktrace - #21889 * Soundness fix for libbacktrace - #33729 * Crash in libbacktrace - #39468 * Support for macOS, never merged - ianlancetaylor/libbacktrace#2 * Performance issues with libbacktrace - #29293, #37477 * Update procedure is quite complicated due to how many patches we need to carry - #50955 * Libbacktrace doesn't work on MinGW with dynamic libs - #71060 * Segfault in libbacktrace on macOS - #71397 Switching to Rust will not make us immune to all of these issues. The crashes are expected to go away, but correctness and performance may still have bugs arise. The gimli and `backtrace` crates, however, are actively maintained unlike libbacktrace, so this should enable us to at least efficiently apply fixes as situations come up.
2020-05-13 14:22:37 -07:00
(&capture.frames[..], backtrace_rs::PrintFmt::Full)
} else {
std: Switch from libbacktrace to gimli This commit is a proof-of-concept for switching the standard library's backtrace symbolication mechanism on most platforms from libbacktrace to gimli. The standard library's support for `RUST_BACKTRACE=1` requires in-process parsing of object files and DWARF debug information to interpret it and print the filename/line number of stack frames as part of a backtrace. Historically this support in the standard library has come from a library called "libbacktrace". The libbacktrace library seems to have been extracted from gcc at some point and is written in C. We've had a lot of issues with libbacktrace over time, unfortunately, though. The library does not appear to be actively maintained since we've had patches sit for months-to-years without comments. We have discovered a good number of soundness issues with the library itself, both when parsing valid DWARF as well as invalid DWARF. This is enough of an issue that the libs team has previously decided that we cannot feed untrusted inputs to libbacktrace. This also doesn't take into account the portability of libbacktrace which has been difficult to manage and maintain over time. While possible there are lots of exceptions and it's the main C dependency of the standard library right now. For years it's been the desire to switch over to a Rust-based solution for symbolicating backtraces. It's been assumed that we'll be using the Gimli family of crates for this purpose, which are targeted at safely and efficiently parsing DWARF debug information. I've been working recently to shore up the Gimli support in the `backtrace` crate. As of a few weeks ago the `backtrace` crate, by default, uses Gimli when loaded from crates.io. This transition has gone well enough that I figured it was time to start talking seriously about this change to the standard library. This commit is a preview of what's probably the best way to integrate the `backtrace` crate into the standard library with the Gimli feature turned on. While today it's used as a crates.io dependency, this commit switches the `backtrace` crate to a submodule of this repository which will need to be updated manually. This is not done lightly, but is thought to be the best solution. The primary reason for this is that the `backtrace` crate needs to do some pretty nontrivial filesystem interactions to locate debug information. Working without `std::fs` is not an option, and while it might be possible to do some sort of trait-based solution when prototyped it was found to be too unergonomic. Using a submodule allows the `backtrace` crate to build as a submodule of the `std` crate itself, enabling it to use `std::fs` and such. Otherwise this adds new dependencies to the standard library. This step requires extra attention because this means that these crates are now going to be included with all Rust programs by default. It's important to note, however, that we're already shipping libbacktrace with all Rust programs by default and it has a bunch of C code implementing all of this internally anyway, so we're basically already switching already-shipping functionality to Rust from C. * `object` - this crate is used to parse object file headers and contents. Very low-level support is used from this crate and almost all of it is disabled. Largely we're just using struct definitions as well as convenience methods internally to read bytes and such. * `addr2line` - this is the main meat of the implementation for symbolication. This crate depends on `gimli` for DWARF parsing and then provides interfaces needed by the `backtrace` crate to turn an address into a filename / line number. This crate is actually pretty small (fits in a single file almost!) and mirrors most of what `dwarf.c` does for libbacktrace. * `miniz_oxide` - the libbacktrace crate transparently handles compressed debug information which is compressed with zlib. This crate is used to decompress compressed debug sections. * `gimli` - not actually used directly, but a dependency of `addr2line`. * `adler32`- not used directly either, but a dependency of `miniz_oxide`. The goal of this change is to improve the safety of backtrace symbolication in the standard library, especially in the face of possibly malformed DWARF debug information. Even to this day we're still seeing segfaults in libbacktrace which could possibly become security vulnerabilities. This change should almost entirely eliminate this possibility whilc also paving the way forward to adding more features like split debug information. Some references for those interested are: * Original addition of libbacktrace - #12602 * OOM with libbacktrace - #24231 * Backtrace failure due to use of uninitialized value - #28447 * Possibility to feed untrusted data to libbacktrace - #21889 * Soundness fix for libbacktrace - #33729 * Crash in libbacktrace - #39468 * Support for macOS, never merged - ianlancetaylor/libbacktrace#2 * Performance issues with libbacktrace - #29293, #37477 * Update procedure is quite complicated due to how many patches we need to carry - #50955 * Libbacktrace doesn't work on MinGW with dynamic libs - #71060 * Segfault in libbacktrace on macOS - #71397 Switching to Rust will not make us immune to all of these issues. The crashes are expected to go away, but correctness and performance may still have bugs arise. The gimli and `backtrace` crates, however, are actively maintained unlike libbacktrace, so this should enable us to at least efficiently apply fixes as situations come up.
2020-05-13 14:22:37 -07:00
(&capture.frames[capture.actual_start..], backtrace_rs::PrintFmt::Short)
};
// When printing paths we try to strip the cwd if it exists, otherwise
// we just print the path as-is. Note that we also only do this for the
// short format, because if it's full we presumably want to print
// everything.
let cwd = crate::env::current_dir();
let mut print_path = move |fmt: &mut fmt::Formatter<'_>, path: BytesOrWideString<'_>| {
output_filename(fmt, path, style, cwd.as_ref().ok())
};
std: Switch from libbacktrace to gimli This commit is a proof-of-concept for switching the standard library's backtrace symbolication mechanism on most platforms from libbacktrace to gimli. The standard library's support for `RUST_BACKTRACE=1` requires in-process parsing of object files and DWARF debug information to interpret it and print the filename/line number of stack frames as part of a backtrace. Historically this support in the standard library has come from a library called "libbacktrace". The libbacktrace library seems to have been extracted from gcc at some point and is written in C. We've had a lot of issues with libbacktrace over time, unfortunately, though. The library does not appear to be actively maintained since we've had patches sit for months-to-years without comments. We have discovered a good number of soundness issues with the library itself, both when parsing valid DWARF as well as invalid DWARF. This is enough of an issue that the libs team has previously decided that we cannot feed untrusted inputs to libbacktrace. This also doesn't take into account the portability of libbacktrace which has been difficult to manage and maintain over time. While possible there are lots of exceptions and it's the main C dependency of the standard library right now. For years it's been the desire to switch over to a Rust-based solution for symbolicating backtraces. It's been assumed that we'll be using the Gimli family of crates for this purpose, which are targeted at safely and efficiently parsing DWARF debug information. I've been working recently to shore up the Gimli support in the `backtrace` crate. As of a few weeks ago the `backtrace` crate, by default, uses Gimli when loaded from crates.io. This transition has gone well enough that I figured it was time to start talking seriously about this change to the standard library. This commit is a preview of what's probably the best way to integrate the `backtrace` crate into the standard library with the Gimli feature turned on. While today it's used as a crates.io dependency, this commit switches the `backtrace` crate to a submodule of this repository which will need to be updated manually. This is not done lightly, but is thought to be the best solution. The primary reason for this is that the `backtrace` crate needs to do some pretty nontrivial filesystem interactions to locate debug information. Working without `std::fs` is not an option, and while it might be possible to do some sort of trait-based solution when prototyped it was found to be too unergonomic. Using a submodule allows the `backtrace` crate to build as a submodule of the `std` crate itself, enabling it to use `std::fs` and such. Otherwise this adds new dependencies to the standard library. This step requires extra attention because this means that these crates are now going to be included with all Rust programs by default. It's important to note, however, that we're already shipping libbacktrace with all Rust programs by default and it has a bunch of C code implementing all of this internally anyway, so we're basically already switching already-shipping functionality to Rust from C. * `object` - this crate is used to parse object file headers and contents. Very low-level support is used from this crate and almost all of it is disabled. Largely we're just using struct definitions as well as convenience methods internally to read bytes and such. * `addr2line` - this is the main meat of the implementation for symbolication. This crate depends on `gimli` for DWARF parsing and then provides interfaces needed by the `backtrace` crate to turn an address into a filename / line number. This crate is actually pretty small (fits in a single file almost!) and mirrors most of what `dwarf.c` does for libbacktrace. * `miniz_oxide` - the libbacktrace crate transparently handles compressed debug information which is compressed with zlib. This crate is used to decompress compressed debug sections. * `gimli` - not actually used directly, but a dependency of `addr2line`. * `adler32`- not used directly either, but a dependency of `miniz_oxide`. The goal of this change is to improve the safety of backtrace symbolication in the standard library, especially in the face of possibly malformed DWARF debug information. Even to this day we're still seeing segfaults in libbacktrace which could possibly become security vulnerabilities. This change should almost entirely eliminate this possibility whilc also paving the way forward to adding more features like split debug information. Some references for those interested are: * Original addition of libbacktrace - #12602 * OOM with libbacktrace - #24231 * Backtrace failure due to use of uninitialized value - #28447 * Possibility to feed untrusted data to libbacktrace - #21889 * Soundness fix for libbacktrace - #33729 * Crash in libbacktrace - #39468 * Support for macOS, never merged - ianlancetaylor/libbacktrace#2 * Performance issues with libbacktrace - #29293, #37477 * Update procedure is quite complicated due to how many patches we need to carry - #50955 * Libbacktrace doesn't work on MinGW with dynamic libs - #71060 * Segfault in libbacktrace on macOS - #71397 Switching to Rust will not make us immune to all of these issues. The crashes are expected to go away, but correctness and performance may still have bugs arise. The gimli and `backtrace` crates, however, are actively maintained unlike libbacktrace, so this should enable us to at least efficiently apply fixes as situations come up.
2020-05-13 14:22:37 -07:00
let mut f = backtrace_rs::BacktraceFmt::new(fmt, style, &mut print_path);
f.add_context()?;
for frame in frames {
if frame.symbols.is_empty() {
f.frame().print_raw(frame.frame.ip(), None, None, None)?;
} else {
for symbol in frame.symbols.iter() {
f.frame().print_raw_with_column(
frame.frame.ip(),
std: Switch from libbacktrace to gimli This commit is a proof-of-concept for switching the standard library's backtrace symbolication mechanism on most platforms from libbacktrace to gimli. The standard library's support for `RUST_BACKTRACE=1` requires in-process parsing of object files and DWARF debug information to interpret it and print the filename/line number of stack frames as part of a backtrace. Historically this support in the standard library has come from a library called "libbacktrace". The libbacktrace library seems to have been extracted from gcc at some point and is written in C. We've had a lot of issues with libbacktrace over time, unfortunately, though. The library does not appear to be actively maintained since we've had patches sit for months-to-years without comments. We have discovered a good number of soundness issues with the library itself, both when parsing valid DWARF as well as invalid DWARF. This is enough of an issue that the libs team has previously decided that we cannot feed untrusted inputs to libbacktrace. This also doesn't take into account the portability of libbacktrace which has been difficult to manage and maintain over time. While possible there are lots of exceptions and it's the main C dependency of the standard library right now. For years it's been the desire to switch over to a Rust-based solution for symbolicating backtraces. It's been assumed that we'll be using the Gimli family of crates for this purpose, which are targeted at safely and efficiently parsing DWARF debug information. I've been working recently to shore up the Gimli support in the `backtrace` crate. As of a few weeks ago the `backtrace` crate, by default, uses Gimli when loaded from crates.io. This transition has gone well enough that I figured it was time to start talking seriously about this change to the standard library. This commit is a preview of what's probably the best way to integrate the `backtrace` crate into the standard library with the Gimli feature turned on. While today it's used as a crates.io dependency, this commit switches the `backtrace` crate to a submodule of this repository which will need to be updated manually. This is not done lightly, but is thought to be the best solution. The primary reason for this is that the `backtrace` crate needs to do some pretty nontrivial filesystem interactions to locate debug information. Working without `std::fs` is not an option, and while it might be possible to do some sort of trait-based solution when prototyped it was found to be too unergonomic. Using a submodule allows the `backtrace` crate to build as a submodule of the `std` crate itself, enabling it to use `std::fs` and such. Otherwise this adds new dependencies to the standard library. This step requires extra attention because this means that these crates are now going to be included with all Rust programs by default. It's important to note, however, that we're already shipping libbacktrace with all Rust programs by default and it has a bunch of C code implementing all of this internally anyway, so we're basically already switching already-shipping functionality to Rust from C. * `object` - this crate is used to parse object file headers and contents. Very low-level support is used from this crate and almost all of it is disabled. Largely we're just using struct definitions as well as convenience methods internally to read bytes and such. * `addr2line` - this is the main meat of the implementation for symbolication. This crate depends on `gimli` for DWARF parsing and then provides interfaces needed by the `backtrace` crate to turn an address into a filename / line number. This crate is actually pretty small (fits in a single file almost!) and mirrors most of what `dwarf.c` does for libbacktrace. * `miniz_oxide` - the libbacktrace crate transparently handles compressed debug information which is compressed with zlib. This crate is used to decompress compressed debug sections. * `gimli` - not actually used directly, but a dependency of `addr2line`. * `adler32`- not used directly either, but a dependency of `miniz_oxide`. The goal of this change is to improve the safety of backtrace symbolication in the standard library, especially in the face of possibly malformed DWARF debug information. Even to this day we're still seeing segfaults in libbacktrace which could possibly become security vulnerabilities. This change should almost entirely eliminate this possibility whilc also paving the way forward to adding more features like split debug information. Some references for those interested are: * Original addition of libbacktrace - #12602 * OOM with libbacktrace - #24231 * Backtrace failure due to use of uninitialized value - #28447 * Possibility to feed untrusted data to libbacktrace - #21889 * Soundness fix for libbacktrace - #33729 * Crash in libbacktrace - #39468 * Support for macOS, never merged - ianlancetaylor/libbacktrace#2 * Performance issues with libbacktrace - #29293, #37477 * Update procedure is quite complicated due to how many patches we need to carry - #50955 * Libbacktrace doesn't work on MinGW with dynamic libs - #71060 * Segfault in libbacktrace on macOS - #71397 Switching to Rust will not make us immune to all of these issues. The crashes are expected to go away, but correctness and performance may still have bugs arise. The gimli and `backtrace` crates, however, are actively maintained unlike libbacktrace, so this should enable us to at least efficiently apply fixes as situations come up.
2020-05-13 14:22:37 -07:00
symbol.name.as_ref().map(|b| backtrace_rs::SymbolName::new(b)),
symbol.filename.as_ref().map(|b| match b {
BytesOrWide::Bytes(w) => BytesOrWideString::Bytes(w),
BytesOrWide::Wide(w) => BytesOrWideString::Wide(w),
}),
symbol.lineno,
symbol.colno,
)?;
}
}
}
f.finish()?;
Ok(())
}
}
struct LazilyResolvedCapture {
sync: Once,
capture: UnsafeCell<Capture>,
}
impl LazilyResolvedCapture {
fn new(capture: Capture) -> Self {
LazilyResolvedCapture { sync: Once::new(), capture: UnsafeCell::new(capture) }
}
fn force(&self) -> &Capture {
self.sync.call_once(|| {
// SAFETY: This exclusive reference can't overlap with any others
// `Once` guarantees callers will block until this closure returns
// `Once` also guarantees only a single caller will enter this closure
unsafe { &mut *self.capture.get() }.resolve();
});
// SAFETY: This shared reference can't overlap with the exclusive reference above
unsafe { &*self.capture.get() }
}
}
// SAFETY: Access to the inner value is synchronized using a thread-safe `Once`
// So long as `Capture` is `Sync`, `LazilyResolvedCapture` is too
unsafe impl Sync for LazilyResolvedCapture where Capture: Sync {}
impl Capture {
fn resolve(&mut self) {
// If we're already resolved, nothing to do!
if self.resolved {
return;
}
self.resolved = true;
// Use the global backtrace lock to synchronize this as it's a
// requirement of the `backtrace` crate, and then actually resolve
// everything.
let _lock = lock();
for frame in self.frames.iter_mut() {
let symbols = &mut frame.symbols;
let frame = match &frame.frame {
RawFrame::Actual(frame) => frame,
#[cfg(test)]
RawFrame::Fake => unimplemented!(),
};
unsafe {
std: Switch from libbacktrace to gimli This commit is a proof-of-concept for switching the standard library's backtrace symbolication mechanism on most platforms from libbacktrace to gimli. The standard library's support for `RUST_BACKTRACE=1` requires in-process parsing of object files and DWARF debug information to interpret it and print the filename/line number of stack frames as part of a backtrace. Historically this support in the standard library has come from a library called "libbacktrace". The libbacktrace library seems to have been extracted from gcc at some point and is written in C. We've had a lot of issues with libbacktrace over time, unfortunately, though. The library does not appear to be actively maintained since we've had patches sit for months-to-years without comments. We have discovered a good number of soundness issues with the library itself, both when parsing valid DWARF as well as invalid DWARF. This is enough of an issue that the libs team has previously decided that we cannot feed untrusted inputs to libbacktrace. This also doesn't take into account the portability of libbacktrace which has been difficult to manage and maintain over time. While possible there are lots of exceptions and it's the main C dependency of the standard library right now. For years it's been the desire to switch over to a Rust-based solution for symbolicating backtraces. It's been assumed that we'll be using the Gimli family of crates for this purpose, which are targeted at safely and efficiently parsing DWARF debug information. I've been working recently to shore up the Gimli support in the `backtrace` crate. As of a few weeks ago the `backtrace` crate, by default, uses Gimli when loaded from crates.io. This transition has gone well enough that I figured it was time to start talking seriously about this change to the standard library. This commit is a preview of what's probably the best way to integrate the `backtrace` crate into the standard library with the Gimli feature turned on. While today it's used as a crates.io dependency, this commit switches the `backtrace` crate to a submodule of this repository which will need to be updated manually. This is not done lightly, but is thought to be the best solution. The primary reason for this is that the `backtrace` crate needs to do some pretty nontrivial filesystem interactions to locate debug information. Working without `std::fs` is not an option, and while it might be possible to do some sort of trait-based solution when prototyped it was found to be too unergonomic. Using a submodule allows the `backtrace` crate to build as a submodule of the `std` crate itself, enabling it to use `std::fs` and such. Otherwise this adds new dependencies to the standard library. This step requires extra attention because this means that these crates are now going to be included with all Rust programs by default. It's important to note, however, that we're already shipping libbacktrace with all Rust programs by default and it has a bunch of C code implementing all of this internally anyway, so we're basically already switching already-shipping functionality to Rust from C. * `object` - this crate is used to parse object file headers and contents. Very low-level support is used from this crate and almost all of it is disabled. Largely we're just using struct definitions as well as convenience methods internally to read bytes and such. * `addr2line` - this is the main meat of the implementation for symbolication. This crate depends on `gimli` for DWARF parsing and then provides interfaces needed by the `backtrace` crate to turn an address into a filename / line number. This crate is actually pretty small (fits in a single file almost!) and mirrors most of what `dwarf.c` does for libbacktrace. * `miniz_oxide` - the libbacktrace crate transparently handles compressed debug information which is compressed with zlib. This crate is used to decompress compressed debug sections. * `gimli` - not actually used directly, but a dependency of `addr2line`. * `adler32`- not used directly either, but a dependency of `miniz_oxide`. The goal of this change is to improve the safety of backtrace symbolication in the standard library, especially in the face of possibly malformed DWARF debug information. Even to this day we're still seeing segfaults in libbacktrace which could possibly become security vulnerabilities. This change should almost entirely eliminate this possibility whilc also paving the way forward to adding more features like split debug information. Some references for those interested are: * Original addition of libbacktrace - #12602 * OOM with libbacktrace - #24231 * Backtrace failure due to use of uninitialized value - #28447 * Possibility to feed untrusted data to libbacktrace - #21889 * Soundness fix for libbacktrace - #33729 * Crash in libbacktrace - #39468 * Support for macOS, never merged - ianlancetaylor/libbacktrace#2 * Performance issues with libbacktrace - #29293, #37477 * Update procedure is quite complicated due to how many patches we need to carry - #50955 * Libbacktrace doesn't work on MinGW with dynamic libs - #71060 * Segfault in libbacktrace on macOS - #71397 Switching to Rust will not make us immune to all of these issues. The crashes are expected to go away, but correctness and performance may still have bugs arise. The gimli and `backtrace` crates, however, are actively maintained unlike libbacktrace, so this should enable us to at least efficiently apply fixes as situations come up.
2020-05-13 14:22:37 -07:00
backtrace_rs::resolve_frame_unsynchronized(frame, |symbol| {
symbols.push(BacktraceSymbol {
name: symbol.name().map(|m| m.as_bytes().to_vec()),
filename: symbol.filename_raw().map(|b| match b {
BytesOrWideString::Bytes(b) => BytesOrWide::Bytes(b.to_owned()),
BytesOrWideString::Wide(b) => BytesOrWide::Wide(b.to_owned()),
}),
lineno: symbol.lineno(),
colno: symbol.colno(),
});
});
}
}
}
}
impl RawFrame {
fn ip(&self) -> *mut c_void {
match self {
RawFrame::Actual(frame) => frame.ip(),
#[cfg(test)]
RawFrame::Fake => crate::ptr::invalid_mut(1),
}
}
}