When using bash-specific features, scripts using env to call bash
are more convenient, as bash be installed in different places
according the OS.
Same applies for other languages' interpreters.
Replace most call to grep in run-make by a script that cat the input.
Introduced a new `src/etc/cat-and-grep.sh` script (called in run-make as `$(CGREP)`), which prints the input and do a grep simultaneously. This is mainly used to debug spurious failures in run-make, such as the spurious error in #45810, as well as real errors such as #46126.
(cc #40713)
Some `grep` still remains, mainly the `grep -c` calls that count the number of matches and print the result to stdout.
This commit allocates a builder to running wasm32 tests on Travis. Not all test
suites pass right now so this is starting out with just the run-pass and the
libcore test suites. This'll hopefully give us a pretty broad set of coverage
for integration in rustc itself as well as a somewhat broad coverage of the llvm
backend itself through integration/unit tests.
Introduced a new src/etc/cat-and-grep.sh script (called in run-make as
$(CGREP)), which prints the input and do a grep simultaneously. This is
mainly used to debug spurious failures in run-make, such as the sanitizer
error in #45810, as well as real errors such as #46126.
std: Add a new wasm32-unknown-unknown target
This commit adds a new target to the compiler: wasm32-unknown-unknown. This target is a reimagining of what it looks like to generate WebAssembly code from Rust. Instead of using Emscripten which can bring with it a weighty runtime this instead is a target which uses only the LLVM backend for WebAssembly and a "custom linker" for now which will hopefully one day be direct calls to lld.
Notable features of this target include:
* There is zero runtime footprint. The target assumes nothing exists other than the wasm32 instruction set.
* There is zero toolchain footprint beyond adding the target. No custom linker is needed, rustc contains everything.
* Very small wasm modules can be generated directly from Rust code using this target.
* Most of the standard library is stubbed out to return an error, but anything related to allocation works (aka `HashMap`, `Vec`, etc).
* Naturally, any `#[no_std]` crate should be 100% compatible with this new target.
This target is currently somewhat janky due to how linking works. The "linking" is currently unconditional whole program LTO (aka LLVM is being used as a linker). Naturally that means compiling programs is pretty slow! Eventually though this target should have a linker.
This target is also intended to be quite experimental. I'm hoping that this can act as a catalyst for further experimentation in Rust with WebAssembly. Breaking changes are very likely to land to this target, so it's not recommended to rely on it in any critical capacity yet. We'll let you know when it's "production ready".
### Building yourself
First you'll need to configure the build of LLVM and enable this target
```
$ ./configure --target=wasm32-unknown-unknown --set llvm.experimental-targets=WebAssembly
```
Next you'll want to remove any previously compiled LLVM as it needs to be rebuilt with WebAssembly support. You can do that with:
```
$ rm -rf build
```
And then you're good to go! A `./x.py build` should give you a rustc with the appropriate libstd target.
### Test support
Currently testing-wise this target is looking pretty good but isn't complete. I've got almost the entire `run-pass` test suite working with this target (lots of tests ignored, but many passing as well). The `core` test suite is [still getting LLVM bugs fixed](https://reviews.llvm.org/D39866) to get that working and will take some time. Relatively simple programs all seem to work though!
In general I've only tested this with a local fork that makes use of LLVM 5 rather than our current LLVM 4 on master. The LLVM 4 WebAssembly backend AFAIK isn't broken per se but is likely missing bug fixes available on LLVM 5. I'm hoping though that we can decouple the LLVM 5 upgrade and adding this wasm target!
### But the modules generated are huge!
It's worth nothing that you may not immediately see the "smallest possible wasm module" for the input you feed to rustc. For various reasons it's very difficult to get rid of the final "bloat" in vanilla rustc (again, a real linker should fix all this). For now what you'll have to do is:
cargo install --git https://github.com/alexcrichton/wasm-gc
wasm-gc foo.wasm bar.wasm
And then `bar.wasm` should be the smallest we can get it!
---
In any case for now I'd love feedback on this, particularly on the various integration points if you've got better ideas of how to approach them!
This commit adds a new target to the compiler: wasm32-unknown-unknown. This
target is a reimagining of what it looks like to generate WebAssembly code from
Rust. Instead of using Emscripten which can bring with it a weighty runtime this
instead is a target which uses only the LLVM backend for WebAssembly and a
"custom linker" for now which will hopefully one day be direct calls to lld.
Notable features of this target include:
* There is zero runtime footprint. The target assumes nothing exists other than
the wasm32 instruction set.
* There is zero toolchain footprint beyond adding the target. No custom linker
is needed, rustc contains everything.
* Very small wasm modules can be generated directly from Rust code using this
target.
* Most of the standard library is stubbed out to return an error, but anything
related to allocation works (aka `HashMap`, `Vec`, etc).
* Naturally, any `#[no_std]` crate should be 100% compatible with this new
target.
This target is currently somewhat janky due to how linking works. The "linking"
is currently unconditional whole program LTO (aka LLVM is being used as a
linker). Naturally that means compiling programs is pretty slow! Eventually
though this target should have a linker.
This target is also intended to be quite experimental. I'm hoping that this can
act as a catalyst for further experimentation in Rust with WebAssembly. Breaking
changes are very likely to land to this target, so it's not recommended to rely
on it in any critical capacity yet. We'll let you know when it's "production
ready".
---
Currently testing-wise this target is looking pretty good but isn't complete.
I've got almost the entire `run-pass` test suite working with this target (lots
of tests ignored, but many passing as well). The `core` test suite is still
getting LLVM bugs fixed to get that working and will take some time. Relatively
simple programs all seem to work though!
---
It's worth nothing that you may not immediately see the "smallest possible wasm
module" for the input you feed to rustc. For various reasons it's very difficult
to get rid of the final "bloat" in vanilla rustc (again, a real linker should
fix all this). For now what you'll have to do is:
cargo install --git https://github.com/alexcrichton/wasm-gc
wasm-gc foo.wasm bar.wasm
And then `bar.wasm` should be the smallest we can get it!
---
In any case for now I'd love feedback on this, particularly on the various
integration points if you've got better ideas of how to approach them!
char_private.rs is generated programmatically by char_private.py, using data
retrieved from the Unicode Consortium's website.
The motivation here was to make `is_printable` crate-visible (with
`pub(crate)`), but it would seem that the Unicode data has changed slightly
since char_private.rs was last generated.
This commit enables configuring the RLS/rustfmt tools to the "broken" state and
actually get it past CI. The main changes here were to update all dist-related
code to handle the situation where the RLS isn't available. This in turn
involved a homegrown preprocessor-like-function to edit the configuration files
we pass to the various combined installer tools.
Implement display_hint in gdb pretty printers
A few pretty-printers were returning a quoted string from their
to_string method. It's preferable in gdb to return a lazy string and to
let gdb handle the display by having a "display_hint" method that
returns "string" -- it lets gdb settings (like "set print ...") work, it
handles corrupted strings a bit better, and it passes the information
along to IDEs.
LLDB's output may be None instead of '', and that will cause type
mismatch when normalize_whitespace() expects a string instead of
None. This commit simply ensures we do pass '' even if the output
is None.
A few pretty-printers were returning a quoted string from their
to_string method. It's preferable in gdb to return a lazy string and to
let gdb handle the display by having a "display_hint" method that
returns "string" -- it lets gdb settings (like "set print ...") work, it
handles corrupted strings a bit better, and it passes the information
along to IDEs.
let htmldocck.py check for directories
Since i messed this up during https://github.com/rust-lang/rust/pull/44613, i wanted to codify this into the rustdoc tests to make sure that doesn't happen again.
Embed MSVC .natvis files into .pdbs and mangle debuginfo for &str, *T, and [T].
No idea if these changes are reasonable - please feel free to suggest changes/rewrites. And these are some of my first real commits to any rust codebase - *don't* be gentle, and nitpick away, I need to learn! ;)
### Overview
Embedding `.natvis` files into `.pdb`s allows MSVC (and potentially other debuggers) to automatically pick up the visualizers without having to do any additional configuration (other than to perhaps add the relevant .pdb paths to symbol search paths.)
The native debug engine for MSVC parses the type names, making various C++ish assumptions about what they mean and adding various limitations to valid type names. `&str` cannot be matched against a visualizer, but if we emit `str&` instead, it'll be recognized as a reference to a `str`, solving the problem. `[T]` is similarly problematic, but emitting `slice<T>` instead works fine as it looks like a template. I've been unable to get e.g. `slice<u32>&` to match visualizers in VS2015u3, so I've gone with `str*` and `slice<u32>*` instead.
### Possible Issues
* I'm not sure if `slice<T>` is a great mangling for `[T]` or if I should worry about name collisions.
* I'm not sure if `linker.rs` is the right place to be enumerating natvis files.
* I'm not sure if these type name mangling changes should actually be MSVC specific. I recall seeing gdb visualizer tests that might be broken if made more general? I'm hesitant to mess with them without a gdb install. But perhaps I'm just wracking up technical debt.
Should I try `pacman -S mingw-w64-x86_64-gdb` and to make things consistent?
* I haven't touched `const` / `mut` yet, and I'm worried MSVC might trip up on `mut` or their placement.
* I may like terse oneliners too much.
* I don't know if there's broader implications for messing with debug type names here.
* I may have been mistaken about bellow test failures being ignorable / unrelated to this changelist.
### Test Failures on `x86_64-pc-windows-gnu`
```
---- [debuginfo-gdb] debuginfo-gdb\associated-types.rs stdout ----
thread '[debuginfo-gdb] debuginfo-gdb\associated-types.rs' panicked at 'gdb not available but debuginfo gdb debuginfo test requested', src\tools\compiletest\src\runtest.rs:48:16
note: Run with `RUST_BACKTRACE=1` for a backtrace.
[...identical panic causes omitted...]
---- [debuginfo-gdb] debuginfo-gdb\vec.rs stdout ----
thread '[debuginfo-gdb] debuginfo-gdb\vec.rs' panicked at 'gdb not available but debuginfo gdb debuginfo test requested', src\tools\compiletest\src\runtest.rs:48:16
```
### Relevant Issues
* https://github.com/rust-lang/rust/issues/40460 Metaissue for Visual Studio debugging Rust
* https://github.com/rust-lang/rust/issues/36503 Investigate natvis for improved msvc debugging
* https://github.com/PistonDevelopers/VisualRust/issues/160 Debug visualization of Rust data structures
### Pretty Pictures
