Allow writing of incomplete UTF-8 sequences to the Windows console via stdout/stderr
# Problem
Writes of just an incomplete UTF-8 byte sequence (e.g. `b"\xC3"` or `b"\xF0\x9F"`) to stdout/stderr with a Windows console attached error with `io::ErrorKind::InvalidData, "Windows stdio in console mode does not support writing non-UTF-8 byte sequences"` even though further writes could complete the codepoint. This is currently a rare occurence since the [linewritershim](2c56ea38b0/library/std/src/io/buffered/linewritershim.rs) implementation flushes complete lines immediately and buffers up to 1024 bytes for incomplete lines. It can still happen as described in #83258.
The problem will become more pronounced once the developer can switch stdout/stderr from line-buffered to block-buffered or immediate when the changes in the "Switchable buffering for Stdout" pull request (#78515) get merged.
# Patch description
If there is at least one valid UTF-8 codepoint all valid UTF-8 is passed through to the extracted `write_valid_utf8_to_console()` fn. The new code only comes into play if `write()` is being passed a short byte slice comprising an incomplete UTF-8 codepoint. In this case up to three bytes are buffered in the `IncompleteUtf8` struct associated with `Stdout` / `Stderr`. The bytes are accepted one at a time. As soon as an error can be detected `io::ErrorKind::InvalidData, "Windows stdio in console mode does not support writing non-UTF-8 byte sequences"` is returned. Once a complete UTF-8 codepoint is received it is passed to the `write_valid_utf8_to_console()` and the buffer length is set to zero.
Calling `flush()` will neither error nor write anything if an incomplete codepoint is present in the buffer.
# Tests
Currently there are no Windows-specific tests for console writing code at all. Writing (regression) tests for this problem is a bit challenging since unit tests and UI tests don't run in a console and suddenly popping up another console window might be surprising to developers running the testsuite and it might not work at all in CI builds. To just test the new functionality in unit tests the code would need to be refactored. Some guidance on how to proceed would be appreciated.
# Public API changes
* `std::str::verifications::utf8_char_width()` would be exposed as `std::str::utf8_char_width()` behind the "str_internals" feature gate.
# Related issues
* Fixes#83258.
* PR #78515 will exacerbate the problem.
# Open questions
* Add tests?
* Squash into one commit with better commit message?
For two reasons:
1. Now that the suggestion span has been corrected, the output is a bit
cluttered and hard to read. Putting the suggestion its own window
creates more space.
2. It's easier to see what's being suggested, since now the version
after the suggestion is applied is shown.
(And same for tuple variants.)
Previously, the span was just for the constructor name, which meant it
would result in syntactically-invalid code when applied. Now, the span
is for the entire expression.
Reference new diagnostic item docs in our docs :)
The title says it all. The rustc dev guide now has some information about diagnostic items that are worthwhile linking to 🙃
---
changelog: none
rustdoc: Box `GenericArgs::Parenthesized.output`
Split out from #88379.
This reduces the size of `GenericArgs` from 104 bytes to 56 bytes,
essentially reducing it by half.
`GenericArgs` is one of the fields of `PathSegment`, so this should
reduce the amount of memory allocated for `PathSegment`s in the cases
where the generics are not for a `Fn`, `FnMut`, or `FnOnce` trait.
r? `@jyn514`
MIR lowering for `if let` expressions is now more complicated now that
`if let` exists in HIR. This PR adds a scope for the variables bound in
an `if let` expression and then uses an approach similar to how we
handle loops to ensure that we reliably drop the correct variables.
Instead, return `Type::Infer` since compilation should fail anyway.
That's how rustdoc handles `hir::TyKind::Err`s, so this just extends
that behavior to `ty::Err`s when analyzing associated types.
For some reason, the error is printed twice with rustdoc (though only
once with rustc). I'm not sure why that is, but it's better than
panicking.
This commit also makes rustdoc fail early in the non-projection,
non-error case, instead of returning a `Res::Err` that would likely
cause rustdoc to panic later on. This change is originally from #88379.
Display associated types of implementors
Fixes#86631.
Contrary to before, it doesn't display methods. I also had to "resurrect" the `auto-hide-trait-implementations` setting. :3
Only question at this point: should I move the `render_impl` boolean arguments into one struct? We're starting to have quite a lot of them...
cc `@cynecx`
r? `@camelid`
Previously, `QueryJobInfo` was composed of two parts: a `QueryInfo` and
a `QueryJob`. However, both `QueryInfo` and `QueryJob` have a `span`
field, which seem to be the same. So, the `span` was recorded twice.
Now, `QueryJobInfo` is composed of a `QueryStackFrame` (the other field
of `QueryInfo`) and a `QueryJob`. So, now, the `span` is only recorded
once.
Update cargo, books
## nomicon
2 commits in 0c7e5bd1428e7838252bb57b7f0fbfda4ec82f02..fe6227eb3c8533200c52dffa42ef1b6f2f02c40e
2021-08-04 10:18:22 -0700 to 2021-08-31 05:42:38 +0900
- update lifetime-elision to show what elided code under `rust_2018_idi… (rust-lang/nomicon#306)
- Change code for `into_iter` on the `RawVec` section for consistency/soundness (rust-lang/nomicon#302)
## cargo
8 commits in f559c109cc79fe413a8535fb620a5a58b3823d94..18751dd3f238d94d384a7fe967abfac06cbfe0b9
2021-08-26 22:54:55 +0000 to 2021-09-01 14:26:00 +0000
- print the full destination path when no track duplicates (rust-lang/cargo#9850)
- Stabilize 2021 edition (rust-lang/cargo#9800)
- Stabilize patch-in-config (and prefer config over manifest) (rust-lang/cargo#9839)
- Adding the cargo doc --examples subcommand (rust-lang/cargo#9808)
- Make library created with `cargo new` clippy happy (rust-lang/cargo#9796)
- Swap out some outdated repo urls in documentation (rust-lang/cargo#9862)
- Change `cargo fix --edition` to only fix edition lints. (rust-lang/cargo#9846)
- Show desc of well known subcommands (fmt, clippy) in cargo --list (rust-lang/cargo#9848)
## reference
1 commits in da6ea9b03f74cae0a292f40315723d7a3a973637..0e5ed7a4bec065f0cc18c35d1c904639e095314d
2021-08-19 21:28:10 -0700 to 2021-08-29 17:33:21 +0900
- expressions.md: Attempt fixing broken grammar in Mutability paragraph (rust-lang/reference#1084)
## book
1 commits in 687e21bde2ea10c261f79fa14797c5137425098d..fcb5e0ea68112d85a1d29a7a7335978ef2a02181
2021-08-18 20:48:38 -0400 to 2021-08-31 21:26:19 -0400
- Improve the reading of the code (rust-lang/book#2845)
## rustc-dev-guide
7 commits in cf0e151b7925a40f13fbc6573c6f97d5f94c7c17..95f1acf9a39d6f402f654e917e2c1dfdb779c5fc
2021-08-22 11:47:02 -0300 to 2021-08-31 12:38:30 -0500
- Add link to `Span`
- Add rustc-source to suggested rust-analyzer config (rust-lang/rustc-dev-guide#1189)
- Fix typo, clarify backtick wording, and use inline code
- Trailing date comments in a line inside of a paragraph caused beginning of a new paragraph. (rust-lang/rustc-dev-guide#1196)
- Fix warning "Renderer command uses a path relative to the renderer output directory ..." (rust-lang/rustc-dev-guide#1194)
- Fix a code block containing ```rust
- date-check: Recognize capitalized 'Date' as well
## edition-guide
1 commits in 3710b0cae783d0bcd2b42452a63b081473f5970a..2d9b1b9da706de24650fdc5c3b0182f55c82115d
2021-07-26 11:34:46 -0700 to 2021-08-31 10:44:09 +0200
- Update for 2021 stabilization (rust-lang/edition-guide#266)
## embedded-book
1 commits in 4f9fcaa30d11ba52b641e6fd5206536d65838af9..c3a51e23859554369e6bbb5128dcef0e4f159fb5
2021-08-06 17:43:12 +0000 to 2021-08-26 07:04:58 +0000
- Make glossary more linkable and add more detail (rust-embedded/book#299)
