Do not ICE in privacy when type inference fails. Fixes https://github.com/rust-lang/rust/issues/122736