Auto merge of #113108 - compiler-errors:normalize-opaques-with-late-bound-vars-again, r=jackh726

Normalize opaques with late-bound vars again

We have a hack in the compiler where if an opaque has escaping late-bound vars, we skip revealing it even though we *could* reveal it from a technical perspective. First of all, this is weird, since we really should be revealing all opaques in `Reveal::All` mode. Second of all, it causes subtle bugs (linked below).

I attempted to fix this in #100980, which was unfortunately reverted due to perf regressions on codebases that used really deeply nested futures in some interesting ways. The worst of which was #103423, which caused the project to hang on build. Another one was #104842, which was just a slow-down, but not a hang. I took some time afterwards to investigate how to rework `normalize_erasing_regions` to take advantage of better caching, but that effort kinda fizzled out (#104133).

However, recently, I was made aware of more bugs whose root cause is not revealing opaques during codegen. That made me want to fix this again -- in the process, interestingly, I took the the minimized example from https://github.com/rust-lang/rust/issues/103423#issuecomment-1292947043, and it doesn't seem to hang any more...

Thinking about this harder, there have been some changes to the way we lower and typecheck async futures that may have reduced the pathologically large number of outlives obligations (see description of #103423) that we were encountering when normalizing opaques with bound vars the last time around:
* #104321 (lower `async { .. }` directly as a generator that implements `Future`, removing the `from_generator` shim)
* #104833 (removing an `identity_future` fn that was wrapping desugared future generators)

... so given that I can see:
* No significant regression on rust perf bot (https://github.com/rust-lang/rust/pull/107620#issuecomment-1600070317)
* No timeouts in crater run I did (https://github.com/rust-lang/rust/pull/107620#issuecomment-1605428952, rechecked failing crates in https://github.com/rust-lang/rust/pull/107620#issuecomment-1605973434)

... and given that this PR:
* Fixes #104601
* Fixes #107557
* Fixes #109464
* Allows us to remove a `DefiningAnchor::Bubble` from codegen (75a8f681837c70051e0200a14f58ae07dbe58e66)

I'm inclined to give this another shot at landing this. Best case, it just works -- worst case, we get more examples to study how we need to improve the compiler to make this work.

r? types

compiler/rustc_trait_selection/src/traits/project.rs

@@ -500,10 +500,7 @@ impl<'a, 'b, 'tcx> TypeFolder<TyCtxt<'tcx>> for AssocTypeNormalizer<'a, 'b, 'tcx
         // to make sure we don't forget to fold the substs regardless.
 
         match kind {
-            // This is really important. While we *can* handle this, this has
-            // severe performance implications for large opaque types with
-            // late-bound regions. See `issue-88862` benchmark.
-            ty::Opaque if !data.substs.has_escaping_bound_vars() => {
+            ty::Opaque => {
                 // Only normalize `impl Trait` outside of type inference, usually in codegen.
                 match self.param_env.reveal() {
                     Reveal::UserFacing => ty.super_fold_with(self),
@@ -529,7 +526,6 @@ impl<'a, 'b, 'tcx> TypeFolder<TyCtxt<'tcx>> for AssocTypeNormalizer<'a, 'b, 'tcx
                     }
                 }
             }
-            ty::Opaque => ty.super_fold_with(self),
 
             ty::Projection if !data.has_escaping_bound_vars() => {
                 // This branch is *mostly* just an optimization: when we don't

compiler/rustc_trait_selection/src/traits/query/normalize.rs

@@ -211,10 +211,7 @@ impl<'cx, 'tcx> FallibleTypeFolder<TyCtxt<'tcx>> for QueryNormalizer<'cx, 'tcx>
 
         // Wrap this in a closure so we don't accidentally return from the outer function
         let res = match kind {
-            // This is really important. While we *can* handle this, this has
-            // severe performance implications for large opaque types with
-            // late-bound regions. See `issue-88862` benchmark.
-            ty::Opaque if !data.substs.has_escaping_bound_vars() => {
+            ty::Opaque => {
                 // Only normalize `impl Trait` outside of type inference, usually in codegen.
                 match self.param_env.reveal() {
                     Reveal::UserFacing => ty.try_super_fold_with(self)?,
@@ -255,8 +252,6 @@ impl<'cx, 'tcx> FallibleTypeFolder<TyCtxt<'tcx>> for QueryNormalizer<'cx, 'tcx>
                 }
             }
 
-            ty::Opaque => ty.try_super_fold_with(self)?,
-
             ty::Projection | ty::Inherent | ty::Weak => {
                 // See note in `rustc_trait_selection::traits::project`
 

compiler/rustc_traits/src/codegen.rs

@@ -5,7 +5,7 @@
 
 use rustc_infer::infer::TyCtxtInferExt;
 use rustc_infer::traits::{FulfillmentErrorCode, TraitEngineExt as _};
-use rustc_middle::traits::{CodegenObligationError, DefiningAnchor};
+use rustc_middle::traits::CodegenObligationError;
 use rustc_middle::ty::{self, TyCtxt};
 use rustc_trait_selection::traits::error_reporting::TypeErrCtxtExt;
 use rustc_trait_selection::traits::{
@@ -29,13 +29,7 @@ pub fn codegen_select_candidate<'tcx>(
 
     // Do the initial selection for the obligation. This yields the
     // shallow result we are looking for -- that is, what specific impl.
-    let infcx = tcx
-        .infer_ctxt()
-        .ignoring_regions()
-        .with_opaque_type_inference(DefiningAnchor::Bubble)
-        .build();
-    //~^ HACK `Bubble` is required for
-    // this test to pass: type-alias-impl-trait/assoc-projection-ice.rs
+    let infcx = tcx.infer_ctxt().ignoring_regions().build();
     let mut selcx = SelectionContext::new(&infcx);
 
     let obligation_cause = ObligationCause::dummy();
@@ -79,10 +73,5 @@ pub fn codegen_select_candidate<'tcx>(
     let impl_source = infcx.resolve_vars_if_possible(impl_source);
     let impl_source = infcx.tcx.erase_regions(impl_source);
 
-    // Opaque types may have gotten their hidden types constrained, but we can ignore them safely
-    // as they will get constrained elsewhere, too.
-    // (ouz-a) This is required for `type-alias-impl-trait/assoc-projection-ice.rs` to pass
-    let _ = infcx.take_opaque_types();
-
     Ok(&*tcx.arena.alloc(impl_source))
 }

tests/ui/async-await/in-trait/normalize-opaque-with-bound-vars.rs

@@ -0,0 +1,64 @@
+// build-pass
+// edition:2021
+// compile-flags: -Cdebuginfo=2
+
+// We were not normalizing opaques with escaping bound vars during codegen,
+// leading to later errors during debuginfo computation.
+
+#![feature(async_fn_in_trait)]
+
+#[derive(Clone, Copy)]
+pub struct SharedState {}
+
+pub trait State {
+    async fn execute(self, shared_state: &SharedState);
+}
+
+pub trait StateComposer {
+    fn and_then<T, F>(self, map_fn: F) -> AndThen<Self, F>
+    where
+        Self: State + Sized,
+        T: State,
+        F: FnOnce() -> T,
+    {
+        AndThen { previous: self, map_fn }
+    }
+}
+
+impl<T> StateComposer for T where T: State {}
+pub struct AndThen<T, F> {
+    previous: T,
+    map_fn: F,
+}
+
+impl<T, U, F> State for AndThen<T, F>
+where
+    T: State,
+    U: State,
+    F: FnOnce() -> U,
+{
+    async fn execute(self, shared_state: &SharedState)
+    where
+        Self: Sized,
+    {
+        self.previous.execute(shared_state).await;
+        (self.map_fn)().execute(shared_state).await
+    }
+}
+
+pub struct SomeState {}
+
+impl State for SomeState {
+    async fn execute(self, shared_state: &SharedState) {}
+}
+
+pub fn main() {
+    let shared_state = SharedState {};
+    async {
+        SomeState {}
+            .and_then(|| SomeState {})
+            .and_then(|| SomeState {})
+            .execute(&shared_state)
+            .await;
+    };
+}

tests/ui/impl-trait/normalize-opaque-with-bound-vars.rs

@@ -0,0 +1,27 @@
+// build-pass
+// edition:2021
+// compile-flags: -Cdebuginfo=2
+
+// We were not normalizing opaques with escaping bound vars during codegen,
+// leading to later linker errors because of differences in mangled symbol name.
+
+fn func<T>() -> impl Sized {}
+
+trait Trait<'a> {
+    type Assoc;
+
+    fn call() {
+        let _ = async {
+            let _value = func::<Self::Assoc>();
+            std::future::ready(()).await
+        };
+    }
+}
+
+impl Trait<'static> for () {
+    type Assoc = ();
+}
+
+fn main() {
+    <()>::call();
+}