2020-12-31 18:53:25 -06:00
|
|
|
use crate::MirPass;
|
2020-03-29 10:19:48 -05:00
|
|
|
use rustc_index::vec::{Idx, IndexVec};
|
2020-03-29 09:41:09 -05:00
|
|
|
use rustc_middle::mir::*;
|
|
|
|
use rustc_middle::ty::TyCtxt;
|
2016-06-07 09:28:36 -05:00
|
|
|
|
2017-08-09 13:30:55 -05:00
|
|
|
#[derive(PartialEq)]
|
|
|
|
pub enum AddCallGuards {
|
|
|
|
AllCallEdges,
|
|
|
|
CriticalCallEdges,
|
|
|
|
}
|
|
|
|
pub use self::AddCallGuards::*;
|
Only break critical edges where actually needed
Currently, to prepare for MIR trans, we break _all_ critical edges,
although we only actually need to do this for edges originating from a
call that gets translated to an invoke instruction in LLVM.
This has the unfortunate effect of undoing a bunch of the things that
SimplifyCfg has done. A particularly bad case arises when you have a
C-like enum with N variants and a derived PartialEq implementation.
In that case, the match on the (&lhs, &rhs) tuple gets translated into
nested matches with N arms each and a basic block each, resulting in N²
basic blocks. SimplifyCfg reduces that to roughly 2*N basic blocks, but
breaking the critical edges means that we go back to N².
In nickel.rs, there is such an enum with roughly N=800. So we get about
640K basic blocks or 2.5M lines of LLVM IR. LLVM takes a while to
reduce that to the final "disr_a == disr_b".
So before this patch, we had 2.5M lines of IR with 640K basic blocks,
which took about about 3.6s in LLVM to get optimized and translated.
After this patch, we get about 650K lines with about 1.6K basic blocks
and spent a little less than 0.2s in LLVM.
cc #33111
2016-05-10 14:03:47 -05:00
|
|
|
|
|
|
|
/**
|
|
|
|
* Breaks outgoing critical edges for call terminators in the MIR.
|
|
|
|
*
|
|
|
|
* Critical edges are edges that are neither the only edge leaving a
|
|
|
|
* block, nor the only edge entering one.
|
|
|
|
*
|
|
|
|
* When you want something to happen "along" an edge, you can either
|
|
|
|
* do at the end of the predecessor block, or at the start of the
|
|
|
|
* successor block. Critical edges have to be broken in order to prevent
|
|
|
|
* "edge actions" from affecting other edges. We need this for calls that are
|
2018-05-08 08:10:16 -05:00
|
|
|
* codegened to LLVM invoke instructions, because invoke is a block terminator
|
Only break critical edges where actually needed
Currently, to prepare for MIR trans, we break _all_ critical edges,
although we only actually need to do this for edges originating from a
call that gets translated to an invoke instruction in LLVM.
This has the unfortunate effect of undoing a bunch of the things that
SimplifyCfg has done. A particularly bad case arises when you have a
C-like enum with N variants and a derived PartialEq implementation.
In that case, the match on the (&lhs, &rhs) tuple gets translated into
nested matches with N arms each and a basic block each, resulting in N²
basic blocks. SimplifyCfg reduces that to roughly 2*N basic blocks, but
breaking the critical edges means that we go back to N².
In nickel.rs, there is such an enum with roughly N=800. So we get about
640K basic blocks or 2.5M lines of LLVM IR. LLVM takes a while to
reduce that to the final "disr_a == disr_b".
So before this patch, we had 2.5M lines of IR with 640K basic blocks,
which took about about 3.6s in LLVM to get optimized and translated.
After this patch, we get about 650K lines with about 1.6K basic blocks
and spent a little less than 0.2s in LLVM.
cc #33111
2016-05-10 14:03:47 -05:00
|
|
|
* in LLVM so we can't insert any code to handle the call's result into the
|
|
|
|
* block that performs the call.
|
|
|
|
*
|
|
|
|
* This function will break those edges by inserting new blocks along them.
|
|
|
|
*
|
|
|
|
* NOTE: Simplify CFG will happily undo most of the work this pass does.
|
|
|
|
*
|
|
|
|
*/
|
|
|
|
|
2019-08-04 15:20:00 -05:00
|
|
|
impl<'tcx> MirPass<'tcx> for AddCallGuards {
|
2020-10-04 13:01:38 -05:00
|
|
|
fn run_pass(&self, _tcx: TyCtxt<'tcx>, body: &mut Body<'tcx>) {
|
2019-11-06 11:00:46 -06:00
|
|
|
self.add_call_guards(body);
|
2017-03-09 12:36:01 -06:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2017-08-09 13:30:55 -05:00
|
|
|
impl AddCallGuards {
|
2020-04-12 12:31:00 -05:00
|
|
|
pub fn add_call_guards(&self, body: &mut Body<'_>) {
|
2021-08-16 19:00:00 -05:00
|
|
|
let mut pred_count: IndexVec<_, _> =
|
|
|
|
body.predecessors().iter().map(|ps| ps.len()).collect();
|
|
|
|
pred_count[START_BLOCK] += 1;
|
Only break critical edges where actually needed
Currently, to prepare for MIR trans, we break _all_ critical edges,
although we only actually need to do this for edges originating from a
call that gets translated to an invoke instruction in LLVM.
This has the unfortunate effect of undoing a bunch of the things that
SimplifyCfg has done. A particularly bad case arises when you have a
C-like enum with N variants and a derived PartialEq implementation.
In that case, the match on the (&lhs, &rhs) tuple gets translated into
nested matches with N arms each and a basic block each, resulting in N²
basic blocks. SimplifyCfg reduces that to roughly 2*N basic blocks, but
breaking the critical edges means that we go back to N².
In nickel.rs, there is such an enum with roughly N=800. So we get about
640K basic blocks or 2.5M lines of LLVM IR. LLVM takes a while to
reduce that to the final "disr_a == disr_b".
So before this patch, we had 2.5M lines of IR with 640K basic blocks,
which took about about 3.6s in LLVM to get optimized and translated.
After this patch, we get about 650K lines with about 1.6K basic blocks
and spent a little less than 0.2s in LLVM.
cc #33111
2016-05-10 14:03:47 -05:00
|
|
|
|
2017-08-09 13:30:55 -05:00
|
|
|
// We need a place to store the new blocks generated
|
|
|
|
let mut new_blocks = Vec::new();
|
Only break critical edges where actually needed
Currently, to prepare for MIR trans, we break _all_ critical edges,
although we only actually need to do this for edges originating from a
call that gets translated to an invoke instruction in LLVM.
This has the unfortunate effect of undoing a bunch of the things that
SimplifyCfg has done. A particularly bad case arises when you have a
C-like enum with N variants and a derived PartialEq implementation.
In that case, the match on the (&lhs, &rhs) tuple gets translated into
nested matches with N arms each and a basic block each, resulting in N²
basic blocks. SimplifyCfg reduces that to roughly 2*N basic blocks, but
breaking the critical edges means that we go back to N².
In nickel.rs, there is such an enum with roughly N=800. So we get about
640K basic blocks or 2.5M lines of LLVM IR. LLVM takes a while to
reduce that to the final "disr_a == disr_b".
So before this patch, we had 2.5M lines of IR with 640K basic blocks,
which took about about 3.6s in LLVM to get optimized and translated.
After this patch, we get about 650K lines with about 1.6K basic blocks
and spent a little less than 0.2s in LLVM.
cc #33111
2016-05-10 14:03:47 -05:00
|
|
|
|
2019-11-06 11:00:46 -06:00
|
|
|
let cur_len = body.basic_blocks().len();
|
Only break critical edges where actually needed
Currently, to prepare for MIR trans, we break _all_ critical edges,
although we only actually need to do this for edges originating from a
call that gets translated to an invoke instruction in LLVM.
This has the unfortunate effect of undoing a bunch of the things that
SimplifyCfg has done. A particularly bad case arises when you have a
C-like enum with N variants and a derived PartialEq implementation.
In that case, the match on the (&lhs, &rhs) tuple gets translated into
nested matches with N arms each and a basic block each, resulting in N²
basic blocks. SimplifyCfg reduces that to roughly 2*N basic blocks, but
breaking the critical edges means that we go back to N².
In nickel.rs, there is such an enum with roughly N=800. So we get about
640K basic blocks or 2.5M lines of LLVM IR. LLVM takes a while to
reduce that to the final "disr_a == disr_b".
So before this patch, we had 2.5M lines of IR with 640K basic blocks,
which took about about 3.6s in LLVM to get optimized and translated.
After this patch, we get about 650K lines with about 1.6K basic blocks
and spent a little less than 0.2s in LLVM.
cc #33111
2016-05-10 14:03:47 -05:00
|
|
|
|
2019-11-06 11:00:46 -06:00
|
|
|
for block in body.basic_blocks_mut() {
|
2019-10-03 23:55:28 -05:00
|
|
|
match block.terminator {
|
2017-08-09 13:30:55 -05:00
|
|
|
Some(Terminator {
|
2019-12-22 16:42:04 -06:00
|
|
|
kind:
|
|
|
|
TerminatorKind::Call {
|
|
|
|
destination: Some((_, ref mut destination)),
|
|
|
|
cleanup,
|
|
|
|
..
|
|
|
|
},
|
|
|
|
source_info,
|
|
|
|
}) if pred_count[*destination] > 1
|
|
|
|
&& (cleanup.is_some() || self == &AllCallEdges) =>
|
2017-08-09 13:30:55 -05:00
|
|
|
{
|
|
|
|
// It's a critical edge, break it
|
|
|
|
let call_guard = BasicBlockData {
|
|
|
|
statements: vec![],
|
2019-10-03 23:55:28 -05:00
|
|
|
is_cleanup: block.is_cleanup,
|
2017-08-09 13:30:55 -05:00
|
|
|
terminator: Some(Terminator {
|
2019-10-03 23:55:28 -05:00
|
|
|
source_info,
|
2019-12-22 16:42:04 -06:00
|
|
|
kind: TerminatorKind::Goto { target: *destination },
|
|
|
|
}),
|
2017-08-09 13:30:55 -05:00
|
|
|
};
|
2016-06-05 01:00:17 -05:00
|
|
|
|
2017-08-09 13:30:55 -05:00
|
|
|
// Get the index it will be when inserted into the MIR
|
|
|
|
let idx = cur_len + new_blocks.len();
|
|
|
|
new_blocks.push(call_guard);
|
|
|
|
*destination = BasicBlock::new(idx);
|
|
|
|
}
|
|
|
|
_ => {}
|
Only break critical edges where actually needed
Currently, to prepare for MIR trans, we break _all_ critical edges,
although we only actually need to do this for edges originating from a
call that gets translated to an invoke instruction in LLVM.
This has the unfortunate effect of undoing a bunch of the things that
SimplifyCfg has done. A particularly bad case arises when you have a
C-like enum with N variants and a derived PartialEq implementation.
In that case, the match on the (&lhs, &rhs) tuple gets translated into
nested matches with N arms each and a basic block each, resulting in N²
basic blocks. SimplifyCfg reduces that to roughly 2*N basic blocks, but
breaking the critical edges means that we go back to N².
In nickel.rs, there is such an enum with roughly N=800. So we get about
640K basic blocks or 2.5M lines of LLVM IR. LLVM takes a while to
reduce that to the final "disr_a == disr_b".
So before this patch, we had 2.5M lines of IR with 640K basic blocks,
which took about about 3.6s in LLVM to get optimized and translated.
After this patch, we get about 650K lines with about 1.6K basic blocks
and spent a little less than 0.2s in LLVM.
cc #33111
2016-05-10 14:03:47 -05:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2017-08-09 13:30:55 -05:00
|
|
|
debug!("Broke {} N edges", new_blocks.len());
|
Only break critical edges where actually needed
Currently, to prepare for MIR trans, we break _all_ critical edges,
although we only actually need to do this for edges originating from a
call that gets translated to an invoke instruction in LLVM.
This has the unfortunate effect of undoing a bunch of the things that
SimplifyCfg has done. A particularly bad case arises when you have a
C-like enum with N variants and a derived PartialEq implementation.
In that case, the match on the (&lhs, &rhs) tuple gets translated into
nested matches with N arms each and a basic block each, resulting in N²
basic blocks. SimplifyCfg reduces that to roughly 2*N basic blocks, but
breaking the critical edges means that we go back to N².
In nickel.rs, there is such an enum with roughly N=800. So we get about
640K basic blocks or 2.5M lines of LLVM IR. LLVM takes a while to
reduce that to the final "disr_a == disr_b".
So before this patch, we had 2.5M lines of IR with 640K basic blocks,
which took about about 3.6s in LLVM to get optimized and translated.
After this patch, we get about 650K lines with about 1.6K basic blocks
and spent a little less than 0.2s in LLVM.
cc #33111
2016-05-10 14:03:47 -05:00
|
|
|
|
2019-11-06 11:00:46 -06:00
|
|
|
body.basic_blocks_mut().extend(new_blocks);
|
2017-08-09 13:30:55 -05:00
|
|
|
}
|
Only break critical edges where actually needed
Currently, to prepare for MIR trans, we break _all_ critical edges,
although we only actually need to do this for edges originating from a
call that gets translated to an invoke instruction in LLVM.
This has the unfortunate effect of undoing a bunch of the things that
SimplifyCfg has done. A particularly bad case arises when you have a
C-like enum with N variants and a derived PartialEq implementation.
In that case, the match on the (&lhs, &rhs) tuple gets translated into
nested matches with N arms each and a basic block each, resulting in N²
basic blocks. SimplifyCfg reduces that to roughly 2*N basic blocks, but
breaking the critical edges means that we go back to N².
In nickel.rs, there is such an enum with roughly N=800. So we get about
640K basic blocks or 2.5M lines of LLVM IR. LLVM takes a while to
reduce that to the final "disr_a == disr_b".
So before this patch, we had 2.5M lines of IR with 640K basic blocks,
which took about about 3.6s in LLVM to get optimized and translated.
After this patch, we get about 650K lines with about 1.6K basic blocks
and spent a little less than 0.2s in LLVM.
cc #33111
2016-05-10 14:03:47 -05:00
|
|
|
}
|