Skip to content

[DSE] Delay deleting non-memory-defs until end of DSE. #83411

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Mar 2, 2024
Merged

[DSE] Delay deleting non-memory-defs until end of DSE. #83411

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
f29e3f7
[DSE] Delay deleting non-memory-defs until end of DSE.
fhahn Feb 29, 2024
952b51b
Merge remote-tracking branch 'origin/main' into perf/dse-delay-delete
fhahn Feb 29, 2024
33f9631
!fixup fix build failure.
fhahn Feb 29, 2024
1d036f4
Merge remote-tracking branch 'origin/main' into perf/dse-delay-delete
fhahn Mar 1, 2024
be3aaa1
!fixup remove assert, directly delete MemDefs with void ty.
fhahn Mar 1, 2024
8fc503c
Merge remote-tracking branch 'origin/main' into perf/dse-delay-delete
fhahn Mar 2, 2024
c839e00
!fixup add extra tests.
fhahn Mar 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 26 additions & 5 deletions llvm/lib/Transforms/Scalar/DeadStoreElimination.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -857,6 +857,9 @@ struct DSEState {
// no longer be captured.
bool ShouldIterateEndOfFunctionDSE;

/// Dead instructions to be removed at the end of DSE.
SmallVector<Instruction *> ToRemove;

// Class contains self-reference, make sure it's not copied/moved.
DSEState(const DSEState &) = delete;
DSEState &operator=(const DSEState &) = delete;
Expand Down Expand Up @@ -1692,7 +1695,8 @@ struct DSEState {
return {MaybeDeadAccess};
}

// Delete dead memory defs
/// Delete dead memory defs and recursively add their operands to ToRemove if
/// they became dead.
void deleteDeadInstruction(Instruction *SI) {
MemorySSAUpdater Updater(&MSSA);
SmallVector<Instruction *, 32> NowDeadInsts;
Expand All @@ -1708,8 +1712,11 @@ struct DSEState {
salvageKnowledge(DeadInst);

// Remove the Instruction from MSSA.
if (MemoryAccess *MA = MSSA.getMemoryAccess(DeadInst)) {
if (MemoryDef *MD = dyn_cast<MemoryDef>(MA)) {
MemoryAccess *MA = MSSA.getMemoryAccess(DeadInst);
bool IsMemDef = MA && isa<MemoryDef>(MA);
if (MA) {
if (IsMemDef) {
auto *MD = cast<MemoryDef>(MA);
SkipStores.insert(MD);
if (auto *SI = dyn_cast<StoreInst>(MD->getMemoryInst())) {
if (SI->getValueOperand()->getType()->isPointerTy()) {
Expand All @@ -1730,13 +1737,21 @@ struct DSEState {
// Remove its operands
for (Use &O : DeadInst->operands())
if (Instruction *OpI = dyn_cast<Instruction>(O)) {
O = nullptr;
O.set(PoisonValue::get(O->getType()));
if (isInstructionTriviallyDead(OpI, &TLI))
NowDeadInsts.push_back(OpI);
}

EI.removeInstruction(DeadInst);
DeadInst->eraseFromParent();
// Remove memory defs directly if they don't produce results, but only
// queue other dead instructions for later removal. They may have been
// used as memory locations that have been cached by BatchAA. Removing
// them here may lead to newly created instructions to be allocated at the
// same address, yielding stale cache entries.
if (IsMemDef && DeadInst->getType()->isVoidTy())
DeadInst->eraseFromParent();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if there could be cases where a MemoryDef also loads a value that is then used as pointer of MemoryLocation

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it safer to just delay the deletion for them too?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 to this question, I'm not clear if this is possible. MemoryDefs could be volatile loads, but this is not the case here since the pass is only looking at store/writing instructions. A memcpy does also load, so a memcpy to a location that is later overwritten, could be deleted here, while copying from a location that's already in the cache.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At this time I'm inclined to say this cannot happen. So let's have this fix in for the existing issue.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deleting a memcpy wouldn't be a problem, as long as its pointer operands aren't deleted.

But we might remove something like a malloc call, which is a memory def, and the result of the malloc could be used as address and be in the cache. Updated the code to remove memory-defs that do not produce a value (i.e. have type void)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to add more tests to cover the void/non-void type cases mentioned?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a couple of extra tests based on the original test.

else
ToRemove.push_back(DeadInst);
}
}

Expand Down Expand Up @@ -2287,6 +2302,12 @@ static bool eliminateDeadStores(Function &F, AliasAnalysis &AA, MemorySSA &MSSA,

MadeChange |= State.eliminateRedundantStoresOfExistingValues();
MadeChange |= State.eliminateDeadWritesAtEndOfFunction();

while (!State.ToRemove.empty()) {
Instruction *DeadInst = State.ToRemove.pop_back_val();
DeadInst->eraseFromParent();
}

return MadeChange;
}
} // end anonymous namespace
Expand Down
189 changes: 189 additions & 0 deletions llvm/test/Transforms/DeadStoreElimination/batchaa-caching-new-pointers.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,189 @@
; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --version 4
; RUN: opt -S -passes=dse < %s | FileCheck %s
;
; DSE kills `store i32 44, ptr %struct.byte.4, align 4` but should not kill
; `call void @llvm.memset.p0.i64(...)` because it has a clobber read:
; `%ret = load ptr, ptr %struct.byte.8`


%struct.type = type { ptr, ptr }

define ptr @foo(ptr noundef %ptr) {
; CHECK-LABEL: define ptr @foo(
; CHECK-SAME: ptr noundef [[PTR:%.*]]) {
; CHECK-NEXT: [[STRUCT_ALLOCA:%.*]] = alloca [[STRUCT_TYPE:%.*]], align 8
; CHECK-NEXT: call void @llvm.lifetime.start.p0(i64 56, ptr nonnull [[STRUCT_ALLOCA]]) #[[ATTR6:[0-9]+]]
; CHECK-NEXT: [[STRUCT_BYTE_8:%.*]] = getelementptr inbounds i8, ptr [[STRUCT_ALLOCA]], i64 8
; CHECK-NEXT: [[TMP1:%.*]] = getelementptr inbounds i8, ptr [[STRUCT_BYTE_8]], i64 4
; CHECK-NEXT: call void @llvm.memset.p0.i64(ptr noundef nonnull align 4 [[TMP1]], i8 42, i64 4, i1 false)
; CHECK-NEXT: store i32 43, ptr [[STRUCT_BYTE_8]], align 4
; CHECK-NEXT: [[RET:%.*]] = load ptr, ptr [[STRUCT_BYTE_8]], align 8
; CHECK-NEXT: call void @llvm.lifetime.end.p0(i64 56, ptr nonnull [[STRUCT_ALLOCA]]) #[[ATTR6]]
; CHECK-NEXT: ret ptr [[RET]]
;
%struct.alloca = alloca %struct.type, align 8
call void @llvm.lifetime.start.p0(i64 56, ptr nonnull %struct.alloca) nounwind
%struct.byte.8 = getelementptr inbounds i8, ptr %struct.alloca, i64 8
; Set %struct.alloca[8, 16) to 42.
call void @llvm.memset.p0.i64(ptr noundef nonnull align 4 %struct.byte.8, i8 42, i64 8, i1 false)
; Set %struct.alloca[8, 12) to 43.
store i32 43, ptr %struct.byte.8, align 4
; Set %struct.alloca[4, 8) to 44.
%struct.byte.4 = getelementptr inbounds i8, ptr %struct.alloca, i64 4
store i32 44, ptr %struct.byte.4, align 4
; Return %struct.alloca[8, 16).
%ret = load ptr, ptr %struct.byte.8
call void @llvm.lifetime.end.p0(i64 56, ptr nonnull %struct.alloca) nounwind
ret ptr %ret
}

; Set of tests based on @foo, but where the memset's operands cannot be erased
; due to other uses. Instead, they contain a number of removable MemoryDefs;
; with non-void types result types.

define ptr @foo_with_removable_malloc() {
; CHECK-LABEL: define ptr @foo_with_removable_malloc() {
; CHECK-NEXT: [[STRUCT_ALLOCA:%.*]] = alloca [[STRUCT_TYPE:%.*]], align 8
; CHECK-NEXT: call void @llvm.lifetime.start.p0(i64 56, ptr nonnull [[STRUCT_ALLOCA]]) #[[ATTR6]]
; CHECK-NEXT: [[STRUCT_BYTE_4:%.*]] = getelementptr inbounds i8, ptr [[STRUCT_ALLOCA]], i64 4
; CHECK-NEXT: [[STRUCT_BYTE_8:%.*]] = getelementptr inbounds i8, ptr [[STRUCT_ALLOCA]], i64 8
; CHECK-NEXT: [[TMP1:%.*]] = getelementptr inbounds i8, ptr [[STRUCT_BYTE_8]], i64 4
; CHECK-NEXT: call void @llvm.memset.p0.i64(ptr noundef nonnull align 4 [[TMP1]], i8 42, i64 4, i1 false)
; CHECK-NEXT: store i32 43, ptr [[STRUCT_BYTE_8]], align 4
; CHECK-NEXT: [[RET:%.*]] = load ptr, ptr [[STRUCT_BYTE_8]], align 8
; CHECK-NEXT: call void @readnone(ptr [[STRUCT_BYTE_4]])
; CHECK-NEXT: call void @readnone(ptr [[STRUCT_BYTE_8]])
; CHECK-NEXT: call void @llvm.lifetime.end.p0(i64 56, ptr nonnull [[STRUCT_ALLOCA]]) #[[ATTR6]]
; CHECK-NEXT: ret ptr [[RET]]
;
%struct.alloca = alloca %struct.type, align 8
call void @llvm.lifetime.start.p0(i64 56, ptr nonnull %struct.alloca) nounwind
%struct.byte.4 = getelementptr inbounds i8, ptr %struct.alloca, i64 4
%struct.byte.8 = getelementptr inbounds i8, ptr %struct.alloca, i64 8

; Set of removable memory deffs
%m2 = tail call ptr @malloc(i64 4)
%m1 = tail call ptr @malloc(i64 4)
store i32 0, ptr %struct.byte.8
store i32 0, ptr %struct.byte.8
store i32 123, ptr %m1
store i32 123, ptr %m2

; Set %struct.alloca[8, 16) to 42.
call void @llvm.memset.p0.i64(ptr noundef nonnull align 4 %struct.byte.8, i8 42, i64 8, i1 false)
; Set %struct.alloca[8, 12) to 43.
store i32 43, ptr %struct.byte.8, align 4
; Set %struct.alloca[4, 8) to 44.
store i32 44, ptr %struct.byte.4, align 4
; Return %struct.alloca[8, 16).
%ret = load ptr, ptr %struct.byte.8
call void @readnone(ptr %struct.byte.4);
call void @readnone(ptr %struct.byte.8);
call void @llvm.lifetime.end.p0(i64 56, ptr nonnull %struct.alloca) nounwind
ret ptr %ret
}

define ptr @foo_with_removable_malloc_free() {
; CHECK-LABEL: define ptr @foo_with_removable_malloc_free() {
; CHECK-NEXT: [[STRUCT_ALLOCA:%.*]] = alloca [[STRUCT_TYPE:%.*]], align 8
; CHECK-NEXT: [[M1:%.*]] = tail call ptr @malloc(i64 4)
; CHECK-NEXT: call void @llvm.lifetime.start.p0(i64 56, ptr nonnull [[STRUCT_ALLOCA]]) #[[ATTR6]]
; CHECK-NEXT: [[STRUCT_BYTE_4:%.*]] = getelementptr inbounds i8, ptr [[STRUCT_ALLOCA]], i64 4
; CHECK-NEXT: [[STRUCT_BYTE_8:%.*]] = getelementptr inbounds i8, ptr [[STRUCT_ALLOCA]], i64 8
; CHECK-NEXT: [[M2:%.*]] = tail call ptr @malloc(i64 4)
; CHECK-NEXT: call void @free(ptr [[M1]])
; CHECK-NEXT: call void @free(ptr [[M2]])
; CHECK-NEXT: [[TMP1:%.*]] = getelementptr inbounds i8, ptr [[STRUCT_BYTE_8]], i64 4
; CHECK-NEXT: call void @llvm.memset.p0.i64(ptr noundef nonnull align 4 [[TMP1]], i8 42, i64 4, i1 false)
; CHECK-NEXT: store i32 43, ptr [[STRUCT_BYTE_8]], align 4
; CHECK-NEXT: [[RET:%.*]] = load ptr, ptr [[STRUCT_BYTE_8]], align 8
; CHECK-NEXT: call void @readnone(ptr [[STRUCT_BYTE_4]])
; CHECK-NEXT: call void @readnone(ptr [[STRUCT_BYTE_8]])
; CHECK-NEXT: call void @llvm.lifetime.end.p0(i64 56, ptr nonnull [[STRUCT_ALLOCA]]) #[[ATTR6]]
; CHECK-NEXT: ret ptr [[RET]]
;
%struct.alloca = alloca %struct.type, align 8
%m1 = tail call ptr @malloc(i64 4)
call void @llvm.lifetime.start.p0(i64 56, ptr nonnull %struct.alloca) nounwind
%struct.byte.4 = getelementptr inbounds i8, ptr %struct.alloca, i64 4
%struct.byte.8 = getelementptr inbounds i8, ptr %struct.alloca, i64 8

store i32 0, ptr %struct.byte.4
store i32 0, ptr %struct.byte.8
%m2 = tail call ptr @malloc(i64 4)
store i32 123, ptr %m1
call void @free(ptr %m1);
store i32 123, ptr %m2
call void @free(ptr %m2);

; Set %struct.alloca[8, 16) to 42.
call void @llvm.memset.p0.i64(ptr noundef nonnull align 4 %struct.byte.8, i8 42, i64 8, i1 false)
; Set %struct.alloca[8, 12) to 43.
store i32 43, ptr %struct.byte.8, align 4
; Set %struct.alloca[4, 8) to 44.
store i32 44, ptr %struct.byte.4, align 4
; Return %struct.alloca[8, 16).
%ret = load ptr, ptr %struct.byte.8
call void @readnone(ptr %struct.byte.4);
call void @readnone(ptr %struct.byte.8);
call void @llvm.lifetime.end.p0(i64 56, ptr nonnull %struct.alloca) nounwind
ret ptr %ret
}

define ptr @foo_with_malloc_to_calloc() {
; CHECK-LABEL: define ptr @foo_with_malloc_to_calloc() {
; CHECK-NEXT: [[STRUCT_ALLOCA:%.*]] = alloca [[STRUCT_TYPE:%.*]], align 8
; CHECK-NEXT: call void @llvm.lifetime.start.p0(i64 56, ptr nonnull [[STRUCT_ALLOCA]]) #[[ATTR6]]
; CHECK-NEXT: [[STRUCT_BYTE_8:%.*]] = getelementptr inbounds i8, ptr [[STRUCT_ALLOCA]], i64 8
; CHECK-NEXT: [[STRUCT_BYTE_4:%.*]] = getelementptr inbounds i8, ptr [[STRUCT_ALLOCA]], i64 4
; CHECK-NEXT: [[CALLOC1:%.*]] = call ptr @calloc(i64 1, i64 4)
; CHECK-NEXT: [[CALLOC:%.*]] = call ptr @calloc(i64 1, i64 4)
; CHECK-NEXT: [[TMP1:%.*]] = getelementptr inbounds i8, ptr [[STRUCT_BYTE_8]], i64 4
; CHECK-NEXT: call void @llvm.memset.p0.i64(ptr noundef nonnull align 4 [[TMP1]], i8 42, i64 4, i1 false)
; CHECK-NEXT: store i32 43, ptr [[STRUCT_BYTE_8]], align 4
; CHECK-NEXT: [[RET:%.*]] = load ptr, ptr [[STRUCT_BYTE_8]], align 8
; CHECK-NEXT: call void @readnone(ptr [[STRUCT_BYTE_4]])
; CHECK-NEXT: call void @readnone(ptr [[STRUCT_BYTE_8]])
; CHECK-NEXT: call void @llvm.lifetime.end.p0(i64 56, ptr nonnull [[STRUCT_ALLOCA]]) #[[ATTR6]]
; CHECK-NEXT: call void @use(ptr [[CALLOC1]])
; CHECK-NEXT: call void @use(ptr [[CALLOC]])
; CHECK-NEXT: ret ptr [[RET]]
;
%struct.alloca = alloca %struct.type, align 8
call void @llvm.lifetime.start.p0(i64 56, ptr nonnull %struct.alloca) nounwind
%struct.byte.8 = getelementptr inbounds i8, ptr %struct.alloca, i64 8
%struct.byte.4 = getelementptr inbounds i8, ptr %struct.alloca, i64 4

; Set of removable memory deffs
%m1 = tail call ptr @malloc(i64 4)
%m2 = tail call ptr @malloc(i64 4)
store i32 0, ptr %struct.byte.4
store i32 0, ptr %struct.byte.8
call void @llvm.memset.p0.i64(ptr noundef nonnull align 4 %m2, i8 0, i64 4, i1 false)
call void @llvm.memset.p0.i64(ptr noundef nonnull align 4 %m1, i8 0, i64 4, i1 false)

; Set %struct.alloca[8, 16) to 42.
call void @llvm.memset.p0.i64(ptr noundef nonnull align 4 %struct.byte.8, i8 42, i64 8, i1 false)
; Set %struct.alloca[8, 12) to 43.
store i32 43, ptr %struct.byte.8, align 4
; Set %struct.alloca[4, 8) to 44.
store i32 44, ptr %struct.byte.4, align 4
; Return %struct.alloca[8, 16).
%ret = load ptr, ptr %struct.byte.8
call void @readnone(ptr %struct.byte.4);
call void @readnone(ptr %struct.byte.8);
call void @llvm.lifetime.end.p0(i64 56, ptr nonnull %struct.alloca) nounwind
call void @use(ptr %m1)
call void @use(ptr %m2)
ret ptr %ret
}

declare void @llvm.memset.p0.i64(ptr nocapture writeonly, i8, i64, i1 immarg)
declare void @llvm.lifetime.end.p0(i64 immarg, ptr nocapture)
declare void @llvm.lifetime.start.p0(i64 immarg, ptr nocapture)

declare noalias ptr @malloc(i64) willreturn allockind("alloc,uninitialized") "alloc-family"="malloc"
declare void @readnone(ptr) readnone nounwind
declare void @free(ptr nocapture) allockind("free") "alloc-family"="malloc"

declare void @use(ptr)