Cleaner would delete bucket index when there is no block in bucket store #6577
Conversation
… block in bucket store Signed-off-by: Alex Le <[email protected]>
Signed-off-by: Alex Le <[email protected]>
| return err | ||
| if idx.IsEmpty() && len(partials) == 0 { | ||
| level.Info(userLogger).Log("msg", "deleting bucket index since it is empty") | ||
| if err := bucketindex.DeleteIndex(ctx, c.bucketClient, userID, c.cfgProvider); err != nil { |
There was a problem hiding this comment.
Unrelated to this PR. Do we need to handle AccessDeniederror in bucketindex.DeleteIndex and bucketindex.DeleteIndexSyncStatus?
I see they only handle object not found
There was a problem hiding this comment.
Cortex itself cannot handle AccessDenied. Once it happens, cleaner should fail the process and enduser should take some action on it.
| begin = time.Now() | ||
| if err := bucketindex.WriteIndex(ctx, c.bucketClient, userID, c.cfgProvider, idx); err != nil { | ||
| return err | ||
| if idx.IsEmpty() && len(partials) == 0 { |
There was a problem hiding this comment.
If the bucket index is empty but we have partial blocks, we are going to still write the index. Are we waiting for cleaner to clean up partial blocks and eventually we will delete the empty bucket index?
There was a problem hiding this comment.
That's right. If there are partial blocks, we need cleaner to delete those first. Bucket index file should be deleted if it is the only file in bucket store for a tenant.
Signed-off-by: Alex Le <[email protected]>
What this PR does:
If there is bucket index file but no blocks in bucket store for one user, cleaner would keep running cleaning logic and update empty bucket index for such user. With this PR, cleaner would delete bucket index and its sync status file if there is no block in bucket store to prevent such dangling user exists in bucket store.
Which issue(s) this PR fixes:
Checklist
CHANGELOG.mdupdated - the order of entries should be[CHANGE],[FEATURE],[ENHANCEMENT],[BUGFIX]