Skip to content

Make memberlist KV a service. #2713

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 12 commits into from
Jun 16, 2020
Merged

Make memberlist KV a service. #2713

merged 12 commits into from
Jun 16, 2020

Conversation

pstibrany
Copy link
Contributor

@pstibrany pstibrany commented Jun 11, 2020 •
edited
Loading

What this PR does: This PR modifies memberlist.KV into a Cortex service:

  • it needs to be started to start gossiping part
  • if configured to join the cluster and abort if join fails, Starting can result in Failed state
  • KV clients now wait for service to enter Running or Stopping state. (Stopping because we still receive updates until cluster is left)

This is first step in adding periodic rejoins (#2638), and should also make #2705 bit easier (we can use backoff in Starting state, and not block NewKV)

Update:

Checklist

  • Tests updated

@pstibrany pstibrany mentioned this pull request Jun 11, 2020
Merged
3 tasks
pstibrany added 8 commits June 15, 2020 09:03
@pstibrany
Make memberlist KV a service.
1fd88ad 
It doesn't start gossiping part until service is started.
If join members on startup is configured with abort flag, service can also fail.

KV Clients wait until service is running.

Signed-off-by: Peter Štibraný <[email protected]>
@pstibrany
Comment.
06af9c7 
Signed-off-by: Peter Štibraný <[email protected]>
@pstibrany
Comments, added check for JoinMembers.
6639da9 
Signed-off-by: Peter Štibraný <[email protected]>
@pstibrany
Fix invalid metrics.
3b68332 
Signed-off-by: Peter Štibraný <[email protected]>
@pstibrany
Public JoinMembers can only be used in Running state, so starting met…
f15d15e 
…hod should use memberlist directly.

Signed-off-by: Peter Štibraný <[email protected]>
@pstibrany
Enable debug log to see why integration tests on CI fail.
3dbd16e 
Signed-off-by: Peter Štibraný <[email protected]>
@pstibrany
Joining members with backoff now happens in Starting.
a996b93 
Abort flag is respected if joining (with backoff) fails.

Signed-off-by: Peter Štibraný <[email protected]>
@pstibrany
Put initWG back.
f8d2e1f 
Signed-off-by: Peter Štibraný <[email protected]>
@pstibrany pstibrany mentioned this pull request Jun 15, 2020
Merged
3 tasks
@pstibrany pstibrany mentioned this pull request Jun 15, 2020
Merged
3 tasks
pracucci
pracucci approved these changes Jun 16, 2020
View reviewed changes
Copy link
Contributor

@pracucci pracucci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job @pstibrany! I think there's a little bug in joinMembersOnStartup(). A part from this (and a couple of nits), LGTM.

pstibrany added 2 commits June 16, 2020 11:50
@pstibrany
Address feedback from review.
a88efab 
Signed-off-by: Peter Štibraný <[email protected]>
@pstibrany
Added test for joining cluster error.
81b954d 
Signed-off-by: Peter Štibraný <[email protected]>
@pstibrany pstibrany merged commit 15a7121 into cortexproject:master Jun 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
1 more reviewer

@pracucci pracucci pracucci approved these changes

Reviewers whose approvals may not affect merge requirements
Assignees
No one assigned
Labels
size/L
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@pstibrany @pracucci