[DPE-5269] Add passwords to Syncobj and Patroni #596

dragomirp · 2024-08-27T14:24:24Z

Add passwords to Patroni and Syncobj
Update libs

Closes #27

codecov · 2024-08-27T14:27:20Z

Codecov Report

Attention: Patch coverage is 68.57143% with 11 lines in your changes missing coverage. Please review.

Project coverage is 70.85%. Comparing base (0c855f4) to head (e16a2f6).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
src/relations/async_replication.py	16.66%	5 Missing ⚠️
src/cluster.py	75.00%	4 Missing ⚠️
src/upgrade.py	66.66%	0 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #596      +/-   ##
==========================================
- Coverage   70.89%   70.85%   -0.04%     
==========================================
  Files          12       12              
  Lines        3030     3043      +13     
  Branches      536      538       +2     
==========================================
+ Hits         2148     2156       +8     
- Misses        768      771       +3     
- Partials      114      116       +2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

templates/patroni.yml.j2

Co-authored-by: Mykola Marzhan <[email protected]>

dragomirp · 2024-08-28T11:02:57Z

actions.yaml

@@ -27,7 +27,7 @@ get-password:
    username:
      type: string
      description: The username, the default value 'operator'.
-        Possible values - operator, replication, rewind.
+        Possible values - operator, replication, rewind, patroni.


Users and tests need to be able to get at least the patroni password.

dragomirp · 2024-08-28T11:03:37Z

src/charm.py

+        for key in (
+            USER_PASSWORD_KEY,
+            REPLICATION_PASSWORD_KEY,
+            REWIND_PASSWORD_KEY,
+            MONITORING_PASSWORD_KEY,
+            RAFT_PASSWORD_KEY,
+            PATRONI_PASSWORD_KEY,
+        ):
+            if self.get_secret(APP_SCOPE, key) is None:
+                self.set_secret(APP_SCOPE, key, new_password())


Loop to reduce McCabe.

dragomirp · 2024-08-28T11:04:36Z

src/cluster.py

@@ -164,6 +174,7 @@ def cluster_members(self) -> set:
            f"{self._patroni_url}/{PATRONI_CLUSTER_STATUS_ENDPOINT}",
            verify=self.verify,
            timeout=API_REQUEST_TIMEOUT,
+            auth=self._patroni_auth,


GETs are usually open, but adding for consistency.

dragomirp · 2024-08-28T11:07:56Z

src/relations/async_replication.py

+        if self.charm.unit.is_leader():
+            return self.charm.model.app.add_secret(content=shared_content, label=SECRET_LABEL)


Started hitting this on follower units

dragomirp · 2024-08-28T11:47:22Z

src/upgrade.py

+        if self.charm.unit.is_leader() and self.charm._peers:
+            for key in (
+                RAFT_PASSWORD_KEY,
+                PATRONI_PASSWORD_KEY,
+            ):
+                if self.charm.get_secret(APP_SCOPE, key) is None:
+                    self.charm.set_secret(APP_SCOPE, key, new_password())
+


Is this the correct place for generating the passes?

^ @marceloneppel

Yeah, it's ok to put it here.

taurus-forever

LGTM. Thank you for the quick fix!
Let's wait @marceloneppel before merging.
P.S. we need to backport Patroni part to K8s charm :-(

taurus-forever · 2024-08-28T12:14:54Z

src/upgrade.py

+        if self.charm.unit.is_leader() and self.charm._peers:
+            for key in (
+                RAFT_PASSWORD_KEY,
+                PATRONI_PASSWORD_KEY,
+            ):
+                if self.charm.get_secret(APP_SCOPE, key) is None:
+                    self.charm.set_secret(APP_SCOPE, key, new_password())
+


^ @marceloneppel

marceloneppel

LGTM! Thanks @dragomirp!

marceloneppel · 2024-08-28T17:36:47Z

src/upgrade.py

+        if self.charm.unit.is_leader() and self.charm._peers:
+            for key in (
+                RAFT_PASSWORD_KEY,
+                PATRONI_PASSWORD_KEY,
+            ):
+                if self.charm.get_secret(APP_SCOPE, key) is None:
+                    self.charm.set_secret(APP_SCOPE, key, new_password())
+


Yeah, it's ok to put it here.

dragomirp · 2024-08-28T21:00:49Z

src/cluster.py

+                        # If raft is getting encrypted some of the calls will fail
+                        if member_status.status_code == 503 and raft_encryption:
+                            logger.warning(
+                                "Failed replication check for %s during raft encryption"
+                                % members_ip
+                            )
+                            continue


Still executing to have warnings in the log

dragomirp · 2024-08-28T21:02:20Z

src/upgrade.py

+        raft_encryption = (
+            int(
+                json.loads(self.peer_relation.data[self.charm.app].get("dependencies", "{}"))
+                .get("charm", {})
+                .get("version", 0)
+            )
+            < 3
+        )


Check for versions that don't encrypt RAFT. Seems to be only an issue for 1 (current 14/stable) but adding both previous version just in case.

taurus-forever

LGTM, Thank you!

Add passowrds to Syncobj and Patroni

fe6ace9

github-actions bot added the Libraries: Out of sync label Aug 27, 2024

Pass pass

60df348

taurus-forever changed the title ~~[DPE-5269] Add passowrds to Syncobj and Patroni~~ [DPE-5269] Add passwords to Syncobj and Patroni Aug 27, 2024

dragomirp added 2 commits August 27, 2024 19:12

Generate passwords during upgrade

8741770

Get password

401ed34

github-actions bot added Libraries: OK and removed Libraries: Out of sync labels Aug 27, 2024

dragomirp added 4 commits August 27, 2024 20:33

Update tests

30266fb

Fix tests

0cb9aa8

Try to regenerate secrets

736d9ab

Check for peers availability before resetting the passes

a00b892

delgod requested changes Aug 28, 2024

View reviewed changes

templates/patroni.yml.j2 Outdated Show resolved Hide resolved

dragomirp and others added 3 commits August 28, 2024 11:55

Update templates/patroni.yml.j2

d47071f

Co-authored-by: Mykola Marzhan <[email protected]>

Switch patroni user

2c49aa8

Only try to create secret on leader

fc6ab55

dragomirp commented Aug 28, 2024

View reviewed changes

dragomirp marked this pull request as ready for review August 28, 2024 11:08

dragomirp requested review from delgod, a team, taurus-forever, marceloneppel and lucasgameiroborges and removed request for a team August 28, 2024 11:08

delgod approved these changes Aug 28, 2024

View reviewed changes

Add replication check logging

b8e736f

Try to observe topology without auth

2880e36

dragomirp commented Aug 28, 2024

View reviewed changes

taurus-forever approved these changes Aug 28, 2024

View reviewed changes

Try to ignore upgrade replication error

4ed4564

marceloneppel approved these changes Aug 28, 2024

View reviewed changes

lucasgameiroborges approved these changes Aug 28, 2024

View reviewed changes

dragomirp added 4 commits August 28, 2024 22:51

Ignore replication only when encrypting raft

eb035b8

Cast version

08b0b60

Merge branch 'main' into dpe-5269-add-patroni-passwords

57389ad

Enable arm upgrade from stable

e16a2f6

dragomirp commented Aug 28, 2024

View reviewed changes

dragomirp requested review from taurus-forever, marceloneppel and lucasgameiroborges August 28, 2024 21:04

dragomirp mentioned this pull request Aug 29, 2024

[DPE-5275] patroni passwords canonical/postgresql-k8s-operator#661

Merged

taurus-forever approved these changes Aug 29, 2024

View reviewed changes

dragomirp merged commit 1ed5706 into main Aug 29, 2024
83 checks passed

dragomirp deleted the dpe-5269-add-patroni-passwords branch August 29, 2024 09:23

		if self.charm.unit.is_leader():
		return self.charm.model.app.add_secret(content=shared_content, label=SECRET_LABEL)

[DPE-5269] Add passwords to Syncobj and Patroni #596

[DPE-5269] Add passwords to Syncobj and Patroni #596

Uh oh!

Conversation

dragomirp commented Aug 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Aug 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

taurus-forever left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

marceloneppel left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

taurus-forever left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

dragomirp commented Aug 27, 2024 •

edited

Loading

codecov bot commented Aug 27, 2024 •

edited

Loading

taurus-forever left a comment •

edited

Loading