Skip to content

[UBP] Duplicate ledger reset entries #14973

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Tracked by #14969
jldec opened this issue Nov 25, 2022 · 3 comments · Fixed by #15006
Closed
Tracked by #14969

[UBP] Duplicate ledger reset entries #14973

jldec opened this issue Nov 25, 2022 · 3 comments · Fixed by #15006
Assignees
Labels
team: webapp Issue belongs to the WebApp team type: bug Something isn't working

Comments

@jldec
Copy link
Contributor

jldec commented Nov 25, 2022

A couple of usage accounts show duplicate reset entries in the ledger.
The impact in the cases below is an incorrect balance with extra credits in favor of the user or team.
More details in discussion (internal)

Attribution IDs below modified for privacy.

[{
  "id": "b5106ec4-bb77-4cc6-8d5e-59936725b3f9",
  "attributionId": "user:xxx",
  "description": "Credits",
  "creditCents": "-8722",
  "effectiveTime": "2022-11-16T12:11:38.545Z",
  "kind": "invoice",
  "workspaceInstanceId": null,
  "draft": "0",
  "metadata": null,
  "_created": "2022-11-16 12:11:38.556265 UTC",
  "_lastModified": "2022-11-16 12:11:38.556265 UTC"
}, {
  "id": "467cb50c-ef9b-4b3d-9914-ca53766dc046",
  "attributionId": "user:xxx",
  "description": "Credits",
  "creditCents": "-8722",
  "effectiveTime": "2022-11-16T12:11:38.510Z",
  "kind": "invoice",
  "workspaceInstanceId": null,
  "draft": "0",
  "metadata": null,
  "_created": "2022-11-16 12:11:38.537253 UTC",
  "_lastModified": "2022-11-16 12:11:38.537253 UTC"
}]

and

[{
  "id": "329243bf-d090-48ef-8bd5-604682b18ab7",
  "attributionId": "team:yyy",
  "description": "Credits",
  "creditCents": "-19539",
  "effectiveTime": "2022-11-10T17:42:15.226Z",
  "kind": "invoice",
  "workspaceInstanceId": null,
  "draft": "0",
  "metadata": null,
  "_created": "2022-11-10 17:42:15.408721 UTC",
  "_lastModified": "2022-11-16 16:26:01.661318 UTC"
}, {
  "id": "60c862f8-98b1-45fb-bfbe-80210ddf7b87",
  "attributionId": "team:yyy",
  "description": "Credits",
  "creditCents": "-19539",
  "effectiveTime": "2022-11-10T17:42:15.267Z",
  "kind": "invoice",
  "workspaceInstanceId": null,
  "draft": "0",
  "metadata": null,
  "_created": "2022-11-10 17:42:15.408987 UTC",
  "_lastModified": "2022-11-16 16:26:01.661318 UTC"
}]
@jldec jldec added type: bug Something isn't working team: webapp Issue belongs to the WebApp team labels Nov 25, 2022
@jldec jldec moved this to Scheduled in 🍎 WebApp Team Nov 25, 2022
@svenefftinge svenefftinge self-assigned this Nov 28, 2022
@svenefftinge svenefftinge moved this from Scheduled to In Progress in 🍎 WebApp Team Nov 28, 2022
@svenefftinge
Copy link
Member

svenefftinge commented Nov 28, 2022

Here are the observations and actions:

  • We don't run the resetUsage at all in saas. This is because we disable the default 15ms in US (explicitly) and in EU (implicitly by not specifying something). We probably wanted to use the default 15 min, but since not specifying anything means empty string we disable always. This also means that a default can never be used as it always get's overwritten by what's coming from the usageConfig. (fix)

  • Since the resetUsage job doesn't run, the duplicate entries might come from the only other place where we run resetUsage. Which is on getOrCreateCostCenter. If two calls to this endpoint happen in parallel there's a rather large window for race. (fix)

  • I'll also check the DB once more if we still see the duplicate entries or whether it was due to a situation in early November @easyCZ mentioned, where two jobs ran in parallel. Edit: We have a lot of such dupülicate entries on November 4th but also some (eight in total) on later days, so I assume there is a possible race.

Repository owner moved this from In Progress to In Validation in 🍎 WebApp Team Nov 28, 2022
@svenefftinge svenefftinge moved this from In Validation to Done in 🍎 WebApp Team Nov 28, 2022
@jldec
Copy link
Contributor Author

jldec commented Dec 27, 2022

Another occurrence on Dec 15.

[{
  "id": "ebfb0215-625c-473b-b559-91d6bc20106a",
  "attributionId": "user:...",
  "description": "Credits",
  "creditCents": "-50000",
  "effectiveTime": "2022-12-15T09:23:41.666Z",
  "kind": "invoice",
  "workspaceInstanceId": null,
  "draft": "0",
  "metadata": null,
  "_created": "2022-12-15 09:23:41.724552 UTC",
  "_lastModified": "2022-12-15 09:23:41.724552 UTC"
}, {
  "id": "f0b6eeee-8489-444c-9315-24071fcc89ba",
  "attributionId": "user:...",
  "description": "Credits",
  "creditCents": "-50000",
  "effectiveTime": "2022-12-15T09:23:41.653Z",
  "kind": "invoice",
  "workspaceInstanceId": null,
  "draft": "0",
  "metadata": null,
  "_created": "2022-12-15 09:23:41.727564 UTC",
  "_lastModified": "2022-12-15 09:23:41.727564 UTC"
}]

@jldec jldec reopened this Dec 27, 2022
@github-project-automation github-project-automation bot moved this from Done to Scheduled in 🍎 WebApp Team Dec 27, 2022
@geropl
Copy link
Member

geropl commented Jan 23, 2023

Closing this issue because from this and other context I assume it's done.

@geropl geropl closed this as completed Jan 23, 2023
@github-project-automation github-project-automation bot moved this from Scheduled to In Validation in 🍎 WebApp Team Jan 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
team: webapp Issue belongs to the WebApp team type: bug Something isn't working
Projects
Status: In Validation
Development

Successfully merging a pull request may close this issue.

3 participants