Skip to content

Conversation

grosser
Copy link
Contributor

@grosser grosser commented Jan 13, 2023

What type of PR is this?

/kind bug

What this PR does / why we need it:

while working on #5411 to fix #4095 I realized that the core bug is that we decrement the currentSize (aka asg desired size) when terminating an instance
... but that's now how aws works, terminating an instance does not decreases the desired size, it only makes a new instance come up

(#5411 is still useful since it reduces api call overhead and log spam)

partially reverts #1378

/cc @alam0rt @johanneswuerbach @aleksandra-malinowska

Which issue(s) this PR fixes:

Fixes #4095

Special notes for your reviewer:

Does this PR introduce a user-facing change?

NONE

@k8s-ci-robot k8s-ci-robot added the kind/bug Categorizes issue or PR as related to a bug. label Jan 13, 2023
@k8s-ci-robot
Copy link
Contributor

@grosser: GitHub didn't allow me to request PR reviews from the following users: alam0rt, johanneswuerbach.

Note that only kubernetes members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

What type of PR is this?

/kind bug

What this PR does / why we need it:

while working on #5411 to fix #4095 I realized that the core bug is that we decrement the currentSize (aka asg desired size) when terminating an instance
... but that's now how aws works, terminating an instance does not decreates the desired size, it only makes a new instance come up

(#5411 is still useful since it reduces api call overhead and log spam)

partially reverts #1378

/cc @alam0rt @johanneswuerbach @aleksandra-malinowska

Which issue(s) this PR fixes:

Fixes #4095

Special notes for your reviewer:

Does this PR introduce a user-facing change?

NONE

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Jan 13, 2023
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: grosser
Once this PR has been reviewed and has the lgtm label, please assign gjtempleton for approval by writing /assign @gjtempleton in a comment. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@johanneswuerbach
Copy link
Contributor

johanneswuerbach commented Jan 14, 2023

terminating an instance does not decreases the desired size, it only makes a new instance come up

The terminate call above is made with ShouldDecrementDesiredCapacity: aws.Bool(true), so I think it actually does both, terminate the instance and decreases the desired size.

@grosser
Copy link
Contributor Author

grosser commented Jan 14, 2023

ah missed that, thx!

@grosser grosser closed this Jan 14, 2023
@grosser grosser deleted the grosser/decrement branch January 14, 2023 17:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/cluster-autoscaler cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[cluster-autoscaler][AWS] Repeated deletion every 10s of same node takes AWS ASG down to min size killing multiple running pods

3 participants