VPA admission controller: TLS handshake error

[Julian-Chu]

Which component are you using?: vpa-admission-controller

What version of the component are you using?:

Component version: 1.0.0

What k8s version are you using (kubectl version)?:

kubectl version Output

$ kubectl version
Client Version: v1.28.1
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.26.14-eks-b9c9ed7
WARNING: version difference between client (1.28) and server (1.26) exceeds the supported minor version skew of +/-1

What environment is this in?:

aws eks 1.26 and 1.27

What did you expect to happen?:

What happened instead?:

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

logs

```shell`` I0420 04:30:08.732144 1 api.go:92] Initial VPA synced successfully I0420 04:30:08.832711 1 fetcher.go:99] Initial sync of Deployment completed I0420 04:30:09.133452 1 fetcher.go:99] Initial sync of ReplicaSet completed I0420 04:30:09.233885 1 fetcher.go:99] Initial sync of StatefulSet completed I0420 04:30:09.335001 1 fetcher.go:99] Initial sync of ReplicationController completed I0420 04:30:09.435531 1 fetcher.go:99] Initial sync of Job completed I0420 04:30:09.536527 1 fetcher.go:99] Initial sync of CronJob completed I0420 04:30:09.637422 1 fetcher.go:99] Initial sync of DaemonSet completed W0420 04:30:09.637540 1 shared_informer.go:419] The sharedIndexInformer has started, run more than once is not allowed W0420 04:30:09.637554 1 shared_informer.go:419] The sharedIndexInformer has started, run more than once is not allowed W0420 04:30:09.637560 1 shared_informer.go:419] The sharedIndexInformer has started, run more than once is not allowed W0420 04:30:09.637566 1 shared_informer.go:419] The sharedIndexInformer has started, run more than once is not allowed W0420 04:30:09.637574 1 shared_informer.go:419] The sharedIndexInformer has started, run more than once is not allowed W0420 04:30:09.637583 1 shared_informer.go:419] The sharedIndexInformer has started, run more than once is not allowed W0420 04:30:09.637590 1 shared_informer.go:419] The sharedIndexInformer has started, run more than once is not allowed 2024/04/20 04:43:44 http: TLS handshake error from 10.17.255.35:53638: EOF 2024/04/20 08:40:00 http: TLS handshake error from 10.17.255.35:59478: EOF 2024/04/20 11:14:27 http: TLS handshake error from 10.17.255.35:50634: EOF 2024/04/20 11:55:46 http: TLS handshake error from 10.17.255.35:57188: EOF 2024/04/20 12:03:28 http: TLS handshake error from 10.17.255.35:40128: EOF 2024/04/20 12:20:08 http: TLS handshake error from 10.17.255.35:40570: EOF 2024/04/20 13:00:57 http: TLS handshake error from 10.17.255.35:55228: EOF 2024/04/20 13:00:58 http: TLS handshake error from 10.17.255.35:55232: EOF 2024/04/20 13:10:00 http: TLS handshake error from 10.17.255.35:53876: EOF 2024/04/20 13:19:19 http: TLS handshake error from 10.17.255.35:35302: EOF 2024/04/20 13:20:00 http: TLS handshake error from 10.17.255.35:34784: EOF 2024/04/20 13:52:27 http: TLS handshake error from 10.17.255.35:51516: EOF 2024/04/20 14:11:59 http: TLS handshake error from 10.17.255.35:39614: EOF 2024/04/20 14:36:53 http: TLS handshake error from 10.17.255.35:52990: EOF 2024/04/20 14:48:01 http: TLS handshake error from 10.17.255.35:48134: EOF 2024/04/20 14:52:50 http: TLS handshake error from 10.17.255.35:52966: EOF 2024/04/20 14:57:21 http: TLS handshake error from 10.17.255.35:57810: read tcp 10.16.219.245:8000->10.17.255.35:57810: read: connection reset by peer 2024/04/20 15:10:31 http: TLS handshake error from 10.17.255.35:46840: EOF 2024/04/20 15:13:54 http: TLS handshake error from 10.17.255.35:59258: EOF 2024/04/20 15:14:26 http: TLS handshake error from 10.17.255.35:55936: EOF 2024/04/20 15:17:52 http: TLS handshake error from 10.17.255.35:54648: EOF 2024/04/20 15:22:18 http: TLS handshake error from 10.17.255.35:43770: read tcp 10.16.219.245:8000->10.17.255.35:43770: read: connection reset by peer 2024/04/20 15:30:50 http: TLS handshake error from 10.17.255.35:55836: EOF 2024/04/20 15:33:49 http: TLS handshake error from 10.17.255.35:58742: EOF 2024/04/20 15:49:12 http: TLS handshake error from 10.17.255.35:39130: EOF 2024/04/20 15:57:37 http: TLS handshake error from 10.17.255.35:56764: EOF 2024/04/20 16:05:33 http: TLS handshake error from 10.17.255.35:42756: EOF 2024/04/20 16:10:00 http: TLS handshake error from 10.17.255.35:43746: EOF 2024/04/20 16:51:12 http: TLS handshake error from 10.17.255.35:50468: EOF 2024/04/20 17:24:59 http: TLS handshake error from 10.17.253.13:36266: read tcp 10.16.219.245:8000->10.17.253.13:36266: read: connection reset by peer 2024/04/20 20:43:24 http: TLS handshake error from 10.17.253.13:38958: EOF 2024/04/20 21:07:26 http: TLS handshake error from 10.17.253.13:47180: read tcp 10.16.219.245:8000->10.17.253.13:47180: read: connection reset by peer 2024/04/20 21:08:16 http: TLS handshake error from 10.17.253.13:56432: read tcp 10.16.219.245:8000->10.17.253.13:56432: read: connection reset by peer 2024/04/20 21:41:00 http: TLS handshake error from 10.17.253.13:56622: EOF 2024/04/20 21:41:33 http: TLS handshake error from 10.17.253.13:48220: EOF 2024/04/20 22:46:18 http: TLS handshake error from 10.17.253.13:36370: EOF 2024/04/20 22:46:34 http: TLS handshake error from 10.17.253.13:54754: EOF 2024/04/20 22:46:48 http: TLS handshake error from 10.17.253.13:42384: EOF 2024/04/20 22:46:49 http: TLS handshake error from 10.17.253.13:42398: EOF 2024/04/20 22:53:04 http: TLS handshake error from 10.17.253.13:53628: EOF 2024/04/20 23:07:07 http: TLS handshake error from 10.17.253.13:39850: EOF 2024/04/20 23:14:45 http: TLS handshake error from 10.17.253.13:44338: read tcp 10.16.219.245:8000->10.17.253.13:44338: read: connection reset by peer 2024/04/20 23:27:34 http: TLS handshake error from 10.17.253.13:42192: EOF 2024/04/20 23:59:00 http: TLS handshake error from 10.17.253.13:58392: read tcp 10.16.219.245:8000->10.17.253.13:58392: read: connection reset by peer 2024/04/21 01:07:35 http: TLS handshake error from 10.17.253.13:58178: EOF 2024/04/21 01:30:00 http: TLS handshake error from 10.17.253.13:37546: EOF 2024/04/21 02:10:00 http: TLS handshake error from 10.17.253.13:59606: EOF 2024/04/21 03:28:13 http: TLS handshake error from 10.17.253.13:56330: EOF 2024/04/21 03:28:38 http: TLS handshake error from 10.17.253.13:57842: EOF 2024/04/21 03:38:42 http: TLS handshake error from 10.17.253.13:57054: read tcp 10.16.219.245:8000->10.17.253.13:57054: read: connection reset by peer 2024/04/21 03:41:27 http: TLS handshake error from 10.17.253.13:41380: EOF 2024/04/21 10:00:00 http: TLS handshake error from 10.17.253.13:40480: read tcp 10.16.219.245:8000->10.17.253.13:40480: read: connection reset by peer 2024/04/21 12:41:30 http: TLS handshake error from 10.17.253.13:47972: read tcp 10.16.219.245:8000->10.17.253.13:47972: read: connection reset by peer 2024/04/21 13:14:35 http: TLS handshake error from 10.17.253.13:57886: EOF 2024/04/21 13:42:02 http: TLS handshake error from 10.17.253.13:60030: EOF 2024/04/21 13:50:00 http: TLS handshake error from 10.17.253.13:48748: EOF 2024/04/21 14:23:39 http: TLS handshake error from 10.17.253.13:43748: EOF 2024/04/21 14:39:42 http: TLS handshake error from 10.17.253.13:35558: EOF 2024/04/21 14:40:00 http: TLS handshake error from 10.17.253.13:50218: EOF 2024/04/21 14:40:38 http: TLS handshake error from 10.17.253.13:45640: EOF 2024/04/21 14:40:56 http: TLS handshake error from 10.17.253.13:49248: EOF 2024/04/21 14:41:01 http: TLS handshake error from 10.17.253.13:49250: EOF 2024/04/21 14:50:00 http: TLS handshake error from 10.17.253.13:52428: EOF 2024/04/21 14:55:29 http: TLS handshake error from 10.17.253.13:54020: EOF 2024/04/21 14:56:12 http: TLS handshake error from 10.17.253.13:56386: read tcp 10.16.219.245:8000->10.17.253.13:56386: read: connection reset by peer 2024/04/21 15:04:28 http: TLS handshake error from 10.17.253.13:54170: read tcp 10.16.219.245:8000->10.17.253.13:54170: read: connection reset by peer 2024/04/21 15:14:39 http: TLS handshake error from 10.17.253.13:44510: EOF 2024/04/21 16:14:58 http: TLS handshake error from 10.17.253.13:57000: EOF 2024/04/21 16:15:00 http: TLS handshake error from 10.17.253.13:57010: EOF 2024/04/21 16:28:38 http: TLS handshake error from 10.17.253.13:56238: EOF 2024/04/21 16:41:01 http: TLS handshake error from 10.17.253.13:35598: EOF 2024/04/21 18:01:31 http: TLS handshake error from 10.17.253.13:50134: read tcp 10.16.219.245:8000->10.17.253.13:50134: read: connection reset by peer 2024/04/21 19:56:00 http: TLS handshake error from 10.17.253.13:36822: read tcp 10.16.219.245:8000->10.17.253.13:36822: read: connection reset by peer 2024/04/21 20:11:25 http: TLS handshake error from 10.17.253.13:55504: EOF 2024/04/21 20:18:52 http: TLS handshake error from 10.17.253.13:54064: EOF 2024/04/21 20:20:00 http: TLS handshake error from 10.17.253.13:48050: read tcp 10.16.219.245:8000->10.17.253.13:48050: read: connection reset by peer 2024/04/21 20:48:33 http: TLS handshake error from 10.17.253.13:44382: read tcp 10.16.219.245:8000->10.17.253.13:44382: read: connection reset by peer 2024/04/21 21:51:41 http: TLS handshake error from 10.17.253.13:52504: EOF 2024/04/21 22:02:40 http: TLS handshake error from 10.17.253.13:41342: read tcp 10.16.219.245:8000->10.17.253.13:41342: read: connection reset by peer 2024/04/21 22:25:25 http: TLS handshake error from 10.17.253.13:58050: EOF 2024/04/21 22:29:08 http: TLS handshake error from 10.17.253.13:60238: read tcp 10.16.219.245:8000->10.17.253.13:60238: read: connection reset by peer 2024/04/21 23:09:17 http: TLS handshake error from 10.17.253.13:54898: EOF 2024/04/21 23:09:18 http: TLS handshake error from 10.17.253.13:54904: EOF 2024/04/21 23:09:18 http: TLS handshake error from 10.17.253.13:54910: EOF 2024/04/21 23:09:18 http: TLS handshake error from 10.17.253.13:54912: EOF 2024/04/21 23:09:18 http: TLS handshake error from 10.17.253.13:54924: EOF 2024/04/21 23:09:19 http: TLS handshake error from 10.17.253.13:54928: EOF 2024/04/21 23:09:19 http: TLS handshake error from 10.17.253.13:54944: EOF 2024/04/21 23:09:19 http: TLS handshake error from 10.17.253.13:54958: EOF 2024/04/21 23:09:19 http: TLS handshake error from 10.17.253.13:54974: read tcp 10.16.219.245:8000->10.17.253.13:54974: read: connection reset by peer 2024/04/22 00:05:30 http: TLS handshake error from 10.17.253.13:49178: EOF 2024/04/22 00:40:00 http: TLS handshake error from 10.17.253.13:32854: EOF 2024/04/22 01:30:00 http: TLS handshake error from 10.17.253.13:48864: EOF 2024/04/22 03:17:00 http: TLS handshake error from 10.17.253.13:42754: read tcp 10.16.219.245:8000->10.17.253.13:42754: read: connection reset by peer 2024/04/22 03:20:00 http: TLS handshake error from 10.17.253.13:59412: read tcp 10.16.219.245:8000->10.17.253.13:59412: read: connection reset by peer 2024/04/22 12:22:33 http: TLS handshake error from 10.17.253.13:57774: EOF 2024/04/22 13:07:22 http: TLS handshake error from 10.17.253.13:58196: EOF 2024/04/22 14:08:41 http: TLS handshake error from 10.17.253.13:52732: EOF 2024/04/22 14:22:13 http: TLS handshake error from 10.17.253.13:53396: EOF 2024/04/22 15:00:00 http: TLS handshake error from 10.17.253.13:36244: read tcp 10.16.219.245:8000->10.17.253.13:36244: read: connection reset by peer 2024/04/22 15:00:34 http: TLS handshake error from 10.17.253.13:40844: read tcp 10.16.219.245:8000->10.17.253.13:40844: read: connection reset by peer 2024/04/22 15:31:37 http: TLS handshake error from 10.17.253.13:50488: EOF 2024/04/22 15:49:25 http: TLS handshake error from 10.17.253.13:38670: EOF 2024/04/22 15:52:56 http: TLS handshake error from 10.17.253.13:46876: read tcp 10.16.219.245:8000->10.17.253.13:46876: read: connection reset by peer 2024/04/22 15:53:25 http: TLS handshake error from 10.17.253.13:55892: EOF 2024/04/22 16:01:57 http: TLS handshake error from 10.17.253.13:47178: EOF 2024/04/22 16:02:24 http: TLS handshake error from 10.17.253.13:33996: read tcp 10.16.219.245:8000->10.17.253.13:33996: read: connection reset by peer 2024/04/22 16:02:24 http: TLS handshake error from 10.17.253.13:34004: read tcp 10.16.219.245:8000->10.17.253.13:34004: read: connection reset by peer 2024/04/22 16:02:25 http: TLS handshake error from 10.17.253.13:34006: EOF 2024/04/22 16:02:25 http: TLS handshake error from 10.17.253.13:34030: EOF 2024/04/22 16:02:25 http: TLS handshake error from 10.17.253.13:34020: read tcp 10.16.219.245:8000->10.17.253.13:34020: read: connection reset by peer 2024/04/22 16:02:25 http: TLS handshake error from 10.17.253.13:34042: EOF 2024/04/22 16:02:26 http: TLS handshake error from 10.17.253.13:34058: read tcp 10.16.219.245:8000->10.17.253.13:34058: read: connection reset by peer 2024/04/22 17:50:00 http: TLS handshake error from 10.17.253.13:42966: read tcp 10.16.219.245:8000->10.17.253.13:42966: read: connection reset by peer 2024/04/22 18:17:23 http: TLS handshake error from 10.17.253.13:50394: read tcp 10.16.219.245:8000->10.17.253.13:50394: read: connection reset by peer 2024/04/22 18:30:12 http: TLS handshake error from 10.17.253.13:57638: read tcp 10.16.219.245:8000->10.17.253.13:57638: read: connection reset by peer 2024/04/22 20:16:23 http: TLS handshake error from 10.17.253.13:41064: read tcp 10.16.219.245:8000->10.17.253.13:41064: read: connection reset by peer 2024/04/22 21:54:56 http: TLS handshake error from 10.17.253.13:53162: EOF 2024/04/22 22:18:44 http: TLS handshake error from 10.17.253.13:43386: EOF 2024/04/22 22:21:56 http: TLS handshake error from 10.17.253.13:56248: EOF 2024/04/22 22:22:32 http: TLS handshake error from 10.17.253.13:41772: EOF 2024/04/22 22:22:32 http: TLS handshake error from 10.17.253.13:41780: EOF 2024/04/22 22:48:34 http: TLS handshake error from 10.17.253.13:45734: EOF 2024/04/22 22:48:35 http: TLS handshake error from 10.17.253.13:45742: EOF 2024/04/22 22:52:12 http: TLS handshake error from 10.17.253.13:52880: read tcp 10.16.219.245:8000->10.17.253.13:52880: read: connection reset by peer 2024/04/22 22:56:39 http: TLS handshake error from 10.17.253.13:40236: read tcp 10.16.219.245:8000->10.17.253.13:40236: read: connection reset by peer 2024/04/22 23:52:43 http: TLS handshake error from 10.17.253.13:37370: EOF 2024/04/23 00:08:11 http: TLS handshake error from 10.17.253.13:39396: EOF 2024/04/23 00:38:54 http: TLS handshake error from 10.17.253.13:49404: EOF 2024/04/23 01:00:00 http: TLS handshake error from 10.17.253.13:47660: read tcp 10.16.219.245:8000->10.17.253.13:47660: read: connection reset by peer 2024/04/23 02:41:35 http: TLS handshake error from 10.17.253.13:43112: EOF 2024/04/23 03:44:58 http: TLS handshake error from 10.17.253.13:41786: EOF 2024/04/23 05:20:00 http: TLS handshake error from 10.17.253.13:58444: EOF 2024/04/23 05:40:00 http: TLS handshake error from 10.17.253.13:40334: read tcp 10.16.219.245:8000->10.17.253.13:40334: read: connection reset by peer 2024/04/23 08:50:00 http: TLS handshake error from 10.17.253.13:55494: EOF 2024/04/23 11:26:19 http: TLS handshake error from 10.17.253.13:42240: read tcp 10.16.219.245:8000->10.17.253.13:42240: read: connection reset by peer 2024/04/23 12:10:00 http: TLS handshake error from 10.17.253.13:43884: EOF 2024/04/23 12:10:35 http: TLS handshake error from 10.17.253.13:57106: EOF 2024/04/23 12:34:24 http: TLS handshake error from 10.17.253.13:51582: EOF 2024/04/23 14:14:29 http: TLS handshake error from 10.17.253.13:39992: EOF 2024/04/23 14:16:44 http: TLS handshake error from 10.17.253.13:42956: EOF 2024/04/23 14:47:17 http: TLS handshake error from 10.17.253.13:55942: EOF 2024/04/23 14:49:46 http: TLS handshake error from 10.17.253.13:50808: EOF 2024/04/23 15:06:08 http: TLS handshake error from 10.17.253.13:57772: EOF 2024/04/23 15:44:08 http: TLS handshake error from 10.17.253.13:42830: read tcp 10.16.219.245:8000->10.17.253.13:42830: read: connection reset by peer 2024/04/23 16:42:24 http: TLS handshake error from 10.17.253.13:50704: EOF 2024/04/23 16:42:24 http: TLS handshake error from 10.17.253.13:50708: read tcp 10.16.219.245:8000->10.17.253.13:50708: read: connection reset by peer 2024/04/23 16:42:25 http: TLS handshake error from 10.17.253.13:50720: EOF 2024/04/23 16:42:25 http: TLS handshake error from 10.17.253.13:50726: EOF 2024/04/23 16:42:25 http: TLS handshake error from 10.17.253.13:50732: EOF 2024/04/23 16:42:25 http: TLS handshake error from 10.17.253.13:50736: EOF 2024/04/23 16:42:25 http: TLS handshake error from 10.17.253.13:50744: read tcp 10.16.219.245:8000->10.17.253.13:50744: read: connection reset by peer 2024/04/23 19:15:00 http: TLS handshake error from 10.17.253.13:42970: EOF 2024/04/23 19:28:12 http: TLS handshake error from 10.17.253.13:50886: EOF 2024/04/23 21:30:07 http: TLS handshake error from 10.17.253.13:56698: EOF 2024/04/23 21:37:20 http: TLS handshake error from 10.17.253.13:43004: read tcp 10.16.219.245:8000->10.17.253.13:43004: read: connection reset by peer 2024/04/23 22:07:27 http: TLS handshake error from 10.17.253.13:34840: EOF 2024/04/23 22:09:18 http: TLS handshake error from 10.17.253.13:43160: read tcp 10.16.219.245:8000->10.17.253.13:43160: read: connection reset by peer 2024/04/23 22:16:22 http: TLS handshake error from 10.17.253.13:51368: EOF 2024/04/23 22:24:07 http: TLS handshake error from 10.17.253.13:52940: read tcp 10.16.219.245:8000->10.17.253.13:52940: read: connection reset by peer 2024/04/23 22:33:11 http: TLS handshake error from 10.17.253.13:45714: EOF 2024/04/23 22:33:45 http: TLS handshake error from 10.17.253.13:54346: EOF 2024/04/23 22:52:43 http: TLS handshake error from 10.17.253.13:48160: EOF 2024/04/23 23:29:27 http: TLS handshake error from 10.17.253.13:49562: EOF 2024/04/24 00:43:47 http: TLS handshake error from 10.17.253.13:43418: read tcp 10.16.219.245:8000->10.17.253.13:43418: read: connection reset by peer 2024/04/24 02:33:50 http: TLS handshake error from 10.17.253.13:38916: EOF 2024/04/24 02:52:41 http: TLS handshake error from 10.17.253.13:37176: EOF 2024/04/24 03:31:24 http: TLS handshake error from 10.17.253.13:54172: EOF 2024/04/24 11:01:53 http: TLS handshake error from 10.17.253.13:33186: read tcp 10.16.219.245:8000->10.17.253.13:33186: read: connection reset by peer 2024/04/24 12:38:40 http: TLS handshake error from 10.17.253.13:45658: EOF 2024/04/24 12:47:40 http: TLS handshake error from 10.17.253.13:37580: read tcp 10.16.219.245:8000->10.17.253.13:37580: read: connection reset by peer 2024/04/24 12:49:02 http: TLS handshake error from 10.17.253.13:38286: EOF 2024/04/24 12:49:14 http: TLS handshake error from 10.17.253.13:37542: EOF 2024/04/24 13:42:10 http: TLS handshake error from 10.17.253.13:55376: read tcp 10.16.219.245:8000->10.17.253.13:55376: read: connection reset by peer 2024/04/24 14:36:08 http: TLS handshake error from 10.17.253.13:40788: read tcp 10.16.219.245:8000->10.17.253.13:40788: read: connection reset by peer 2024/04/24 14:36:38 http: TLS handshake error from 10.17.253.13:45216: read tcp 10.16.219.245:8000->10.17.253.13:45216: read: connection reset by peer 2024/04/24 15:20:20 http: TLS handshake error from 10.17.253.13:53344: EOF 2024/04/24 15:47:42 http: TLS handshake error from 10.17.253.13:37378: EOF 2024/04/24 16:02:35 http: TLS handshake error from 10.17.253.13:39344: EOF 2024/04/24 16:07:05 http: TLS handshake error from 10.17.253.13:42104: EOF 2024/04/24 16:19:29 http: TLS handshake error from 10.17.253.13:53160: EOF 2024/04/24 17:10:00 http: TLS handshake error from 10.17.253.13:58466: read tcp 10.16.219.245:8000->10.17.253.13:58466: read: connection reset by peer 2024/04/24 17:35:51 http: TLS handshake error from 10.17.253.13:38166: EOF 2024/04/24 17:39:11 http: TLS handshake error from 10.17.253.13:35260: read tcp 10.16.219.245:8000->10.17.253.13:35260: read: connection reset by peer 2024/04/24 18:11:16 http: TLS handshake error from 10.17.253.13:47576: EOF 2024/04/24 18:11:16 http: TLS handshake error from 10.17.253.13:47578: EOF 2024/04/24 19:12:58 http: TLS handshake error from 10.17.253.13:51606: EOF 2024/04/24 19:21:31 http: TLS handshake error from 10.17.253.13:60950: read tcp 10.16.219.245:8000->10.17.253.13:60950: read: connection reset by peer 2024/04/24 19:25:15 http: TLS handshake error from 10.17.253.13:55624: read tcp 10.16.219.245:8000->10.17.253.13:55624: read: connection reset by peer 2024/04/24 19:45:36 http: TLS handshake error from 10.17.253.13:39076: EOF 2024/04/24 20:13:12 http: TLS handshake error from 10.17.253.13:39466: EOF 2024/04/24 20:22:06 http: TLS handshake error from 10.17.253.13:48240: EOF 2024/04/24 20:44:14 http: TLS handshake error from 10.17.253.13:52360: EOF 2024/04/24 20:47:44 http: TLS handshake error from 10.17.253.13:46514: read tcp 10.16.219.245:8000->10.17.253.13:46514: read: connection reset by peer 2024/04/24 21:06:27 http: TLS handshake error from 10.17.253.13:37374: EOF 2024/04/24 22:08:17 http: TLS handshake error from 10.17.253.13:47072: EOF 2024/04/24 22:32:11 http: TLS handshake error from 10.17.253.13:36608: EOF 2024/04/24 22:57:53 http: TLS handshake error from 10.17.253.13:35422: EOF 2024/04/24 23:12:31 http: TLS handshake error from 10.17.253.13:59566: read tcp 10.16.219.245:8000->10.17.253.13:59566: read: connection reset by peer 2024/04/24 23:25:26 http: TLS handshake error from 10.17.253.13:38534: read tcp 10.16.219.245:8000->10.17.253.13:38534: read: connection reset by peer 2024/04/24 23:25:27 http: TLS handshake error from 10.17.253.13:38550: read tcp 10.16.219.245:8000->10.17.253.13:38550: read: connection reset by peer 2024/04/25 00:53:19 http: TLS handshake error from 10.17.253.13:54564: read tcp 10.16.219.245:8000->10.17.253.13:54564: read: connection reset by peer 2024/04/25 00:53:19 http: TLS handshake error from 10.17.253.13:54574: read tcp 10.16.219.245:8000->10.17.253.13:54574: read: connection reset by peer ``

looks it's the issue related to golang and common in the admission webhook. https://github.com/kubernetes/kubernetes/issues/109022

I don't find the error affects any functionality, just annoying.

[itspngu]

Duplicate of #6272, would be fixed by #6665.

@Julian-Chu it does affect functionality, but might not on EKS specifically. On kubeadm-based Clusters we're observing that an expired certificate not being reloaded leads to the VPA Updater triggering a re-deploy of Pods with pending recommendations, but then the VPA Admission Webhook "fails open" (if configured as such, which is the default), not mutating resource requests/limits according to the Recommender's specs, resulting in a rollout of Pods with "stale" requests/limits, which the VPA Updater then triggers a rollout for 5 minutes later, continuing the cycle ad infinitum.

If you see behaviour like this (Pods being rolled out every 5 minutes), it's likely due to this issue, and the current band-aid fix is to restart the Pod(s) serving the Admission Webhook.

[adrianmoisey]

/area vertical-pod-autoscaler

[voelzmo]

/assign @voelzmo

[voelzmo]

@itspngu You're right, this also happens when your certificate is not correct and therefore the TLS handshake fails all the time. In this case, it is expected that you're observing Pods being restarted all the time, because vpa-updater continues to evict Pods.

This issue, however, is about a general issue that despite everything working fine you will still see those error logs – and it seems that people traced it down as being related to golang/go#50984

So apart from providing a way to have the admission-controller detect an updated cert with the next release, I don't think we can do something about the issue that the OP mentioned. For the first issue we already have #6272, and for the other issue, I'll just close it with "we can't do anything about it, sorry".

/close

[k8s-ci-robot]

@voelzmo: Closing this issue.

In response to this:

@itspngu You're right, this also happens when your certificate is not correct and therefore the TLS handshake fails all the time. In this case, it is expected that you're observing Pods being restarted all the time, because vpa-updater continues to evict Pods.

This issue, however, is about a general issue that despite everything working fine you will still see those error logs – and it seems that people traced it down as being related to golang/go#50984

So apart from providing a way to have the admission-controller detect an updated cert with the next release, I don't think we can do something about the issue that the OP mentioned. For the first issue we already have #6272, and for the other issue, I'll just close it with "we can't do anything about it, sorry".

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.