feat(k8s): improve artifact selections for specific namespaces #8248

afdesk · 2025-01-15T12:05:36Z

Description

This PR improves selections of Kubernetes artifacts for users with limited credentials.
Now Trivy can receive resources only for included namespaces.

$ trivy k8s --report summary --kubeconfig myconfig mycontext --include-namespaces rbac-test

Before:

2025-01-27T12:06:58+06:00       ERROR   Unable to list resources        error="failed listing resources for gvr: apps/v1, Resource=deployments - deployments.apps is forbidden: User \"system:serviceaccount:default:limiteduser\" cannot list resource \"deployments\" in API group \"apps\" at the cluster scope"
2025-01-27T12:06:58+06:00       ERROR   Unable to list resources        error="failed listing resources for gvr: /v1, Resource=pods - pods is forbidden: User \"system:serviceaccount:default:limiteduser\" cannot list resource \"pods\" in API group \"\" at the cluster scope"
2025-01-27T12:06:58+06:00       ERROR   Unable to list resources        error="failed listing resources for gvr: apps/v1, Resource=replicasets - replicasets.apps is forbidden: User \"system:serviceaccount:default:limiteduser\" cannot list resource \"replicasets\" in API group \"apps\" at the cluster scope"
2025-01-27T12:06:58+06:00       ERROR   Unable to list resources        error="failed listing resources for gvr: /v1, Resource=replicationcontrollers - replicationcontrollers is forbidden: User \"system:serviceaccount:default:limiteduser\" cannot list resource \"replicationcontrollers\" in API group \"\" at the cluster scope"
2025-01-27T12:06:58+06:00       ERROR   Unable to list resources        error="failed listing resources for gvr: apps/v1, Resource=statefulsets - statefulsets.apps is forbidden: User \"system:serviceaccount:default:limiteduser\" cannot list resource \"statefulsets\" in API group \"apps\" at the cluster scope"
2025-01-27T12:06:58+06:00       ERROR   Unable to list resources        error="failed listing resources for gvr: apps/v1, Resource=daemonsets - daemonsets.apps is forbidden: User \"system:serviceaccount:default:limiteduser\" cannot list resource \"daemonsets\" in API group \"apps\" at the cluster scope"
2025-01-27T12:06:58+06:00       ERROR   Unable to list resources        error="failed listing resources for gvr: batch/v1, Resource=cronjobs - cronjobs.batch is forbidden: User \"system:serviceaccount:default:limiteduser\" cannot list resource \"cronjobs\" in API group \"batch\" at the cluster scope"
2025-01-27T12:06:58+06:00       ERROR   Unable to list resources        error="failed listing resources for gvr: batch/v1, Resource=jobs - jobs.batch is forbidden: User \"system:serviceaccount:default:limiteduser\" cannot list resource \"jobs\" in API group \"batch\" at the cluster scope"
2025-01-27T12:06:58+06:00       ERROR   Unable to list resources        error="failed listing resources for gvr: /v1, Resource=services - services is forbidden: User \"system:serviceaccount:default:limiteduser\" cannot list resource \"services\" in API group \"\" at the cluster scope"
2025-01-27T12:06:58+06:00       ERROR   Unable to list resources        error="failed listing resources for gvr: /v1, Resource=serviceaccounts - serviceaccounts is forbidden: User \"system:serviceaccount:default:limiteduser\" cannot list resource \"serviceaccounts\" in API group \"\" at the cluster scope"
2025-01-27T12:06:58+06:00       ERROR   Unable to list resources        error="failed listing resources for gvr: /v1, Resource=configmaps - configmaps is forbidden: User \"system:serviceaccount:default:limiteduser\" cannot list resource \"configmaps\" in API group \"\" at the cluster scope"
2025-01-27T12:06:58+06:00       ERROR   Unable to list resources        error="failed listing resources for gvr: rbac.authorization.k8s.io/v1, Resource=roles - roles.rbac.authorization.k8s.io is forbidden: User \"system:serviceaccount:default:limiteduser\" cannot list resource \"roles\" in API group \"rbac.authorization.k8s.io\" at the cluster scope"
2025-01-27T12:06:59+06:00       ERROR   Unable to list resources        error="failed listing resources for gvr: rbac.authorization.k8s.io/v1, Resource=rolebindings - rolebindings.rbac.authorization.k8s.io is forbidden: User \"system:serviceaccount:default:limiteduser\" cannot list resource \"rolebindings\" in API group \"rbac.authorization.k8s.io\" at the cluster scope"
2025-01-27T12:06:59+06:00       ERROR   Unable to list resources        error="failed listing resources for gvr: networking.k8s.io/v1, Resource=networkpolicies - networkpolicies.networking.k8s.io is forbidden: User \"system:serviceaccount:default:limiteduser\" cannot list resource \"networkpolicies\" in API group \"networking.k8s.io\" at the cluster scope"
2025-01-27T12:06:59+06:00       ERROR   Unable to list resources        error="failed listing resources for gvr: networking.k8s.io/v1, Resource=ingresses - ingresses.networking.k8s.io is forbidden: User \"system:serviceaccount:default:limiteduser\" cannot list resource \"ingresses\" in API group \"networking.k8s.io\" at the cluster scope"
2025-01-27T12:06:59+06:00       ERROR   Unable to list resources        error="failed listing resources for gvr: /v1, Resource=resourcequotas - resourcequotas is forbidden: User \"system:serviceaccount:default:limiteduser\" cannot list resource \"resourcequotas\" in API group \"\" at the cluster scope"
2025-01-27T12:06:59+06:00       ERROR   Unable to list resources        error="failed listing resources for gvr: /v1, Resource=limitranges - limitranges is forbidden: User \"system:serviceaccount:default:limiteduser\" cannot list resource \"limitranges\" in API group \"\" at the cluster scope"
2025-01-27T12:07:00+06:00       ERROR   Unable to list resources        error="failed listing resources for gvr: rbac.authorization.k8s.io/v1, Resource=clusterroles - clusterroles.rbac.authorization.k8s.io is forbidden: User \"system:serviceaccount:default:limiteduser\" cannot list resource \"clusterroles\" in API group \"rbac.authorization.k8s.io\" at the cluster scope"
2025-01-27T12:07:00+06:00       ERROR   Unable to list resources        error="failed listing resources for gvr: rbac.authorization.k8s.io/v1, Resource=clusterrolebindings - clusterrolebindings.rbac.authorization.k8s.io is forbidden: User \"system:serviceaccount:default:limiteduser\" cannot list resource \"clusterrolebindings\" in API group \"rbac.authorization.k8s.io\" at the cluster scope"
2025-01-27T12:07:00+06:00       ERROR   Unable to list resources        error="failed listing resources for gvr: /v1, Resource=nodes - nodes is forbidden: User \"system:serviceaccount:default:limiteduser\" cannot list resource \"nodes\" in API group \"\" at the cluster scope"                                                                                                                      2025-01-27T12:07:00+06:00       ERROR   Unable to list node resources   error="nodes is forbidden: User \"system:serviceaccount:default:limiteduser\" cannot list resource \"nodes\" in API group \"\" at the cluster scope"
2025-01-27T12:07:00+06:00       INFO    Node scanning is enabled                                                                                                                                     2025-01-27T12:07:00+06:00       INFO    If you want to disable Node scanning via an in-cluster Job, please try '--disable-node-collector' to disable the Node-Collector job.                         2025-01-27T12:07:00+06:00       INFO    Scanning K8s... K8s="mycontext"                                                                                                                              0 [________________________________________________________________________________________________________________________________________________________________________________________] ?% ? p/s
Summary Report for mycontext


Workload Assessment
┌───────────┬──────────┬───────────────────┬───────────────────┬───────────────────┐
│ Namespace │ Resource │  Vulnerabilities  │ Misconfigurations │      Secrets      │
│           │          ├───┬───┬───┬───┬───┼───┬───┬───┬───┬───┼───┬───┬───┬───┬───┤
│           │          │ C │ H │ M │ L │ U │ C │ H │ M │ L │ U │ C │ H │ M │ L │ U │
└───────────┴──────────┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┘
Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN

Infra Assessment                                                                                                                                                                                     ┌───────────┬──────────┬───────────────────┬───────────────────┬───────────────────┐
│ Namespace │ Resource │  Vulnerabilities  │ Misconfigurations │      Secrets      │                                                                                                                 │           │          ├───┬───┬───┬───┬───┼───┬───┬───┬───┬───┼───┬───┬───┬───┬───┤
│           │          │ C │ H │ M │ L │ U │ C │ H │ M │ L │ U │ C │ H │ M │ L │ U │                                                                                                                 └───────────┴──────────┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┘
Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN
                                                                                                                                                                                                     RBAC Assessment
┌───────────┬──────────┬───────────────────┐                                                                                                                                                         │ Namespace │ Resource │  RBAC Assessment  │
│           │          ├───┬───┬───┬───┬───┤                                                                                                                                                         │           │          │ C │ H │ M │ L │ U │
└───────────┴──────────┴───┴───┴───┴───┴───┘                                                                                                                                                         Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN

Afrer:

2025-01-27T12:17:11+06:00       INFO    Node scanning is enabled
2025-01-27T12:17:11+06:00       INFO    If you want to disable Node scanning via an in-cluster Job, please try '--disable-node-collector' to disable the Node-Collector job.
2025-01-27T12:17:11+06:00       INFO    Scanning K8s... K8s="mycontext"
5 / 5 [-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 1 p/s

Summary Report for mycontext


Workload Assessment
┌───────────┬───────────────────────┬──────────────────────┬────────────────────┬───────────────────┐
│ Namespace │       Resource        │   Vulnerabilities    │ Misconfigurations  │      Secrets      │
│           │                       ├───┬────┬────┬────┬───┼───┬───┬───┬────┬───┼───┬───┬───┬───┬───┤
│           │                       │ C │ H  │ M  │ L  │ U │ C │ H │ M │ L  │ U │ C │ H │ M │ L │ U │
├───────────┼───────────────────────┼───┼────┼────┼────┼───┼───┼───┼───┼────┼───┼───┼───┼───┼───┼───┤
│ rbac-test │ Pod/my-multiimage-pod │ 2 │ 26 │ 77 │ 99 │ 1 │   │ 3 │ 8 │ 18 │   │   │   │   │   │   │
└───────────┴───────────────────────┴───┴────┴────┴────┴───┴───┴───┴───┴────┴───┴───┴───┴───┴───┴───┘
Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN


Infra Assessment
┌───────────┬──────────┬───────────────────┬───────────────────┬───────────────────┐
│ Namespace │ Resource │  Vulnerabilities  │ Misconfigurations │      Secrets      │
│           │          ├───┬───┬───┬───┬───┼───┬───┬───┬───┬───┼───┬───┬───┬───┬───┤
│           │          │ C │ H │ M │ L │ U │ C │ H │ M │ L │ U │ C │ H │ M │ L │ U │
└───────────┴──────────┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┘
Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN


RBAC Assessment
┌───────────┬───────────────────┬───────────────────┐
│ Namespace │     Resource      │  RBAC Assessment  │
│           │                   ├───┬───┬───┬───┬───┤
│           │                   │ C │ H │ M │ L │ U │
├───────────┼───────────────────┼───┼───┼───┼───┼───┤
│ rbac-test │ Role/limited-role │ 2 │   │   │   │   │
└───────────┴───────────────────┴───┴───┴───┴───┴───┘
Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN

Testing

There was added use case here 6e690b9 and tests were broken as expected.
After trivy-kubernetes was update to the latest version 2859ad3, this test case is passed now.

Documentation

there were added parts:

Related issues

"--include-kinds pod --disable-node-collector" fails with the "nodes is forbidden" error. #7094

Related PRs

fix: add using include/exclude kinds and namespaces trivy-kubernetes#395

Checklist

I've read the guidelines for contributing to this repository.
I've followed the conventions in the PR title.
I've added tests that prove my fix is effective or that my feature works.
I've updated the documentation with the relevant information (if needed).
I've added usage information (if the PR introduces new options)
I've included a "before" and "after" example to the description (if the PR is a user interface change).

simar7 · 2025-01-20T18:57:05Z

Really strange this PR has no changes? 🫨 I wonder what happened.

afdesk · 2025-01-21T15:43:53Z

Really strange this PR has no changes? 🫨 I wonder what happened.

it's a draft. I prepare test cases at first, and then will update the docs and trivy-kubernetes

# Conflicts: # go.mod # go.sum

afdesk · 2025-01-27T06:21:47Z

@simar7 @knqyf263
this PR is ready for your review
could you please take a look when you have time? thanks!

knqyf263 · 2025-01-27T06:37:07Z

@simar7 You are now also maintaining the K8s area, right? Should we update the code owners?

trivy/.github/CODEOWNERS

Lines 17 to 22 in cc66d6d

    
           # Helm chart 
        
           helm/trivy/ @afdesk 
        
           # Kubernetes scanning 
        
           pkg/k8s/                         @afdesk 
        
           docs/docs/target/kubernetes.md   @afdesk

simar7 · 2025-01-28T06:06:09Z

@simar7 You are now also maintaining the K8s area, right? Should we update the code owners?

trivy/.github/CODEOWNERS

Lines 17 to 22 in cc66d6d

# Helm chart

helm/trivy/ @afdesk

# Kubernetes scanning

pkg/k8s/ @afdesk

docs/docs/target/kubernetes.md @afdesk

Yes, we should add myself to it.

knqyf263 · 2025-01-28T06:25:07Z

OK, done #8303

# Conflicts: # go.mod # go.sum

docs/docs/target/kubernetes.md

Co-authored-by: simar7 <[email protected]>

afdesk · 2025-01-29T07:51:53Z

@knqyf263 do you mind if we merge this PR? thanks

knqyf263 · 2025-01-29T07:58:24Z

@afdesk One comment: isn't it a feature enhancement or bug fix? We might want to update the PR title. It looks like an enhancement to me, but if you don't think so, please feel free to merge this.

afdesk · 2025-01-29T08:11:10Z

@knqyf263 thanks a lot! you're right!
Now we can use Role instead of ClusterRole for scanning, so it looks like a feature.
I'll add it to the release notes

…ecurity#8248) Co-authored-by: simar7 <[email protected]>

chore(k8s): improve artifact selections

Loading
Loading status checks…

4eb82d4

afdesk added 6 commits January 24, 2025 17:02

chore(test): add resources for limited user

42a1bfa

test(k8s): add test case for limited user

Loading
Loading status checks…

a10c4ef

fix: use common syntax for sed

Loading
Loading status checks…

95fd607

chore: use a default context to correct test case

Loading
Loading status checks…

5dd9625

fix(test): use correct image for testing

Loading
Loading status checks…

2368ba0

chore: fix linter errors

Loading
Loading status checks…

6eb1e9c

afdesk requested a review from simar7 January 24, 2025 12:41

afdesk added 4 commits January 24, 2025 18:44

test: add a broken use case for failure

Loading
Loading status checks…

6e690b9

chore: bump up Trivy-kubernetes to v0.7.0

Loading
Loading status checks…

2859ad3

Merge branch 'main' of github.com:afdesk/trivy into chore/update-k8s

Loading
Loading status checks…

70f3b98

# Conflicts: # go.mod # go.sum

docs(k8s): update info about permissions

Loading
Loading status checks…

6a6e269

afdesk changed the title ~~chore(k8s): improve artifact selections~~ chore(k8s): improve artifact selections for specific namespaces Jan 27, 2025

afdesk marked this pull request as ready for review January 27, 2025 06:20

afdesk requested a review from knqyf263 as a code owner January 27, 2025 06:20

Merge branch 'main' of github.com:afdesk/trivy into chore/update-k8s

Loading
Loading status checks…

12cd544

# Conflicts: # go.mod # go.sum

simar7 approved these changes Jan 29, 2025

View reviewed changes

simar7 reviewed Jan 29, 2025

View reviewed changes

docs/docs/target/kubernetes.md Outdated Show resolved Hide resolved

simar7 reviewed Jan 29, 2025

View reviewed changes

docs/docs/target/kubernetes.md Outdated Show resolved Hide resolved

afdesk and others added 2 commits January 29, 2025 13:21

Update docs/docs/target/kubernetes.md

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

GPG key ID: B5690EEEBB952194

Verified
Learn about vigilant mode

Loading
Loading status checks…

362471d

Co-authored-by: simar7 <[email protected]>

Update docs/docs/target/kubernetes.md

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

GPG key ID: B5690EEEBB952194

Verified
Learn about vigilant mode

Loading
Loading status checks…

4ad32d6

Co-authored-by: simar7 <[email protected]>

knqyf263 added this pull request to the merge queue Jan 29, 2025

knqyf263 removed this pull request from the merge queue due to a manual request Jan 29, 2025

afdesk changed the title ~~chore(k8s): improve artifact selections for specific namespaces~~ feat(k8s): improve artifact selections for specific namespaces Jan 29, 2025

afdesk added this pull request to the merge queue Jan 29, 2025

afdesk mentioned this pull request Jan 29, 2025

docs(k8s): a note about k8s credentials #7394

Closed

6 tasks

Merged via the queue into aquasecurity:main with commit db9e57a Jan 29, 2025
18 checks passed

afdesk deleted the chore/update-k8s branch January 29, 2025 08:34

aqua-bot mentioned this pull request Jan 29, 2025

release: v0.59.0 [main] #8041

Merged

afdesk added this to the v0.59.0 milestone Jan 30, 2025

sfowl mentioned this pull request Jan 30, 2025

Update trivy tool in container RedHatProductSecurity/rapidast#290

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(k8s): improve artifact selections for specific namespaces #8248

feat(k8s): improve artifact selections for specific namespaces #8248

afdesk commented Jan 15, 2025 •

edited

Loading

simar7 commented Jan 20, 2025

afdesk commented Jan 21, 2025

afdesk commented Jan 27, 2025

knqyf263 commented Jan 27, 2025

simar7 commented Jan 28, 2025

knqyf263 commented Jan 28, 2025

afdesk commented Jan 29, 2025

knqyf263 commented Jan 29, 2025

afdesk commented Jan 29, 2025

feat(k8s): improve artifact selections for specific namespaces #8248

feat(k8s): improve artifact selections for specific namespaces #8248

Conversation

afdesk commented Jan 15, 2025 • edited Loading

Description

Testing

Documentation

Related issues

Related PRs

Checklist

simar7 commented Jan 20, 2025

afdesk commented Jan 21, 2025

afdesk commented Jan 27, 2025

knqyf263 commented Jan 27, 2025

simar7 commented Jan 28, 2025

knqyf263 commented Jan 28, 2025

afdesk commented Jan 29, 2025

knqyf263 commented Jan 29, 2025

afdesk commented Jan 29, 2025

afdesk commented Jan 15, 2025 •

edited

Loading