feat(core): add cache manager #2449

jrschumacher · 2025-06-16T19:01:14Z

Proposed Changes

Add cache manager which allows services to implement cache while offering platform governance

Checklist

I have added or updated unit tests
I have added or updated integration tests (if appropriate)
I have added or updated documentation

Testing Instructions

gemini-code-assist

Summary of Changes

Hello @jrschumacher, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new, centralized cache management system to the platform. Its primary purpose is to enable individual services to implement caching for performance optimization while ensuring consistent configuration and governance across the entire platform. The changes involve adding core cache logic, integrating it into the server's startup and service registration processes, and providing documentation for configuration.

Highlights

Centralized Cache Management: A new cache.Manager has been introduced, providing a unified and configurable system for caching across the platform. This allows for centralized governance over cache resources.
Service-Scoped Caching: Individual services can now obtain and utilize their own cache.Cache instances from the central manager. This enables services to implement caching tailored to their specific needs while adhering to platform-wide policies.
Ristretto Backend Integration: The caching solution leverages the high-performance Ristretto cache. Configuration options, such as maxCost (maximum memory usage), can be specified in human-readable formats (e.g., '1GB') and are dynamically translated to appropriate Ristretto parameters.
Configuration and Documentation: New configuration keys for the cache have been added to the main server configuration, allowing administrators to easily set cache parameters. Comprehensive documentation is also provided to guide users on how to configure the new cache manager.
File Size Utility: A new utility function, RelativeFileSizeToBytes, has been added to convert human-readable file size strings (e.g., '10MB', '2GB') into their byte equivalents, simplifying cache size configuration.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Data flows so fast, Cache holds what is often sought, Speed, a new delight.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces a cache manager using Ristretto. Review focuses on test correctness, documentation accuracy, and robustness in test setups. Key areas: tests for RelativeFileSizeToBytes, initialization of CacheManager mocks, and clarification in CacheManager's configuration field type.

service/pkg/util/relativesizes_test.go

service/pkg/server/start_test.go

service/internal/server/server.go

service/pkg/cache/cache.go

service/pkg/util/relativesizes.go

github-actions · 2025-06-16T19:37:33Z

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	5000
Denied Decision Requests	0
Total Time	534.013105ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	5000
Denied Decision Requests	0
Total Time	323.199747ms

Standard Benchmark Metrics Skipped or Failed

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	344.535663ms
Throughput	290.25 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	35.953816929s
Average Latency	357.600177ms
Throughput	139.07 requests/second

NANOTDF Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	24.858066951s
Average Latency	247.651947ms
Throughput	201.14 requests/second

github-actions · 2025-06-16T22:43:23Z

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	5000
Denied Decision Requests	0
Total Time	525.074179ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	5000
Denied Decision Requests	0
Total Time	350.813372ms

Standard Benchmark Metrics Skipped or Failed

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	336.782978ms
Throughput	296.93 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	36.347093471s
Average Latency	360.46723ms
Throughput	137.56 requests/second

NANOTDF Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	25.28581525s
Average Latency	251.887783ms
Throughput	197.74 requests/second

github-actions · 2025-06-17T04:41:44Z

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	5000
Denied Decision Requests	0
Total Time	543.619765ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	5000
Denied Decision Requests	0
Total Time	351.067245ms

Standard Benchmark Metrics Skipped or Failed

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	349.744494ms
Throughput	285.92 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	37.494613969s
Average Latency	373.296931ms
Throughput	133.35 requests/second

NANOTDF Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	26.112577458s
Average Latency	259.691841ms
Throughput	191.48 requests/second

docs/Configuring.md

service/internal/server/server.go

service/pkg/cache/cache.go

service/pkg/cache/cache_test.go

service/pkg/server/start.go

github-actions · 2025-06-17T16:31:47Z

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	5000
Denied Decision Requests	0
Total Time	544.528006ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	5000
Denied Decision Requests	0
Total Time	353.779515ms

Standard Benchmark Metrics Skipped or Failed

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	345.533063ms
Throughput	289.41 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	36.693691707s
Average Latency	364.403938ms
Throughput	136.26 requests/second

NANOTDF Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	25.797496476s
Average Latency	256.622621ms
Throughput	193.82 requests/second

github-actions · 2025-06-17T16:46:21Z

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	5000
Denied Decision Requests	0
Total Time	500.403967ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	5000
Denied Decision Requests	0
Total Time	311.308637ms

Standard Benchmark Metrics Skipped or Failed

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	337.668964ms
Throughput	296.15 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	36.940123916s
Average Latency	367.753365ms
Throughput	135.35 requests/second

NANOTDF Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	25.714644257s
Average Latency	255.958443ms
Throughput	194.44 requests/second

github-actions · 2025-06-17T16:50:34Z

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	5000
Denied Decision Requests	0
Total Time	547.773514ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	5000
Denied Decision Requests	0
Total Time	341.124495ms

Standard Benchmark Metrics Skipped or Failed

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	343.512242ms
Throughput	291.11 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	35.522410287s
Average Latency	353.502898ms
Throughput	140.76 requests/second

NANOTDF Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	25.289241113s
Average Latency	251.864369ms
Throughput	197.71 requests/second

docs/Configuring.md

github-actions · 2025-06-17T16:58:27Z

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	5000
Denied Decision Requests	0
Total Time	515.002965ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	5000
Denied Decision Requests	0
Total Time	330.591426ms

Standard Benchmark Metrics Skipped or Failed

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	340.075613ms
Throughput	294.05 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	36.598383233s
Average Latency	364.418142ms
Throughput	136.62 requests/second

NANOTDF Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	25.661188392s
Average Latency	255.14238ms
Throughput	194.85 requests/second

github-actions · 2025-06-17T16:58:38Z

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	5000
Denied Decision Requests	0
Total Time	535.745746ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	5000
Denied Decision Requests	0
Total Time	318.969876ms

Standard Benchmark Metrics Skipped or Failed

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	337.243908ms
Throughput	296.52 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	36.348671553s
Average Latency	360.87898ms
Throughput	137.56 requests/second

NANOTDF Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	25.246675939s
Average Latency	250.973327ms
Throughput	198.05 requests/second

github-actions · 2025-06-17T16:58:54Z

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	5000
Denied Decision Requests	0
Total Time	519.560618ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	5000
Denied Decision Requests	0
Total Time	338.804261ms

Standard Benchmark Metrics Skipped or Failed

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	352.827057ms
Throughput	283.42 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	37.979893829s
Average Latency	378.069692ms
Throughput	131.65 requests/second

NANOTDF Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	26.068640147s
Average Latency	258.237095ms
Throughput	191.80 requests/second

🤖 I have created a release *beep* *boop* --- ## [0.7.0](service/v0.6.0...service/v0.7.0) (2025-06-24) ### ⚠ BREAKING CHANGES * **policy:** disable kas grants in favor of key mappings ([#2220](#2220)) ### Features * **authz:** Add caching to keycloak ERS ([#2466](#2466)) ([f5b0a06](f5b0a06)) * **authz:** auth svc registered resource GetDecision support ([#2392](#2392)) ([5405674](5405674)) * **authz:** authz v2 GetBulkDecision ([#2448](#2448)) ([0da3363](0da3363)) * **authz:** cache entitlement policy within authorization service ([#2457](#2457)) ([c16361c](c16361c)) * **authz:** ensure logging parity between authz v2 and v1 ([#2443](#2443)) ([ef68586](ef68586)) * **core:** add cache manager ([#2449](#2449)) ([2b062c5](2b062c5)) * **core:** consume RPC interceptor request context metadata in logging ([#2442](#2442)) ([2769c48](2769c48)) * **core:** DSPX-609 - add cli-client to keycloak provisioning ([#2396](#2396)) ([48e7489](48e7489)) * **core:** ERS cache setup, fix cache initialization ([#2458](#2458)) ([d0c6938](d0c6938)) * inject logger and cache manager to key managers ([#2461](#2461)) ([9292162](9292162)) * **kas:** expose provider config from key details. ([#2459](#2459)) ([0e7d39a](0e7d39a)) * **main:** Add Close() method to cache manager ([#2465](#2465)) ([32630d6](32630d6)) * **policy:** disable kas grants in favor of key mappings ([#2220](#2220)) ([30f8cf5](30f8cf5)) * **policy:** Restrict deletion of pc with used key. ([#2414](#2414)) ([3b40a46](3b40a46)) * **sdk:** allow Connect-Protocol-Version RPC header for cors ([#2437](#2437)) ([4bf241e](4bf241e)) ### Bug Fixes * **core:** remove generics on new platform cache manager and client ([#2456](#2456)) ([98c3c16](98c3c16)) * **core:** replace opentdf-public client with cli-client ([#2422](#2422)) ([fb18525](fb18525)) * **deps:** bump github.com/casbin/casbin/v2 from 2.106.0 to 2.107.0 in /service in the external group ([#2416](#2416)) ([43afd48](43afd48)) * **deps:** bump github.com/opentdf/platform/protocol/go from 0.4.0 to 0.5.0 in /service ([#2470](#2470)) ([3a73fc9](3a73fc9)) * **deps:** bump github.com/opentdf/platform/sdk from 0.4.7 to 0.5.0 in /service ([#2473](#2473)) ([ad37476](ad37476)) * **deps:** bump the external group across 1 directory with 2 updates ([#2450](#2450)) ([9d8d1f1](9d8d1f1)) * **deps:** bump the external group across 1 directory with 2 updates ([#2472](#2472)) ([d45b3c8](d45b3c8)) * only request a token when near expiration ([#2370](#2370)) ([556d95e](556d95e)) * **policy:** fix casing bug and get provider config on update. ([#2403](#2403)) ([a52b8f9](a52b8f9)) * **policy:** properly formatted pem in test fixtures ([#2409](#2409)) ([54ffd23](54ffd23)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: opentdf-automation[bot] <149537512+opentdf-automation[bot]@users.noreply.github.com>

feat(core): add cache manager

e5cf9b4

jrschumacher requested a review from a team as a code owner June 16, 2025 19:01

github-actions bot added size/m docs Documentation and removed size/m labels Jun 16, 2025

gemini-code-assist bot reviewed Jun 16, 2025

View reviewed changes

jrschumacher added 2 commits June 16, 2025 14:10

fix lint issues

39afdb7

Resolve some defects

33feae1

Refactor ristretto config

ea045fd

Update service/pkg/util/relativesizes.go

58e2d22

elizabethhealy previously approved these changes Jun 17, 2025

View reviewed changes

docs/Configuring.md Show resolved Hide resolved

jakedoublev reviewed Jun 17, 2025

View reviewed changes

jrschumacher mentioned this pull request Jun 17, 2025

Enhance logger to support a group value to enable subservice logging #2452

Open

Add generics

f7b97ba

jrschumacher dismissed elizabethhealy’s stale review via f7b97ba June 17, 2025 16:27

jrschumacher added 2 commits June 17, 2025 11:41

Move config

1abcbae

Update config

ab1fec5

jakedoublev reviewed Jun 17, 2025

View reviewed changes

docs/Configuring.md Outdated Show resolved Hide resolved

jakedoublev reviewed Jun 17, 2025

View reviewed changes

docs/Configuring.md Outdated Show resolved Hide resolved

jakedoublev added 2 commits June 17, 2025 09:53

Update docs/Configuring.md

96987c5

Update docs/Configuring.md

aaa456e

jakedoublev reviewed Jun 17, 2025

View reviewed changes

docs/Configuring.md Outdated Show resolved Hide resolved

Update docs/Configuring.md

a4f3435

jakedoublev approved these changes Jun 17, 2025

View reviewed changes

github-actions bot reviewed Jun 17, 2025

View reviewed changes

docs/Configuring.md Show resolved Hide resolved

jrschumacher mentioned this pull request Jun 17, 2025

Add support for disabling caching globally #2453

Open

jrschumacher enabled auto-merge June 17, 2025 17:03

strantalis approved these changes Jun 17, 2025

View reviewed changes

jrschumacher added this pull request to the merge queue Jun 17, 2025

Merged via the queue into main with commit 2b062c5 Jun 17, 2025
30 checks passed

jrschumacher deleted the cache-manager-extract branch June 17, 2025 17:20

opentdf-automation bot mentioned this pull request Jun 17, 2025

chore(main): release service 0.7.0 #2406

Merged

feat(core): add cache manager #2449

feat(core): add cache manager #2449

Uh oh!

Conversation

jrschumacher commented Jun 16, 2025

Proposed Changes

Checklist

Testing Instructions

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Jun 16, 2025

Benchmark authorization.GetDecisions Results:

Benchmark authorization.v2.GetMultiResourceDecision Results:

Standard Benchmark Metrics Skipped or Failed

Bulk Benchmark Results

TDF3 Benchmark Results:

NANOTDF Benchmark Results:

Uh oh!

github-actions bot commented Jun 16, 2025

Benchmark authorization.GetDecisions Results:

Benchmark authorization.v2.GetMultiResourceDecision Results:

Standard Benchmark Metrics Skipped or Failed

Bulk Benchmark Results

TDF3 Benchmark Results:

NANOTDF Benchmark Results:

Uh oh!

github-actions bot commented Jun 17, 2025

Benchmark authorization.GetDecisions Results:

Benchmark authorization.v2.GetMultiResourceDecision Results:

Standard Benchmark Metrics Skipped or Failed

Bulk Benchmark Results

TDF3 Benchmark Results:

NANOTDF Benchmark Results:

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Jun 17, 2025

Benchmark authorization.GetDecisions Results:

Benchmark authorization.v2.GetMultiResourceDecision Results:

Standard Benchmark Metrics Skipped or Failed

Bulk Benchmark Results

TDF3 Benchmark Results:

NANOTDF Benchmark Results:

Uh oh!

github-actions bot commented Jun 17, 2025

Benchmark authorization.GetDecisions Results:

Benchmark authorization.v2.GetMultiResourceDecision Results:

Standard Benchmark Metrics Skipped or Failed

Bulk Benchmark Results

TDF3 Benchmark Results:

NANOTDF Benchmark Results:

Uh oh!

github-actions bot commented Jun 17, 2025

Benchmark authorization.GetDecisions Results:

Benchmark authorization.v2.GetMultiResourceDecision Results:

Standard Benchmark Metrics Skipped or Failed

Bulk Benchmark Results

TDF3 Benchmark Results: