Skip to content

Conversation

aaronburtle
Copy link
Contributor

Why make this change?

Closes #2604

What is this change?

Inside of the SqlQueryStructure, there is a private constructor, that all of the public constructors will call. This private constructor offers a single point where all of the queries will bottleneck, and therefore we add the cache control information to the query structure at this point. Then when we are in the SqlQueryEngine, we can check for this cache control information and handle the query appropriately.

The cache control options can be found

here: #2253

and here: https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Cache-Control#request_directives

But for reference, they are no-cache, no-store, only-if-cached, which will mean we do not get from the cache, we do not store in the cache, and if there is a cache miss we return gateway timeout, respectively.

How was this tested?

Run against test suite.

Sample Request(s)

To test you need to include the relevant cache headers in the request, "no-cache", "no-store", or "only-if-cached"

@aaronburtle aaronburtle self-assigned this Apr 15, 2025
@aaronburtle aaronburtle added the enhancement New feature or request label Apr 15, 2025
@aaronburtle aaronburtle moved this from Todo to Review In Progress in Data API builder Apr 15, 2025
@aaronburtle aaronburtle added this to the 1.5 milestone Apr 15, 2025
@aaronburtle
Copy link
Contributor Author

/azp run

Copy link

Azure Pipelines successfully started running 6 pipeline(s).

@aaronburtle
Copy link
Contributor Author

/azp run

Copy link

Azure Pipelines successfully started running 6 pipeline(s).

@aaronburtle
Copy link
Contributor Author

/azp run

Copy link

Azure Pipelines successfully started running 6 pipeline(s).

@aaronburtle
Copy link
Contributor Author

/azp run

Copy link

Azure Pipelines successfully started running 6 pipeline(s).

@Aniruddh25 Aniruddh25 requested a review from Copilot April 16, 2025 18:11
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds support for handling caching directives via the HTTP Cache-Control header by incorporating new properties and logic into the SQL query processing and cache service layers. It introduces new cache getter and setter methods in the cache service, adds cache control handling in the SQL query engine, and updates the SQL query structure and REST request context to capture and validate the Cache-Control header.

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
src/Core/Services/Cache/DabCacheService.cs Added TryGet and Set methods to retrieve/store JSON responses in the cache based on a derived cache key.
src/Core/Resolvers/SqlQueryEngine.cs Updated the query execution flow to handle cache control options and to throw exceptions when only-if-cached is used with a cache miss.
src/Core/Resolvers/Sql Query Structures/SqlQueryStructure.cs Introduced CacheControlOption property and constants, with logic to extract and validate Cache-Control header values.
src/Core/Models/RestRequestContexts/RestRequestContext.cs Added a new property to capture the Cache-Control header from REST requests.

@aaronburtle
Copy link
Contributor Author

/azp run

Copy link

Azure Pipelines successfully started running 6 pipeline(s).

@aaronburtle
Copy link
Contributor Author

/azp run

Copy link

Azure Pipelines successfully started running 6 pipeline(s).

Copy link
Contributor

@RubenCerna2079 RubenCerna2079 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@aaronburtle
Copy link
Contributor Author

/azp run

Copy link

Azure Pipelines successfully started running 6 pipeline(s).

Copy link
Contributor

@souvikghosh04 souvikghosh04 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@aaronburtle
Copy link
Contributor Author

/azp run

Copy link

Azure Pipelines successfully started running 6 pipeline(s).

@aaronburtle
Copy link
Contributor Author

/azp run

Copy link

Azure Pipelines successfully started running 6 pipeline(s).

@aaronburtle
Copy link
Contributor Author

/azp run

Copy link

Azure Pipelines successfully started running 6 pipeline(s).

@RubenCerna2079 RubenCerna2079 dismissed Aniruddh25’s stale review May 23, 2025 19:30

The changes that were asked for are completed.

@RubenCerna2079 RubenCerna2079 merged commit de1477c into main May 23, 2025
11 checks passed
@RubenCerna2079 RubenCerna2079 deleted the dev/aaronburtle/HeaderCacheSupport branch May 23, 2025 19:31
@github-project-automation github-project-automation bot moved this from Review In Progress to Done in Data API builder May 23, 2025
RubenCerna2079 added a commit that referenced this pull request May 29, 2025
## Why make this change?

Closes #2604

## What is this change?

Inside of the `SqlQueryStructure`, there is a private constructor, that
all of the public constructors will call. This private constructor
offers a single point where all of the queries will bottleneck, and
therefore we add the cache control information to the query structure at
this point. Then when we are in the `SqlQueryEngine`, we can check for
this cache control information and handle the query appropriately.

The cache control options can be found 

here: #2253 

and here:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Cache-Control#request_directives

But for reference, they are `no-cache`, `no-store`, `only-if-cached`,
which will mean we do not get from the cache, we do not store in the
cache, and if there is a cache miss we return gateway timeout,
respectively.

## How was this tested?

Run against test suite.

## Sample Request(s)

To test you need to include the relevant cache headers in the request,
"no-cache", "no-store", or "only-if-cached"

---------

Co-authored-by: Copilot <[email protected]>
Co-authored-by: Aniruddh Munde <[email protected]>
Co-authored-by: RubenCerna2079 <[email protected]>
RubenCerna2079 added a commit that referenced this pull request May 29, 2025
## Why make this change?

Closes #2604

## What is this change?

Inside of the `SqlQueryStructure`, there is a private constructor, that
all of the public constructors will call. This private constructor
offers a single point where all of the queries will bottleneck, and
therefore we add the cache control information to the query structure at
this point. Then when we are in the `SqlQueryEngine`, we can check for
this cache control information and handle the query appropriately.

The cache control options can be found

here: #2253

and here:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Cache-Control#request_directives

But for reference, they are `no-cache`, `no-store`, `only-if-cached`,
which will mean we do not get from the cache, we do not store in the
cache, and if there is a cache miss we return gateway timeout,
respectively.

## How was this tested?

Run against test suite.

## Sample Request(s)

To test you need to include the relevant cache headers in the request,
"no-cache", "no-store", or "only-if-cached"

---------

Co-authored-by: Copilot <[email protected]>
Co-authored-by: Aniruddh Munde <[email protected]>
Co-authored-by: RubenCerna2079 <[email protected]>
aaronburtle added a commit that referenced this pull request May 29, 2025
## Why make this change?

This change is made in order to add all of the commits for milestone 1.5
into its respective branch.

## What is this change?

This change cherry-picks all of the commits that were added after the
first release candidate.
Cherry-picked commits:
 - #2648 
#2657
#2617 
#2659 
#2655 
#2633 
#2667 
#2673 
#2650 
#2695 
#2702 
#2688

## How was this tested?

- [ ] Integration Tests
- [ ] Unit Tests

## Sample Request(s)

---------

Co-authored-by: Sezal Chug <[email protected]>
Co-authored-by: sezalchug <[email protected]>
Co-authored-by: Tommaso Stocchi <[email protected]>
Co-authored-by: Aaron Powell <[email protected]>
Co-authored-by: aaronburtle <[email protected]>
Co-authored-by: Aniruddh Munde <[email protected]>
Co-authored-by: Jerry Nixon <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Michael Staib <[email protected]>
Co-authored-by: souvikghosh04 <[email protected]>
Co-authored-by: Souvik Ghosh <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Add support for request and response HTTP headers for caching.

4 participants