Skip to content

Redis Cache Module - 1 - Prepare Code #3073

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 85 commits into from
May 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
d848184
some thoughts before starting
antonpirker Apr 11, 2024
8926c93
some work
antonpirker Apr 16, 2024
618157b
Added address and port to cache span.
antonpirker Apr 24, 2024
5ab4ce0
Formatting
antonpirker Apr 24, 2024
ae7a995
Save correct itemsize
antonpirker Apr 24, 2024
11f7e15
Made cache item_size optional
antonpirker Apr 24, 2024
9ba4ccf
Added some documentation
antonpirker Apr 24, 2024
7a4e0cd
explaining docs
antonpirker Apr 24, 2024
8768373
Merge branch 'master' into antonpirker/django-caching-module
antonpirker Apr 29, 2024
8d51038
Testing cache module
antonpirker Apr 29, 2024
9791ec8
forking tests
antonpirker Apr 29, 2024
3d5d08a
changed op name
antonpirker May 3, 2024
fa47da2
Updated tests
antonpirker May 3, 2024
94421d2
Merge branch 'master' into antonpirker/django-caching-module
antonpirker May 3, 2024
be1d4fd
Test for host/port/cluster
antonpirker May 7, 2024
c8d86d4
More tests
antonpirker May 7, 2024
8ec780e
Merge branch 'master' into antonpirker/django-caching-module
antonpirker May 7, 2024
c9bff4b
Removed debug statement
antonpirker May 7, 2024
34743e0
Even more tests
antonpirker May 7, 2024
93b539c
fix
antonpirker May 7, 2024
3a50d66
Get cache settings also from old Django versions
antonpirker May 7, 2024
5a03f12
make mypy happy
antonpirker May 7, 2024
2e41ae9
Fixed accessor to settings
antonpirker May 7, 2024
11a662e
Cleanup
antonpirker May 7, 2024
0fad52b
Apply suggestions from code review
antonpirker May 8, 2024
2dcdeb9
Use tuple
antonpirker May 8, 2024
b398172
Updated comment
antonpirker May 8, 2024
bcf702c
Instrument get_many and set_many
antonpirker May 8, 2024
cefe47f
Moved parsing url outside to not call it that often
antonpirker May 8, 2024
c051197
Comment
antonpirker May 8, 2024
11be8fa
Make port an int early on
antonpirker May 8, 2024
3718f41
Linting
antonpirker May 8, 2024
66e1c6a
Merge branch 'master' into antonpirker/django-caching-module
antonpirker May 8, 2024
16a9edf
Added test
antonpirker May 8, 2024
1ad1cb8
Another test
antonpirker May 8, 2024
3b89394
linting
antonpirker May 8, 2024
0115843
oh, man...
antonpirker May 8, 2024
827858e
tests in CI actually do what is expected (compared to locally)
antonpirker May 8, 2024
cc0f1af
Removed cache.hit assert because it is unrealiable because of LocMemC…
antonpirker May 8, 2024
2de1d03
Always add item_size because load testing revealed no performance impact
antonpirker May 10, 2024
00e1aa5
Merge branch 'master' into antonpirker/django-caching-module
antonpirker May 10, 2024
f2d8c77
Updated tests
antonpirker May 10, 2024
ca65d17
Removed unused var
antonpirker May 10, 2024
7adb5a1
Make mypy happy
antonpirker May 10, 2024
7232c44
Linked GH issue in comment
antonpirker May 10, 2024
a319a51
Linked GH issue in comment
antonpirker May 10, 2024
a691861
Apply suggestions from code review
antonpirker May 10, 2024
5dbdf35
Merge branch 'antonpirker/django-caching-module' of github.com:getsen…
antonpirker May 10, 2024
8b710ab
Linked GH issue in comment
antonpirker May 10, 2024
fe52b14
Improved cache location parsing
antonpirker May 10, 2024
747e0c8
added test and fixed typo
antonpirker May 10, 2024
4fa92d0
Improved stripping sensitive data from keys
antonpirker May 10, 2024
e7986b8
whitespace
antonpirker May 10, 2024
66eff13
Improved url parsing
antonpirker May 10, 2024
2648759
return early
antonpirker May 10, 2024
f12ea4c
Organized code in modules
antonpirker May 15, 2024
3cca0d1
Added missing files
antonpirker May 15, 2024
051dd3e
Moved _set_db_data_on_span into queries module
antonpirker May 15, 2024
42f2db9
Does this work?
antonpirker May 15, 2024
1603e44
Fixed circular import
antonpirker May 15, 2024
9a21385
Updated test
antonpirker May 15, 2024
c777aa9
Cleanup
antonpirker May 15, 2024
c56e77d
Added missing file
antonpirker May 15, 2024
fe14fa5
Nicer code
antonpirker May 17, 2024
4004633
Cleanup
antonpirker May 17, 2024
fe5e4c0
fix
antonpirker May 17, 2024
dd8e508
Cleanup
antonpirker May 17, 2024
67408d7
Less confusing naming
antonpirker May 21, 2024
2d6e6cb
Same naming as in asgi/wsgi integrations
antonpirker May 21, 2024
051e4f7
Added missing files
antonpirker May 21, 2024
7d3a2a6
Merge branch 'master' into antonpirker/django-caching-module
antonpirker May 21, 2024
87e3b8b
Merge branch 'antonpirker/django-caching-module' into antonpirker/red…
antonpirker May 21, 2024
2b84023
use new names and only key in cache span description
antonpirker May 21, 2024
a001e15
cleanup
antonpirker May 21, 2024
7ac3c62
cleanup
antonpirker May 21, 2024
d6e5b18
Merge branch 'antonpirker/django-caching-module' into antonpirker/red…
antonpirker May 21, 2024
62336fc
Updated test matrix
antonpirker May 21, 2024
dbdf1e9
Fixed tests
antonpirker May 21, 2024
891b453
Fixed tests
antonpirker May 21, 2024
8f23a9a
Merge branch 'antonpirker/django-caching-module' into antonpirker/red…
antonpirker May 22, 2024
941ca77
Merge branch 'master' into antonpirker/django-caching-module
antonpirker May 23, 2024
80f50d5
Merge branch 'antonpirker/django-caching-module' into antonpirker/red…
antonpirker May 23, 2024
8c33a76
Redis Cache Module - 2 - Add Cache Spans (#3075)
antonpirker May 23, 2024
a3d6eb1
Merge branch 'antonpirker/redis-cache-module-1' of github.com:getsent…
antonpirker May 23, 2024
da57715
Merge branch 'master' into antonpirker/redis-cache-module-1
antonpirker May 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/workflows/test-integrations-databases.yml
Original file line number Diff line number Diff line change
Expand Up @@ -77,10 +77,10 @@ jobs:
run: |
set -x # print commands that are executed
./scripts/runtox.sh "py${{ matrix.python-version }}-redis-latest" --cov=tests --cov=sentry_sdk --cov-report= --cov-branch
- name: Test rediscluster latest
- name: Test redis_py_cluster_legacy latest
run: |
set -x # print commands that are executed
./scripts/runtox.sh "py${{ matrix.python-version }}-rediscluster-latest" --cov=tests --cov=sentry_sdk --cov-report= --cov-branch
./scripts/runtox.sh "py${{ matrix.python-version }}-redis_py_cluster_legacy-latest" --cov=tests --cov=sentry_sdk --cov-report= --cov-branch
- name: Test sqlalchemy latest
run: |
set -x # print commands that are executed
Expand Down Expand Up @@ -152,10 +152,10 @@ jobs:
run: |
set -x # print commands that are executed
./scripts/runtox.sh --exclude-latest "py${{ matrix.python-version }}-redis" --cov=tests --cov=sentry_sdk --cov-report= --cov-branch
- name: Test rediscluster pinned
- name: Test redis_py_cluster_legacy pinned
run: |
set -x # print commands that are executed
./scripts/runtox.sh --exclude-latest "py${{ matrix.python-version }}-rediscluster" --cov=tests --cov=sentry_sdk --cov-report= --cov-branch
./scripts/runtox.sh --exclude-latest "py${{ matrix.python-version }}-redis_py_cluster_legacy" --cov=tests --cov=sentry_sdk --cov-report= --cov-branch
- name: Test sqlalchemy pinned
run: |
set -x # print commands that are executed
Expand Down
2 changes: 1 addition & 1 deletion scripts/split-tox-gh-actions/split-tox-gh-actions.py
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@
"clickhouse_driver",
"pymongo",
"redis",
"rediscluster",
"redis_py_cluster_legacy",
"sqlalchemy",
],
"GraphQL": [
Expand Down
2 changes: 1 addition & 1 deletion sentry_sdk/consts.py
Original file line number Diff line number Diff line change
Expand Up @@ -368,7 +368,7 @@ class SPANDATA:
class OP:
ANTHROPIC_MESSAGES_CREATE = "ai.messages.create.anthropic"
CACHE_GET = "cache.get"
CACHE_SET = "cache.set"
CACHE_PUT = "cache.put"
COHERE_CHAT_COMPLETIONS_CREATE = "ai.chat_completions.create.cohere"
COHERE_EMBEDDINGS_CREATE = "ai.embeddings.create.cohere"
DB = "db"
Expand Down
30 changes: 6 additions & 24 deletions sentry_sdk/integrations/django/caching.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import functools
from typing import TYPE_CHECKING
from sentry_sdk.integrations.redis.utils import _get_safe_key
from urllib3.util import parse_url as urlparse

from django import VERSION as DJANGO_VERSION
Expand All @@ -8,7 +9,6 @@
import sentry_sdk
from sentry_sdk.consts import OP, SPANDATA
from sentry_sdk.utils import (
SENSITIVE_DATA_SUBSTITUTE,
capture_internal_exceptions,
ensure_integration_enabled,
)
Expand All @@ -28,27 +28,9 @@
]


def _get_key(args, kwargs):
# type: (list[Any], dict[str, Any]) -> str
key = ""

if args is not None and len(args) >= 1:
key = args[0]
elif kwargs is not None and "key" in kwargs:
key = kwargs["key"]

if isinstance(key, dict):
# Do not leak sensitive data
# `set_many()` has a dict {"key1": "value1", "key2": "value2"} as first argument.
# Those values could include sensitive data so we replace them with a placeholder
key = {x: SENSITIVE_DATA_SUBSTITUTE for x in key}

return str(key)


def _get_span_description(method_name, args, kwargs):
# type: (str, list[Any], dict[str, Any]) -> str
return _get_key(args, kwargs)
# type: (str, tuple[Any], dict[str, Any]) -> str
return _get_safe_key(method_name, args, kwargs)


def _patch_cache_method(cache, method_name, address, port):
Expand All @@ -61,11 +43,11 @@ def _patch_cache_method(cache, method_name, address, port):
def _instrument_call(
cache, method_name, original_method, args, kwargs, address, port
):
# type: (CacheHandler, str, Callable[..., Any], list[Any], dict[str, Any], Optional[str], Optional[int]) -> Any
# type: (CacheHandler, str, Callable[..., Any], tuple[Any, ...], dict[str, Any], Optional[str], Optional[int]) -> Any
is_set_operation = method_name.startswith("set")
is_get_operation = not is_set_operation

op = OP.CACHE_SET if is_set_operation else OP.CACHE_GET
op = OP.CACHE_PUT if is_set_operation else OP.CACHE_GET
description = _get_span_description(method_name, args, kwargs)

with sentry_sdk.start_span(op=op, description=description) as span:
Expand All @@ -78,7 +60,7 @@ def _instrument_call(
if port is not None:
span.set_data(SPANDATA.NETWORK_PEER_PORT, port)

key = _get_key(args, kwargs)
key = _get_safe_key(method_name, args, kwargs)
if key != "":
span.set_data(SPANDATA.CACHE_KEY, key)

Expand Down
Loading
Loading