-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Sentinel Support #2664
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Sentinel Support #2664
Changes from 49 commits
Commits
Show all changes
80 commits
Select commit
Hold shift + click to select a range
59cf984
redis client socket changes needed for sentinel
sjpotter cd254b1
Sentinel Implementation [EXPERIMENTAL]
sjpotter 18fc971
add pooling
sjpotter 29d4da0
Merge remote-tracking branch 'leibale/v5' into sentinel
sjpotter 27d7f4e
improve typing with SENTINEL_ client members
sjpotter 1b7cfc6
cleanup - remove unused comments / commented code
sjpotter 13a18bd
small sendCommand change + revert change to tsconfig
sjpotter 8f59b35
add more sentinel commands needed for testing.
sjpotter b7e59d5
lots of fixups and a reasonable first pass test suite
sjpotter 8ae7d1f
add a timer option to update topology in background
sjpotter d3505ce
format all the things
sjpotter 300889f
more progress
sjpotter 6f14410
Merge remote-tracking branch 'upstream/v5' into sentinel
sjpotter 9e3cda6
small cleanup
sjpotter 18bb2d0
try to group promises together to minimize the internal await points
sjpotter 6bfe7b7
redo events, to keep a single topology event to listen on
sjpotter 5e722ef
nits + readme
sjpotter f0be3a8
add RedisSentinelFactory to provide lower level access to sentinel
sjpotter 45e64f1
Merge remote-tracking branch 'upstream/v5' into sentinel
sjpotter 1e9e65b
nit
sjpotter 1731059
update
sjpotter c74c4ba
add RedisSentinelClient/Type for leased clients
sjpotter c8f74f3
add self for private access + improve emitting
sjpotter 882c1c7
nit
sjpotter 9a59887
nits
sjpotter 84fa90b
improve testing
sjpotter 93cd57a
ismall nit for typing
sjpotter 43a40a1
bunch of changes
sjpotter 57e9889
improve pub sub proxy.
sjpotter 2e6091d
wrap the passed through RedisClient error to make clear where its com…
sjpotter 02bbb13
refactor sentinel object / factory tests apart
sjpotter 53bd858
harden tests a little bit more
sjpotter a661d89
add pipeline test
sjpotter 6bd1c67
add scripts/function tests + fixups / cleanups to get them to work
sjpotter 173fcc4
change to use redis-stack-server for redis nodes to enable module tes…
sjpotter c150bfc
fix test, forgot to return in use function with module
sjpotter 65f543e
rename test
sjpotter 5f83c09
improve tests to test with redis/sentinel nodes with and withput pass…
sjpotter ce35310
cleanup for RedisSentinel type generic typing in tests
sjpotter 0f331dc
remove debugLog, just rely on traace mechanism
sjpotter 9a77d5e
added multi tests for script/function/modules
sjpotter 3da082c
don't emit errors on lease object, only on main object
sjpotter 73e0536
improve testing
sjpotter 89f33f9
extract out common code to reduce duplication
sjpotter cea648e
nit
sjpotter 8dcd811
nits
sjpotter 60974ca
nit
sjpotter 1f3e19d
remove SENTINEL_... commands from main client, load them via module i…
sjpotter 3dea400
missed adding RedisSentinelModule to correct places in RedisSentinelF…
sjpotter 9ec1838
nits
sjpotter 873014e
fix test logging on error
sjpotter c567741
invalidate watches when client reconnects
sjpotter 5259124
remove WATCH and UNWATCH command files, fix WATCH and UNWATCH return …
leibale 4f0b040
missing file in last commit :P
leibale 19288ef
support for custom message in `WatchError`
leibale 91b27cd
setDirtyWatch
leibale 7bf97ec
update watch docs
leibale 4567564
Merge branch 'watch-invalidate' into sentinel
sjpotter 8945cf5
fixes needed
sjpotter c289366
Merge branch 'watch-invalidate' into sentinel
sjpotter 26f72ae
wip
sjpotter 4968e4c
get functions/modules to work again
sjpotter 2554c0c
reuse leased client on pipelined commands.
sjpotter cc0768b
test tweaks
sjpotter 2b865f1
nit
sjpotter 6959b17
change how "sentinel" object client works, allow it to be reserved
sjpotter 54e8e5e
Merge branch 'v5' of github.com:redis/node-redis into sentinel
leibale 28d6587
Merge branch 'v5' of github.com:redis/node-redis into sentinel
leibale bb9b466
review
leibale 1920858
Merge branch 'v5' of github.com:redis/node-redis into sentinel
leibale a3d198e
Merge branch 'v5' of github.com:redis/node-redis into sentinel
leibale 5df6069
fixes to get more tests to pass
sjpotter 838a6f1
Merge branch 'v5' of github.com:redis/node-redis into sentinel
leibale 27cb970
Merge branch 'v5' of github.com:redis/node-redis into sentinel
leibale bd5a820
handle dirtyWatch and watchEpoch in reset and resetIfDirty
leibale 07e62f3
"fix", but not correct, needs more work
sjpotter d917967
fix pubsub proxy
leibale 4cb9564
remove timeout from steadyState function in test, caused problems
sjpotter 8da0970
improve restarting nodes
sjpotter 0aefb77
fix pubsub proxy and test
sjpotter File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,87 @@ | ||
# Redis Sentinel | ||
|
||
The Redis Sentinel object of node-redis provides a high level object that provides access to a high availability redis installation managed by Redis Sentinel to provide enumeration of master and replica nodes belonging to an installation as well as reconfigure itself on demand for failover and topology changes. | ||
|
||
## Usage | ||
|
||
```typescript | ||
//FIXME: are these imports, correct? | ||
import { createSentinel } from '@redis/client'; | ||
import { RedisSentinelOptions } from '@redis/client/sentinel/types'; | ||
|
||
const options: RedisSentinelOptions = {name: "sentinel-db", sentinelRootNodes: [{host: "example", port: 1234}]}; | ||
const sentinel = createSentinel(options); | ||
await sentinel.connect(); | ||
``` | ||
|
||
In the above example, we configure the sentinel object to fetch the configuration for the database Redis Sentinel is monitoring as "sentinel-db" with one of the sentinels being located at `exampe:1234`. | ||
|
||
Once one has this object, one can use it like a normal Redis client | ||
|
||
```typescript | ||
assert(await sentinel.set('x', 1) == 'OK); | ||
const value = await sentinel.get('x'); | ||
``` | ||
|
||
It supports pubsub via the normal mechanisms, including migrating the listeners if the node they are connected to goes down. | ||
|
||
```typescript | ||
await sentinel.subscribe('test', (msg) => {}); | ||
await sentinel.unsubscribe('test'); | ||
``` | ||
|
||
The sentinel object provides the ability to direct read only commands against replica nodes if configured to do so | ||
|
||
```typescript | ||
const options: RedisSentinelOptions = { | ||
name: "sentinel-db", | ||
sentinelRootNodes: [{host: "example", port: 1234}], | ||
useReplicas: true | ||
}; | ||
const sentinel = createSentinel(options); | ||
await sentinel.connect(); | ||
``` | ||
|
||
the sentinel object provides the ability to manage a pool of clients for both the master and replicas (if using replica reads) | ||
|
||
```typescript | ||
const options: RedisSentinelOptions = { | ||
name: "sentinel-db", | ||
sentinelRootNodes: [{host: "example", port: 1234}], | ||
useReplicas: true, | ||
masterPoolSize: 16, | ||
replicaPoolSize: 4 | ||
}; | ||
const sentinel = createSentinel(options); | ||
await sentinel.connect(); | ||
``` | ||
|
||
the sentinel object enables the user to get a persistent client lease against the master replicas, if desired. For instance, if one wants to perform redis transactions with `WATCH/MULTI/EXEC` | ||
|
||
```typescript | ||
const clientLease = await sentinel.getMasterClientLease(); | ||
await clientLease.watch("x"); | ||
const resp = await clientLease.multi().get("x").exec(); | ||
clientLease.release(); | ||
``` | ||
|
||
If no clients are available to get a lease, the aquistion will block until a client lease is available. | ||
|
||
In addition, even without explicit leases, whenever the sentinel object does actions against the master node (ex: | ||
|
||
```typescript | ||
await sentinel.set('x', 1)`) | ||
``` | ||
it will take a temporary lease on a client, so that it will not step on top of any other clients | ||
|
||
In addition, the sentinel object provides the `use()` member to provide the ability to pass in a function that takes a `RedisClientType` and this will aquire a lease for the function's execution and release it upon compeletion | ||
|
||
```typescript | ||
let promise = sentinel.use(async (client) => { | ||
await client.set("x", 1); | ||
await client.watch("x"); | ||
return client.multi().get("x").exec(); | ||
}); | ||
``` | ||
|
||
the usages of `use()` can be non reslient (default) where the sentinel client will not attempt to perform them again if the connections breaks in the middle / a topology reconfiguration is required or in a reslient mode where it will retry it when we reconnect to the master. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
import { BlobStringReply, Command, MapReply, RedisArgument } from "../../RESP/types"; | ||
import { transformTuplesReply } from "../../commands/generic-transformers"; | ||
|
||
export default { | ||
transformArguments(dbname: RedisArgument) { | ||
return ['SENTINEL', 'MASTER', dbname]; | ||
}, | ||
transformReply: { | ||
2: transformTuplesReply, | ||
3: undefined as unknown as () => MapReply<BlobStringReply, BlobStringReply> | ||
} | ||
} as const satisfies Command; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
import { SimpleStringReply, Command, RedisArgument } from "../../RESP/types"; | ||
|
||
export default { | ||
transformArguments(dbname: RedisArgument, host: RedisArgument, port: RedisArgument, quorum: RedisArgument) { | ||
return ['SENTINEL', 'MONITOR', dbname, host, port, quorum]; | ||
}, | ||
transformReply: undefined as unknown as () => SimpleStringReply<'OK'> | ||
} as const satisfies Command; |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm pretty sure this solves #2679, need to check (and if it does, cherry-pick it)