Skip to content

Support multiple types of datastore #57

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
dgarros opened this issue Jun 20, 2021 · 1 comment
Closed

Support multiple types of datastore #57

dgarros opened this issue Jun 20, 2021 · 1 comment
Labels
status: gathering feedback Further discussion is needed to determine this issue's scope and/or implementation type: enhancement

Comments

@dgarros
Copy link
Contributor

dgarros commented Jun 20, 2021

Environment

  • DiffSync version: 1.3.0

Proposed Functionality

Currently DiffSync Adapters are always leveraging an internal in-memory datastore that is storing the entire dataset.
It would be great to support different types of datastore, like Redis in addition to the in-memory datastore.
As an option it would be useful to deactivate the internal datastore as well or provide a solution to pull the data directly from the remote system.

Use Case

When we are dealing with a large dataset, the volume of data stored in-memory can become very large and can present some challenges. And external datastore like Redis would reduce the volume of data stored in memory.
In some cases, DiffSync is running very closely to an existing database and duplicating the data in memory is redundant and inefficient.

@dgarros dgarros added status: gathering feedback Further discussion is needed to determine this issue's scope and/or implementation type: enhancement labels Jun 20, 2021
@chadell
Copy link
Collaborator

chadell commented May 26, 2022

First abstraction and Redis support done in #106

@chadell chadell closed this as completed May 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status: gathering feedback Further discussion is needed to determine this issue's scope and/or implementation type: enhancement
Projects
None yet
Development

No branches or pull requests

2 participants