Append-Only Support for RedisAI

[habichta]

Is there any progress concerning Append-Only Support for RedisAI?
Right now we just use (excessive) RDB (is there a better Strategy?)

[lantiga]

Hi @habichta, we will start supporting AOF after the next Redis release (see #157).

[lantiga]

BTW, what is your usage pattern? Like: how frequently you need to save, do you need to save all tensors or would you rather avoid that, etc. It is interesting to collect ideas on possible future improvements of these aspects. Thanks in advance

[habichta]

Sorry for the late reply. Important to us: Mainly the models that were set. For now we also use tensors to save certain parameters that we derived during training (for scaling data before inference, or thresholds/parameters for anomaly detection). The tensors we set as inputs to the models (for inference) are not that important. With the RDB Approach, we have a lot of dumps because inference causes many updates of tensors. Basically we would like to persist deployed models and parameters but not the input/output tensors of the models.

[lantiga]

Hi @habichta, thanks for your feedback. It's really important.
We have a DAG design (#88) that would solve this independently from RDB/AOF. It's in the pipe (it has been since the early days) but it won't be implemented before a few weeks time.

In the meantime we'll enable AOF asap (see my comment above).

[lantiga]

One more comment @habichta. It is very likely something you thought about, but I'm going to mention it anyway: for the time being you could consider disabling save in redis.conf, and instead make explicit calls to BGSAVE whenever you update a model or make a change in the parameter tensors.

[habichta]

Thanks for the tip. I'm new to RedisAI and Redis for that matter.

[lantiga]

Following up on this: #157 has been merged. We'll activate AOF tests shortly.

[lantiga]

AOF support has been merged, closing this one.