Skip to content

Binary safe strings #538

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Dec 21, 2020
Merged

Binary safe strings #538

merged 11 commits into from
Dec 21, 2020

Conversation

lantiga
Copy link
Contributor

@lantiga lantiga commented Dec 15, 2020

Address #508 by converting all use of user-supplied strings (except device names, TF node names and script source) to binary-safe strings (RedisModuleString).

@lantiga lantiga changed the title Mark locations that need change for binary strings Binary safe strings Dec 15, 2020
@lantiga lantiga marked this pull request as draft December 15, 2020 00:06
@lantiga lantiga marked this pull request as ready for review December 15, 2020 01:17
alonre24
alonre24 previously approved these changes Dec 15, 2020
Copy link
Collaborator

@alonre24 alonre24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, I don't have any comments.
One thing - Meir told me that it is better to use "RedisModule_HoldString" instead of "RedisModule_RetainString" if it matters...

@lantiga
Copy link
Contributor Author

lantiga commented Dec 15, 2020

Looks good, I don't have any comments.
One thing - Meir told me that it is better to use "RedisModule_HoldString" instead of "RedisModule_RetainString" if it matters...

Thank you Alon. That's interesting, I just read the docstring for HoldString. I'll make the change, thanks.

@lantiga
Copy link
Contributor Author

lantiga commented Dec 16, 2020

@alonre24 done with the HoldString change

src/util/dict.c Outdated
}

static void *rstringsKeyDup(void *privdata, const void *key) {
return RedisModule_CreateStringFromString(NULL, (RedisModuleString *)key);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe RAI_HoldString to avoid strdup

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd avoid this as it could lead to inter-thread issues (in fact I'm getting a segfault). Need to track if further, let's keep it as is for this PR.

src/run_info.c Outdated

static void *RAI_TensorDictKeyDup(void *privdata, const void *key) {
return RedisModule_Strdup((char *)key);
return RedisModule_CreateStringFromString(NULL, (RedisModuleString *)key);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RAI_HoldString ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's now the same code as above, so same consideration.

src/run_info.c Outdated
Comment on lines 21 to 37
size_t len;
const char *buffer = RedisModule_StringPtrLen((RedisModuleString *)key, &len);
return AI_dictGenHashFunction(buffer, len);
}

static int RAI_TensorDictKeyStrcmp(void *privdata, const void *key1, const void *key2) {
const char *strKey1 = key1;
const char *strKey2 = key2;
return strcmp(strKey1, strKey2) == 0;
RedisModuleString *strKey1 = (RedisModuleString *)key1;
RedisModuleString *strKey2 = (RedisModuleString *)key2;
return RedisModule_StringCompare(strKey1, strKey2) == 0;
}

static void RAI_TensorDictKeyFree(void *privdata, void *key) { RedisModule_Free(key); }
static void RAI_TensorDictKeyFree(void *privdata, void *key) {
RedisModule_FreeString(NULL, (RedisModuleString *)key);
}

static void *RAI_TensorDictKeyDup(void *privdata, const void *key) {
return RedisModule_Strdup((char *)key);
return RedisModule_CreateStringFromString(NULL, (RedisModuleString *)key);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is duplicated logic of the RSring dictionaries. Any chance to consolidate?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

#include <string.h>
#include "../redisai_memory.h"

RedisModuleString *RAI_HoldString(RedisModuleCtx *ctx, RedisModuleString *str) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This util is a good idea! But the first argument is redundant (we always send NULL as ctx).

When I come to think of it, if we know that we won't use the auto_memory mechanism, we may consider adding a util function such as "RAI_FreeString(RedisModuleString *str)", and replace every call for RedisModule_FreeString with that - so we avoid the annoying bugs. This can probably be done in another PR...

@codecov
Copy link

codecov bot commented Dec 21, 2020

Codecov Report

Merging #538 (fb5a017) into master (f707a56) will increase coverage by 0.01%.
The diff coverage is 89.58%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #538      +/-   ##
==========================================
+ Coverage   75.31%   75.33%   +0.01%     
==========================================
  Files          22       22              
  Lines        5226     5221       -5     
==========================================
- Hits         3936     3933       -3     
+ Misses       1290     1288       -2     
Impacted Files Coverage Δ
src/tensor.c 82.73% <ø> (ø)
src/script.c 65.94% <50.00%> (+0.37%) ⬆️
src/model.c 71.62% <91.66%> (+0.15%) ⬆️
src/redisai.c 79.84% <92.85%> (+0.09%) ⬆️
src/dag.c 87.36% <94.11%> (+0.24%) ⬆️
src/run_info.c 69.18% <100.00%> (-1.43%) ⬇️
src/stats.c 84.93% <100.00%> (-0.41%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f707a56...fb5a017. Read the comment docs.

@lantiga
Copy link
Contributor Author

lantiga commented Dec 21, 2020

Yes! 🎉 Thank you @alonre24 for spotting the bug that was making coverage fail. And thank you coverage for failing BTW :-)

@lantiga lantiga merged commit 740bbba into master Dec 21, 2020
@lantiga lantiga deleted the binary-safe-strings branch December 21, 2020 13:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants