Skip to content

Missing call to addPoint for python bindings? #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
theolivenbaum opened this issue Apr 23, 2018 · 3 comments
Closed

Missing call to addPoint for python bindings? #8

theolivenbaum opened this issue Apr 23, 2018 · 3 comments

Comments

@theolivenbaum
Copy link

@yurymalkov Awesome code, really enjoying it compared to the other similar libraries.

Just one remark I noticed here, it seems that the python bindings provided are not actually adding any vector to the index - there is no call to addPoint on the code:

if(normalize==false) {
    ParallelFor(start, rows, num_threads, [&](size_t row, size_t threadId) {
        size_t id = ids.size() ? ids.at(row) : (cur_l+row);
    });
} else{
    std::vector<float> norm_array(num_threads * dim);
    ParallelFor(start, rows, num_threads, [&](size_t row, size_t threadId) {
        // normalize vector:
        size_t start_idx = threadId * dim;
        normalize_vector((float *) items.data(row), (norm_array.data()+start_idx));

        size_t id = ids.size() ? ids.at(row) : (cur_l+row);
    });
};


Also, one question. Is the library supposed to be thread safe for adding new items? Or just for performing search?

@yurymalkov
Copy link
Member

Thanks! I definitely need to set up Travis 😃

Adding new items should be thread safe, i.e. can be done with python threads in parallel. I have not tested it yet, but if there is a problem - it is in python bindings part (C++ code is well tested).

However, adding and searching simultaneously should not be thread safe now. This was done intentionally to reduce the locks (which might slow down the search), assuming that user does not run insertions during the search.
I am considering adding a separate fully thread safe search method. Please let me know if it is interesting.

@tlyalex
Copy link

tlyalex commented May 28, 2020

"I am considering adding a separate fully thread safe search method. Please let me know if it is interesting." Is there any update on this one? It would be a very useful feature if add and search could be fully threadsafe with each other.

@yurymalkov
Copy link
Member

@tlyalex Not much has changed, as there is not much ask. After merging #216 I can look into that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants