Perf improvement for dimension not of factor 4 and 16

[2ooom]

Currently SIMD (SSE or AVX) is used for the cases when dimension is
multiple of 4 or 16, while when dimension size is not strictly equal
to multiple of 4 or 16 a slower non-vectorized method is used.

To improve performnance for these cases new methods are added:
L2SqrSIMD(4|16)ExtResidual - it relies on existing L2SqrSIMD(4|16)Ext
to compute up to *4 and *16 dimensions and finishes residual
computation by non-vectorized method L2Sqr.

[2ooom]

Hello @yurymalkov, thanks for the feedback and apologies for late response. I've closed previous PR and took into account your comment about strict equality.
Indeed you were right and residual method call adds some overhead. So I've added a wrapping method for this specific case and kept the existing ones, hope that addressed your concerns.

[yurymalkov]

@2ooom Thank you for the update! I'll check it within this week.

[yurymalkov]

I've looked though the PR. Thanks!
Some comments:

1. One file is missing, so it does not compile.
2. I think it makes sense to separate the benchmark into a folder, e.g. ./benchs to keep the library isolated.
3. Also please do PR against develop branch.

[2ooom]

Hi @yurymalkov. Apologies for messy PR, I didn't expect my 2 last commit (bench +.gitignore) would end up here. I've excluded both of them (it's WIP and I might push them in separate request to dev branch).
Could you please have another look and let me know if there any comments about space_l2.h changes?

[yurymalkov]

@2ooom Thank you! I'll take a look.

[2ooom]

Hello @yurymalkov I've updated InnerProduct computation with the same approach.
New methods added in space_ip: InnerProductSIMD(4|16)ExtResidual - relies on existing
InnerProductSIMD(4|16)Ext to compute up to *4 and *16 dimensions and
finishes residual computation by non-vectorized method InnerProduct.

Performance improvement compared to baseline is x3-4 times depending on
dimension. Benchmark results:

Run on (4 X 3300 MHz CPU s)
CPU Caches:
  L1 Data 32 KiB (x2)
  L1 Instruction 32 KiB (x2)
  L2 Unified 256 KiB (x2)
  L3 Unified 4096 KiB (x1)
Load Average: 2.10, 2.25, 2.46

----------------------------------------------------------
Benchmark          Time             CPU        Iterations
----------------------------------------------------------
TstDim65        14.0 ns         14.0 ns     20 * 48676012
RefDim65        50.3 ns         50.2 ns     20 * 12907985
TstDim101       23.8 ns         23.8 ns     20 * 27976276
RefDim101       91.4 ns         91.3 ns     20 *  7364003
TetDim129       30.0 ns         30.0 ns     20 * 23413955
RefDim129        123 ns          123 ns     20 *  5656383
TstDim257       57.8 ns         57.7 ns     20 * 11263073
RefDim257        268 ns          267 ns     20 *  2617478

[yurymalkov]

@2ooom Impressive results! Thank you!
Can you please redo the PR to the develop branch?

[2ooom]

Thanks @yurymalkov , I can change destination branch, but there is not dev or develop branch in nmslib/hnswlib and I'm not sure I can create one. Could you please help?

[yurymalkov]

@2ooom Sorry, I've missed you response :(
I've created the branch and look through the code and tested the performance.

I also turns out I can switch the base myself :)

Thank you so much for the PR!

[2ooom]

Thank you @yurymalkov, happy to contribute.
I'm planing to submit a couple more PRs related to distance computation (further perf improvements and float16 support) so I was thinking maybe you could share you testing and benchmarking scenarios?

[yurymalkov]

That would be nice! Looking forward for it.
Regarding the benchmarks, a simple benchmark is to run 1M SIFT.
For more reliable results, when I was developing the HNSW I was comparing two search algorithms by implementing them both and running on the same index (checking the result and time), usually for small dim data (d=4) and high dim data (d=1024), as reloading the index from the disk leads to performance deviations.
Maybe it is worth to abstract the search method in the library for easy experimentation. I can do that within a week or two.

[2ooom]

Thanks for sharing. I'll try SIFT. For now I've been doing micro-benchmarking on distance functions (l2, inner product), which are quite well isolated already, but it's got to have end-to-end comparison.

[yurymalkov]

@2ooom Got it. BTW, the sift test should be already in the repository.