Bug: Slow index.add

[vprelovac]

Describe the bug

Index creation, vector addition and search loop is 90x slower than identical loop for faiss.

Reproduced in Google colab.

Steps to reproduce

import faiss
from usearch.index import Index

import numpy as np

# Set the seed for reproducibility
np.random.seed(42)

# Generate 500 random vectors of length 1024 with values between 0 and 1
vectors = np.random.rand(500, 1024)
vector=np.random.rand(1024)

%%timeit
# FAISS

indexf = faiss.IndexFlatL2(vectors.shape[-1])
indexf.add(np.array(vectors).astype(np.float32))
D, I = indexf.search(np.array([vector]), 50)
# 1.14 ms ± 4.34 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)


%%timeit

index = Index(
    ndim=vectors.shape[-1], # Define the number of dimensions in input vectors
    metric='l2_sq', # Choose 'cos', 'l2sq', 'haversine' or other metric, default = 'ip'
  
   )
index.add(labels= np.arange(len(vectors)), vectors=vectors)
matches, distances, count = index.search(vector, 50, exact=True)
  
# 94.7 ms ± 20.9 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

Expected behavior

n/a

USearch version

latest pip

Operating System

ubuntu

Hardware architecture

x86

Which interface are you using?

Python bindings

Contact Details

No response

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

[ashvardanian]

@vprelovac, oh, I see! So IndexFlatL2 is not actually an index. It just appends vectors and brute-forces on search. On tiny collections with 1000 entries, it makes sense. The kNN benchmarks are generally meaningful from 1M entries onwards. The correct comparison there would be against IndexHNSW from FAISS.

That being said, we can probably automatically switch to an exact search on tiny collections :) I will consider it in the next release.

[vprelovac]

the kNN benchmarks are generally meaningful from 1M entries onwards.

Except when they are not :) Our production use case is 500-50000 items, and the index has to be recreated every time. Every ms matters.

Is there a configuration where usearch can be faster than faiss on this use case?

[ashvardanian]

@vprelovac, probably. You can pass the additional exact=True for the' search' queries, which will switch to brute force.
As for construction, I've just added a task to the 1.0 TODO list.
Can you please provide more details about your setup?

1. What's the dimensionality of the vectors?
2. Which metrics do you use?
3. Which interface/platform do you prefer? Python, C++, WASM, ...

We may be able to adjust the implementation accordingly 🤗

[vprelovac]

Thanks for looking into it.

1. 512-1024
2. l2sq
3. Python

This is the typical retrieval scenario in web search when LLMs are used.

I am passing exact=True for search as you see in my test code.

[ashvardanian]

Hey, @vprelovac! I have implemented an exact search shortcut, which avoids index construction altogether. The interface is as simple as it gets:

from usearch.index import search, MetricKind

import numpy as np

# Set the seed for reproducibility
np.random.seed(42)

# Generate 500 random vectors of length 1024 with values between 0 and 1
vectors = np.random.rand(500, 1024).astype(np.float32)
vector = np.random.rand(1024).astype(np.float32)

search(vectors, vector, 50, MetricKind.L2sq, exact=True)

I have repeated your benchmark on my MacBook Pro with the M2 CPU, and the speed is higher than for FAISS, even without having a BLAS dependency and no SimSIMD optimizations for the L2 metric. This is going to be merged into 1.0. More optimizations coming soon.

[ashvardanian]

🎉 This issue has been resolved in version 0.23.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

[vprelovac]

I can indeed confirm vastly better performance on my setup (by a factor 20). I can only say fantastic work! 🚀

Question: How does one get the distances?