6 ms·
HNSW became the defacto default specifically because you don’t need to precalculate the index and it updates as writes come in. This is a great article, and al
by binarymax 2y ago
HNSW became the defacto default specifically because you don’t need to precalculate the index and it updates as writes come in.
This is a great article, and all the points are there, but the truth is that most teams running million scale vectors don’t want the operational complexity of quantizing offline in some frequency. They’ll gladly outsource the costs to paying for RAM instead of some IVFPQ calculation.
However if there were a solution that “just worked” to handle PQ for shards in near real time for updates that also had sane filtering, that would be really nice.
- nostrebored 2y agoYes, real time updates, filtering, and multi vector support make most of these on device, in memory approaches untenable. If you really are just doing a similarity search against a fixed set of things, often you know the queries ahead of time and can just make a lookup table.
- bddicken 2y agoAt PlanetScale, we have a really nice solution for this in MySQL using SPANN + SPFresh. The way we've implemented it allows for pre and post filtering, full compatibility with where clause filtering, and has the same kind of acid compliance you'd expect from a relational database. You can read about it here: https://planetscale.com/blog/announcing-planetscale-vectors-public-beta https://planetscale.com/blog/announcing-planetscale-vectors-...
- VoVAllen 2y agoHi, I'm the author of the article. In our product, VectorChord, we use a quantization algorithm called RaBitQ, which doesn’t require a separate codebook. Unlike IVFPQ, it avoids the need to maintain and update the corresponding codebook, so the update issue you mentioned is not a problem. Regarding filtering, I’m not sure which specific scenario you’re referring to, but we currently support iterative post-filtering and are technically capable of perfectly supporting pre-filtering as well.
- binarymax 2y agoPre and post filtering are both not great. Some HNSW implementations in products like Vespa and Qdrant have filter-during-search. This remains an unsolved problem in cluster-based indices.