VoVAllen

11 ms·

1.
Researchers accuse ICLR 2026 paper TurboQuant of misrepresenting RaBitQ(twitter.com)
2 points by VoVAllen 6mo ago|0 comments
2.
by VoVAllen 11mo ago
It's recall. Thanks for pointing out this, we'll update the diagram. The core part is a quantization technique called RaBitQ. We can scan over the bit vector to have an estimation about the real distance between query and data. I&
3.
by VoVAllen 11mo ago
We do have some benchmark number at https://blog.vectorchord.ai/vector-search-over-postgresql-a-... . It varies on different dataset, but most cases it's 2x or more QPS comparing to pgvector's hnsw at same recall.
4.
by VoVAllen 11mo ago
And we do have user hosting 3 Billion vectors with Postgres + VectorChord with sharding. And they're using vectors to save the earth! Check https://blog.vectorchord.ai/3-billion-vectors-in-postgresql-...
5.
by VoVAllen 11mo ago
We at https://github.com/tensorchord/VectorChord solved most of the pgvector issues mentioned in this blog: - We're IVF + quantization, can support 15x more updates per second comparing to pgvector's HNSW. In
6.
VectorChord 0.3: Multi-Vector Contextual Late Interaction in PostgreSQL(blog.vectorchord.ai)
3 points by VoVAllen 1y ago|0 comments
7.
Protect the Earth with 3B Vector Search(blog.vectorchord.ai)
2 points by VoVAllen 1y ago|0 comments
8.
by VoVAllen 1y ago
It's coming in the Postgres 18. https://www.depesz.com/2025/02/28/waiting-for-postgresql-18-...
9.
PostgreSQL Full-Text Search: Fast When Done Right (Debunking the Slow Myth)(blog.vectorchord.ai)
427 points by VoVAllen 1y ago|85 comments
10.
by VoVAllen 2y ago
Hi, I'm the tech lead of VectorChord-bm25. It's not based on pg_search (pg_bm25). We just chose the same name during our internal development, and changed it to the formal name VectorChord-bm25 when we released it.
11.
by VoVAllen 2y ago
I'm not sure I fully understand your point. This can be implemented quite easily by organizing each posting list with specific attributes to locate the values efficiently. For example, you could build each posting list into a B-tree, o
12.
by VoVAllen 2y ago
Hi, I'm the author of the article. Please check out our vector search extension in postgres, VectorChord [1]. It's based on RabitQ (a new quantization method) + IVF. It achieves 10ms-level latency for top-10 searches on a 100M dat
13.
by VoVAllen 2y ago
Hi, I'm the author of the article. Please check out our vector search extension in postgres, VectorChord [1]. It's based on RabitQ (a new quantization method) + IVF. It achieves 10ms-level latency for top-10 searches on a 100M dat
14.
by VoVAllen 2y ago
Hi, I'm the author of the article. Please check out our vector search extension in postgres, VectorChord [1]. It achieves 10ms-level latency for top-10 searches on a 100M dataset and 60ms-level latency when using SSD with limited memor
15.
by VoVAllen 2y ago
Hi, I'm the author of the article. In our actual product, VectorChord, we adopted a new quantization algorithm called RaBitQ. The accuracy has not been compromised by the quantization process. We’ve provided recall-QPS comparison curve
16.
by VoVAllen 2y ago
Hi, I'm the author of the article. The sequential access pattern of IVF makes prefetching and large block sequential reads much easier, whereas it's almost impossible for HNSW to achieve efficient prefetching.
17.
by VoVAllen 2y ago
Hi, I'm the author of the article. The sequential access pattern of IVF makes prefetching and large block sequential reads much easier, whereas it's almost impossible for HNSW to achieve efficient prefetching.
18.
by VoVAllen 2y ago
Hi, I'm the author of the article. I agree with your point. The model from https://www.mixedbread.ai/blog/mxbai-embed-xsmall-v1 also looks great, though I haven’t had the chance to try it yet.
19.
by VoVAllen 2y ago
Hi, I'm the author of the article. In our product, VectorChord, we use a quantization algorithm called RaBitQ, which doesn’t require a separate codebook. Unlike IVFPQ, it avoids the need to maintain and update the corresponding codeboo
20.
by VoVAllen 2y ago
Hi, I'm the author of the article. Meta have conducted some experiments on dynamic IVF with datasets of several hundred million records. The conclusion was that recall can be maintained through simple repartitioning and rebalancing str
21.
by VoVAllen 2y ago
Hi, I'm the author of the article. We actually rely on a new quantization method called RaBitQ instead of ScaNN. You can read more about it at https://dev.to/gaoj0017/quantization-in-the-counterintuitive... .
22.
by VoVAllen 2y ago
Hi, I'm the author of the article. I think you're right, and we’ll update the description in the blog. Since binary operations are simpler than floating-point operations, the speedup could indeed be greater than 32x.
23.
by VoVAllen 2y ago
Hi, I'm the author of the article. I actually think the opposite of what you mentioned. IVF is more suitable for prefiltering compared to HNSW. For prefiltering in HNSW, there is a certain requirement for the filtering rate—it can&#x27
24.
by VoVAllen 2y ago
It's hard to compare the cost with the Serverless pricing model, as write and read have extra costs. On the pricing page, datastax costs $4000 to write 100M 768-dim vectors. And 10M query will cost $300, which is only 4 QPS. As compari
25.
by VoVAllen 2y ago
We don’t perform any reclustering. As you said, users would need to rebuild the index if they want to recluster. However, based on our observations, the speed remains acceptable even with significant data growth. We did a simple experiment
26.
by VoVAllen 2y ago
I recently came across a project that looks promising: [WordLlama]( https://github.com/dleemiller/WordLlama?tab=readme-ov-file#s... ). It appears to be well-suited for semantic chunking, though I haven’t had a chance to
27.
by VoVAllen 2y ago
In the LAION-5M benchmark, we’ve compared our performance against ElasticSearch and OpenSearch. However, comparing ingestion performance is more challenging due to differences in architecture. Both ElasticSearch and OpenSearch, like most ve
28.
by VoVAllen 2y ago
There's no easy way to index ColBert multi-vectors in a scalable way that I know of. Vespa seems to rely heavily on binary quantization, which can cost a lot in recall loss. And for most cases, using ColBert as a reranker is good enoug
29.
by VoVAllen 2y ago
I'm the project lead for VectorChord. I have tested ScaNN on AlloyDB Omni but have struggled to achieve reasonable recall on the GIST 1M dataset, with results peaking at only around 0.8. The limited documentation makes it challenging t
30.
by VoVAllen 2y ago
I'm the project lead for both projects. We're still in the process of supporting all the function from pgvecto.rs in VectorChord (int8, more than 2000 dim vec, etc.). We'll provide the migration docs for pgvecto.rs users to V

More ›