6 ms·
Hello, I have a few questions: - how much time to insert 15 millions of vectors of 768 f32? - how much RAM needed for this operation? - if inserting another
by dureuill 3y ago
Hello, I have a few questions:
- how much time to insert 15 millions of vectors of 768 f32?
- how much RAM needed for this operation?
- if inserting another vector, how incremental is the insertion? Is it faster than reindexing the 15M + 1 vectors from scratch?
- does the structure need to stay in RAM or can it be efficiently queried from a serialized représentation?
- how fast is the search in the 15M vectors on average?
- teaearlgraycold 3y agoI can answer #3. HNSW will allow for incremental index rebuilding. So each additional insert is a sublinear, but greater than constant time, operation.
- andre-z 3y agoI can answer how it would be in Qdrant if interested. The index will take around 70GB RAM. New vectors are first placed in a non-indexed segment and are immediately available for search while the index is being built. The vectors and the index can be offloaded to disk. Search will take some milliseconds.