6 ms·

I fail to understand why I should use it over a different embedded vector DB like LanceDB or Chroma. Both are written in more performant languages, have a simpl

I fail to understand why I should use it over a different embedded vector DB like LanceDB or Chroma. Both are written in more performant languages, have a simple API with a lot of integrations and power if one needs it

dmezzetti 3y ago
To be fair, Chroma is also written in Python. And while LanceDB and others are written in Rust, that doesn't automatically give it super powers.
- PossiblyKyle 3y ago
  Fair point, then you could claim it's similar to this DB with its reliance on Faiss. Despite that, Chroma at this point is more feature rich. I was mostly referring to this https://thedataquarry.com/posts/vector-db-1/ https://thedataquarry.com/posts/vector-db-1/ You are not wrong about the performance from Rust, but LanceDB is inherently written with performance in mind. SIMD support for both x86 and ARM, and an underlying vector storage approach that's built for speed (Lance)
  - marcinzm 3y ago
    >more feature rich Not necessarily a good thing when the product is made by a VC backed startup that may die or pivot in six months leaving you the need to maintain it yourself.
  - dmezzetti 3y ago
    I've seen a number of projects come over the last couple years. I'm the author of txtai (https://github.com/neuml/txtai https://github.com/neuml/txtai) which I started in 2020. How you approach performance is the key point. You can write performant code in any language. For example, for standard keyword search, I wrote a component to make sparse/keyword search just as efficient as Apache Lucene in Python. https://neuml.hashnode.dev/building-an-efficient-sparse-keyword-index-in-python https://neuml.hashnode.dev/building-an-efficient-sparse-keyw....
- 6r17 3y ago
  Python programmer for 15 years and i picked up rust to write an oAuth gateway not long ago ; i wrote it in python beforehand - rust DOES give you superpowers ; especially if you compare it to something like python that isn't nowhere as fast and has no typing
  - DSingularity 3y ago
    Python does have typing. Although it doesn’t feel as “first class” like as rust or golang it gets the job done.
  - dmezzetti 3y ago
    There are plenty of examples of Python libraries that can be performant such as NumPy and PyTorch (which both rely on C/C++). Some libraries such as Hugging Face's tokenizers even use Rust. I referenced this article below but will reference it again here too. https://neuml.hashnode.dev/building-an-efficient-sparse-keyword-index-in-python https://neuml.hashnode.dev/building-an-efficient-sparse-keyw.... You can write performant code in any language if you try.
    - iopq 3y ago
      So to make Python fast you just need to write a library in another language, brilliant
      - dmezzetti 3y ago
        If you read the article referenced, I discussed a number of ways to write performant Python such as using this package (https://docs.python.org/3/library/array.html https://docs.python.org/3/library/array.html).
      - deleted 3y ago
        [deleted]
    - aldanor 3y ago
      NumPy is a C library with Python frontend, moreover lots of functionality based on other existing C libraries like blas etc. PyTorch, quoting themselves, is a Python binding into a monolithic C++ framework; also optionally depending on existing libs like mkl etc. > You can write performant code in any language if you try. Unfortunately, only to a certain extent. Sure, if you just need to multiply a handful of matrices and you want your blas ops to be blas'ed where the sheer size of data outweighs any of your actual code, it doesn't really matter. Once you need to implement lower-level logic, ie traversing and processing the data in some custom way, especially without eating extra memory, you're out of luck with Python/numpy and the rest.
hantusk 3y ago
I thought the API here was quite neat. It's fairly simple to implement a lancedb backend for it instead of sklearn/faiss/mrpt as the source code is really simple. This repo is basically just a nice api and the needed chunking and batching logic. Using lancedb, you'd still have to write that, as exemplified here: https://github.com/prrao87/lancedb-study/blob/main/lancedb/index.py#L69-L103 https://github.com/prrao87/lancedb-study/blob/main/lancedb/i...
mark_l_watson 3y ago
Same for me. I started using Chroma (about) a year ago, I am used to it, and if I am using Python I look no further. When I use Common Lisp or Racket I roll my own simple vector embeddings data store, but that is just me having fun.
deleted 3y ago
[deleted]
freediver 3y ago
It is faster! We needed a low latency, on premise solution that we can run on edge nodes with sane defaults that anyone in the team can whim in a sec. Also worth noting is that our use case is end to end retrieval of usually few hundred to few thousand chunks of text (for example in Kagi Assistant research mode) that need to be processed once at run time with minimal latency. Result is this. We periodically benchmark the performance of different embeddings to ensure best defaults: https://github.com/kagisearch/vectordb#embeddings-performance-analysis https://github.com/kagisearch/vectordb#embeddings-performanc...

dmezzetti 3y ago

To be fair, Chroma is also written in Python. And while LanceDB and others are written in Rust, that doesn't automatically give it super powers.

PossiblyKyle 3y ago
Fair point, then you could claim it's similar to this DB with its reliance on Faiss. Despite that, Chroma at this point is more feature rich. I was mostly referring to this https://thedataquarry.com/posts/vector-db-1/ https://thedataquarry.com/posts/vector-db-1/ You are not wrong about the performance from Rust, but LanceDB is inherently written with performance in mind. SIMD support for both x86 and ARM, and an underlying vector storage approach that's built for speed (Lance)
- marcinzm 3y ago
  >more feature rich Not necessarily a good thing when the product is made by a VC backed startup that may die or pivot in six months leaving you the need to maintain it yourself.
- dmezzetti 3y ago
  I've seen a number of projects come over the last couple years. I'm the author of txtai (https://github.com/neuml/txtai https://github.com/neuml/txtai) which I started in 2020. How you approach performance is the key point. You can write performant code in any language. For example, for standard keyword search, I wrote a component to make sparse/keyword search just as efficient as Apache Lucene in Python. https://neuml.hashnode.dev/building-an-efficient-sparse-keyword-index-in-python https://neuml.hashnode.dev/building-an-efficient-sparse-keyw....
6r17 3y ago
Python programmer for 15 years and i picked up rust to write an oAuth gateway not long ago ; i wrote it in python beforehand - rust DOES give you superpowers ; especially if you compare it to something like python that isn't nowhere as fast and has no typing
- DSingularity 3y ago
  Python does have typing. Although it doesn’t feel as “first class” like as rust or golang it gets the job done.
- dmezzetti 3y ago
  There are plenty of examples of Python libraries that can be performant such as NumPy and PyTorch (which both rely on C/C++). Some libraries such as Hugging Face's tokenizers even use Rust. I referenced this article below but will reference it again here too. https://neuml.hashnode.dev/building-an-efficient-sparse-keyword-index-in-python https://neuml.hashnode.dev/building-an-efficient-sparse-keyw.... You can write performant code in any language if you try.
  - iopq 3y ago
    So to make Python fast you just need to write a library in another language, brilliant
    - dmezzetti 3y ago
      If you read the article referenced, I discussed a number of ways to write performant Python such as using this package (https://docs.python.org/3/library/array.html https://docs.python.org/3/library/array.html).
    - deleted 3y ago
      [deleted]
  - aldanor 3y ago
    NumPy is a C library with Python frontend, moreover lots of functionality based on other existing C libraries like blas etc. PyTorch, quoting themselves, is a Python binding into a monolithic C++ framework; also optionally depending on existing libs like mkl etc. > You can write performant code in any language if you try. Unfortunately, only to a certain extent. Sure, if you just need to multiply a handful of matrices and you want your blas ops to be blas'ed where the sheer size of data outweighs any of your actual code, it doesn't really matter. Once you need to implement lower-level logic, ie traversing and processing the data in some custom way, especially without eating extra memory, you're out of luck with Python/numpy and the rest.

hantusk 3y ago

I thought the API here was quite neat. It's fairly simple to implement a lancedb backend for it instead of sklearn/faiss/mrpt as the source code is really simple. This repo is basically just a nice api and the needed chunking and batching logic. Using lancedb, you'd still have to write that, as exemplified here: https://github.com/prrao87/lancedb-study/blob/main/lancedb/index.py#L69-L103 https://github.com/prrao87/lancedb-study/blob/main/lancedb/i...

mark_l_watson 3y ago

Same for me. I started using Chroma (about) a year ago, I am used to it, and if I am using Python I look no further. When I use Common Lisp or Racket I roll my own simple vector embeddings data store, but that is just me having fun.

deleted 3y ago

[deleted]

freediver 3y ago

It is faster! We needed a low latency, on premise solution that we can run on edge nodes with sane defaults that anyone in the team can whim in a sec. Also worth noting is that our use case is end to end retrieval of usually few hundred to few thousand chunks of text (for example in Kagi Assistant research mode) that need to be processed once at run time with minimal latency. Result is this. We periodically benchmark the performance of different embeddings to ensure best defaults: https://github.com/kagisearch/vectordb#embeddings-performance-analysis https://github.com/kagisearch/vectordb#embeddings-performanc...