xavcochran

4 ms·

1.
HelixDB Query Insights(helix-db.com)
2 points by xavcochran 23d ago|0 comments
2.
I made a byte-range cache for object storage(github.com)
8 points by xavcochran 24d ago|1 comments
3.
by xavcochran 24d ago
I made a byte-range cache for object storage. For full hits it's zero copy. On a miss, it fetches only the missing bytes from object storage needed to serve the request. It merges overlapping ranges and coalesces concurrent misses into
4.
The Battle of the Storage Engines: RocksDB vs. LMDB(helix-db.com)
1 points by xavcochran 1y ago|0 comments
5.
by xavcochran 1y ago
The benefit of baking in the dimension and size of individual elements (the precision) is the fact that the size will be known at compile time meaning it can be allocated on the stack instead of being heap allocated.
6.
by xavcochran 1y ago
We utilize some of LMDB's optimizations such as the APPEND put flags. We also make use of LMDB handling duplicates as a one-to-many key instead of duplicating keys. This means we can get all values for one key in one call rather than a
7.
by xavcochran 1y ago
Looking at your benchmarks you say for inserting 1k edges its around 500,000 ns/iteration. Is this 500,000 ns/per edge insertion or for all 1k of them?
8.
by xavcochran 1y ago
thank you! any feedback would be much appreciated
9.
by xavcochran 1y ago
SPALDE*
10.
by xavcochran 1y ago
there is also the fact that the more dimensions you have for embedded data the more diluted the embedding becomes so it is unusual to go anywhere near the limits of vector length!
11.
by xavcochran 1y ago
very interesting, will look into this. I know for a fact that you cannot compile the likes of LMDB and RocksDB to work with WASM but this looks promising for our custom storage engine to be able to make it work with the browser. Thanks for
12.
by xavcochran 1y ago
We will definitely look into it. The SPLADE models look promising!
13.
by xavcochran 1y ago
to add to George's reply, for helix to run on the browser with WASM the storage engine has to be completely in memory. At the moment we use LMDB which uses file based storage so that does't work with the browser. As George said, w
14.
by xavcochran 1y ago
thanks for the question! we chose f64 as a default for now as just to cover all cases and we believed that basic vector operations would not be our bottleneck initially. As we optimize our HNSW implementation, we are going to add support fo
15.
by xavcochran 1y ago
apart from the fact Cozo seems to be pretty dead, we use a different storage engine which makes our reads much faster. based on their benchmarks I estimate our most of our reads to be 10x faster. I think our query language is much simpler,
16.
by xavcochran 1y ago
Assuming you are using GPUs for model inference, the best way to set it up would have the DB and a separate server to send inference requests. Note that we plan on support custom model endpoints and on the database side so you probably won&
17.
by xavcochran 1y ago
Thanks for the kind words! At the moment the query language transpilation is quite unstable but we are in the process of a large remodel which we aim to finish in the next day or so. This will make the query language compilation far more ro