jtarchie

6 ms·

1.
Yet Another SQLite-Vector(github.com)
1 points by jtarchie 6mo ago|1 comments
2.
by jtarchie 6mo ago
sqlite extension that uses database tables for indexing HNSW values. This is something I wanted to build after trying to build my own in pure Go. Warning: Was vibe coded with my instructions of architecture, testing preferences (LuaJIT for
3.
by jtarchie 11mo ago
ncruces helped me with some code I made for VFS. It uses [zstd seekable]( https://github.com/jtarchie/sqlitezstd ) for reading a file. I thought it would be really well-suited for S3. - Support for HTTP range queries - &
4.
by jtarchie 2y ago
There are two sections that describe the same thing. - Try: turning off synchronous mode - Try: synchronous “OFF” Was this AI generated?
5.
by jtarchie 2y ago
I'm doing a little detective work here. 1. A posting like this appeared on [Reddit]( https://www.reddit.com/r/programming/comments/1e79xoz/creati... ) two months ago. This repo started two months ago
6.
by jtarchie 2y ago
Note: We've crossed paths on another thread (Reddit?). I have an experiment going (as part of a talk) of how compressed of a sqlite database I can get all of English Wikipedia with full-text search. So far, the smallest with full artic
7.
by jtarchie 2y ago
I'm glad it was helpful. It was a list of things I had known before I used it. So it was meant to be a quick getting started like doc.
8.
by jtarchie 2y ago
Someone on Reddit mentioned in Norwegian, it means "nothing."
9.
by jtarchie 2y ago
I've used Pocketbase for some other projects. A quick small web app, but it was so easy.
10.
by jtarchie 2y ago
I did see that. I'm going to let the fork simmer for awhile. I didn't require module support for my runtime. Since its meant to be very basic ETL runtime.
11.
by jtarchie 2y ago
Not doing it all, Go allows my API endpoint to accept sandboxed javascript code to enable users to run more complex queries against my data. It is also faster because they don't have to make individual API calls for each data set. Ther
12.
by jtarchie 2y ago
I just wanted to share the details of my experience with Goja. It was not relevant to compare Otto.
13.
SQLite Newsletter(sqlite.fun)
3 points by jtarchie 2y ago|0 comments
14.
by jtarchie 2y ago
Currently 55GB, for read-only work at the moment. Using full text search, rtree, JSON, and indexes.
15.
by jtarchie 2y ago
This is almost the same. We both do compression. The compression layer it transparent to the application via VFS. I have tried this trick with Wikipedia too. It don’t have the numbers. I’ll push up the code when I can.
16.
by jtarchie 2y ago
I can help answer this a bit. Processing the data with code and SQLite takes about 4 hours for the United States and Canada. If I can figure out how to parallelize some of the work, it could be faster. The benefit of using SQLite is its low
17.
by jtarchie 2y ago
Oh, you!
18.
by jtarchie 2y ago
That's great. I was impressed with DuckDB's speed with the OSM data. I'm doing it with all the data, but I intended to have: - a single file that can be queried - a single file that can be easily compressed When I index the e
19.
by jtarchie 2y ago
I do some sharding, mainly by grid-based bounding boxes. This allows for searching with an area or overlapping areas FTS index. It means there is some data duplication, but it is minimal compared to the speed boost.
20.
by jtarchie 2y ago
The FTS table also has the extra benefit of different tokenizers -- porter, unicode, trigram, etc. It allows less precise searches to happen but still returns some results. I liked this because I wanted the best effort, not exact results fo
21.
by jtarchie 2y ago
That's an interesting take. I also found [flatgeobuf]( https://github.com/flatgeobuf/flatgeobuf ), which promises something similar-ish. Perhaps not the same interface for querying, but organizing data in an r-tree
22.
by jtarchie 2y ago
I understand this is how they would work for a JSON/JSONB payload in sqlite. In Postgres, there are [indexes]( https://www.postgresql.org/docs/current/datatype-json.html#J... ), but sqlite doesn't support
23.
by jtarchie 2y ago
Grammarly, but point taken. Will fix.
24.
by jtarchie 2y ago
I have not tried that. Will PGO work with CGO external libraries? I'm using https://github.com/mattn/go-sqlite3
25.
by jtarchie 2y ago
At first, I didn't do it, but I had that shower thought of "What If?" It worked! It was super helpful in finding more accurate results.
26.
by jtarchie 2y ago
I did try DuckDB at first. Not for tags, but the relationships between the nodes, ways, and relations. It was slow. Really fast to important, however!
27.
by jtarchie 11y ago
Would you say your knowledge of Cloud Foundry and Diego is directly proportional to how fancy you dress on Fridays? Because today is Friday and I don't see fancy dress.