tjgreen

6 ms·

1.
by tjgreen 3mo ago
ParadeDB is AGPL so not generally available on the hyperscalars. However, you can use https://github.com/timescale/pg_textsearch on Azure HorizonDB (and likely soon Flex). Disclosure: I'm the pg_textsearch maint
2.
by tjgreen 6mo ago
A little birdie told me that efforts are underway to support the extension in Alloy, at least!
3.
by tjgreen 6mo ago
Partitions would be one option, and we've got pretty robust partitioned table support in the extension. (Timescaledb uses partitioning for hypertables, so we had to front-load that support). Expression indexes would be another option
4.
by tjgreen 6mo ago
There is indeed such a tradeoff. The architecture is designed with an eye towards making this tradeoff tunable (frequency of memtable spills, aggressiveness of compaction) but the work here is not yet finished. We chose to prioritize opti
5.
by tjgreen 6mo ago
Yes, hybrid search is one of the main current use cases we had in mind developing the extension, but it works for old-fashioned standalone keyword-only search as well. There is a lot of art to how you combine keyword and semantic search (t
6.
by tjgreen 6mo ago
You'll have to ask Supabase!
7.
by tjgreen 6mo ago
For 2M scale dataset, you should be able to index in about 1 minute on low-end hardware. See the MS-MARCO v1 (8M documents) numbers, measured on cheap Github runners.
8.
by tjgreen 6mo ago
Yep, there are numbers in the blog post and repo. We are able to index MS-MARCO v2 (138M documents, around 50GB of raw data) in a bit under 18 minutes.
9.
by tjgreen 6mo ago
Okay then!
10.
by tjgreen 6mo ago
I actually don't love this example either, for the reasons you mention, but at some point we had questions about how to filter based on numeric ranking. Thanks for the reminder to revisit this. Re filtering, there are often reasonable
11.
Show HN: Postgres extension for BM25 relevance-ranked full-text search(github.com)
203 points by tjgreen 6mo ago|56 comments
12.
Show HN: Pg_textsearch – BM25 Ranking for Postgres(docs.tigerdata.com)
13 points by tjgreen 11mo ago|0 comments