4 ms·
You could just run a local LLM over every document and ask it "is this related to this query". I don't think you actually want to wait a week (and holding all t
by osmarks 1y ago
You could just run a local LLM over every document and ask it "is this related to this query". I don't think you actually want to wait a week (and holding all the documents you might ever want to search would run to petabytes).
(the reasonable way is embedding search, which runs much faster with some precomputation, but you still have to store things)
- kortilla 1y agoThe entire library of Congress is like 10TB. You don’t need anything near petabytes until you get out of text into rich media.
- osmarks 1y agoCommon Crawl is petabytes. Anna's Archive is about a petabyte, but it includes PDFs with images.
- amelius 1y agoA better way would be to ask the LLM to generate keywords (or queries). And then use old school techniques to find a set of documents, and then filter those using another LLM.
- brookst 1y agoHow is that better than embeddings? You’re using embeddings to get a finite list of keywords, throwing out the extra benefits of embeddings (support for every human language, for instance), using a conventional index, and then going back to embeddings space for the final LLM? That whole thing can be simplified to: compute and store embeddings for docs, compute embeddings for query, find most similar docs.
- osmarks 1y agohttps://arxiv.org/abs/2212.10496 https://arxiv.org/abs/2212.10496