7 ms·
Embedding based RAG will always just be OK at best. It is useful for little parts of a chain or tech demos, but in real life use it will always falter.
by leetharris 11mo ago
Embedding based RAG will always just be OK at best. It is useful for little parts of a chain or tech demos, but in real life use it will always falter.
- sgt 11mo agoWhat do you recommend? Query generation?
- esafak 11mo agoCompared with what?
- leetharris 11mo agoFull text agentic retrieval. Instead of cosine similarity on vectors, parsing metadata through an agentic loop. To give a real world example, the way Claude Code works versus how Cursor's embedded database works.
- lifty 11mo agoHow do you do that on 5 million documents?
- leetharris 11mo agoPeople are usually not querying across 5 million documents in a single scope. If you want something as simple as "suggest similar tweets" or something across millions of things then embeddings still work. But if you want something like "compare the documents across these three projects" then you would use full text metadata extraction. Keywords, summaries, table of contents, etc to determine data about each document and each chunk.
- charcircuit 11mo agoMost of my ChatGPT queries use RAG (based on the query ChatGPT will decide if it needs to search the web) to get up to date information about the world. In reality life it's effective and it's why every large provider supports it.
- underlines 11mo agorag will be pronounced differently ad again and again. it has its use cases. we moved to agentic search having rag as a tool while other retrieval strategies we added use real time search in the sources. often skipping ingested and chunked soueces. large changes next windows allow for putting almost whole documents into one request.
- phillipcarter 11mo agoNot necessarily? It's been the basis of one of the major ways people would query their data since 2023 on a product I worked on: https://www.honeycomb.io/blog/introducing-query-assistant https://www.honeycomb.io/blog/introducing-query-assistant The difference is this feature explicitly isn't designed to do a whole lot, which is still the best way to build most LLM-based products and sandwich it between non-LLM stuff.
- DSingularity 11mo agoSuper useful for grounding which is often the only way to robustly protect against hallucinations.