8 ms·
I believe embedding-based RAG, everybody is using, will end. As chips advance, you would use a big llm instead of word embedding for retrieval. It's much more a
by respectattentio 21d ago
I believe embedding-based RAG, everybody is using, will end. As chips advance, you would use a big llm instead of word embedding for retrieval. It's much more accurate and extensive covering every topic.
Still need ~2 years to be replaced.
- inigyou 21d agoHow would you use a big LLM for retrieval?
- respectattentio 21d agoAs simple as a prompting it with structural output or restrictions for your criteria. With agents, the prompting could be dynamic for maximum accuracy for every retrieval. This absolutely would beat the best of the best embedding-based RAG models. Nobody uses this now mainly due to speed. An llm retrieval would be 10x or more slower than embedding. You can try that now Take some failing cases or bad retrieval from your current system Prompt an llm wisely like a perfect prompt to get what you want and provide it the context to it. And see the results. For context, you are limited now by models contexts (1m), so mostly you would need to split what you have and prompt twice....or more...and so on
- inigyou 21d agoSo uh ... Where's the retrieval part? You know RAG is used to implement that, right? You're basically saying "we don't need an ALU, we can just use the Windows calculator"
- respectattentio 21d agoThe only difference is using LLMs instead of Embedding models