5 ms·
As simple as a prompting it with structural output or restrictions for your criteria. With agents, the prompting could be dynamic for maximum accuracy for ever
by respectattentio 21d ago
As simple as a prompting it with structural output or restrictions for your criteria.
With agents, the prompting could be dynamic for maximum accuracy for every retrieval.
This absolutely would beat the best of the best embedding-based RAG models.
Nobody uses this now mainly due to speed. An llm retrieval would be 10x or more slower than embedding.
You can try that now
Take some failing cases or bad retrieval from your current system
Prompt an llm wisely like a perfect prompt to get what you want and provide it the context to it.
And see the results.
For context, you are limited now by models contexts (1m), so mostly you would need to split what you have and prompt twice....or more...and so on
- inigyou 21d agoSo uh ... Where's the retrieval part? You know RAG is used to implement that, right? You're basically saying "we don't need an ALU, we can just use the Windows calculator"
- respectattentio 21d agoThe only difference is using LLMs instead of Embedding models