6 ms·
If like me you run models locally, it's pretty easy to run your own RAG locally also using a Vector Database like Qdrant for persistence, and a middle-layer lik
by saltysalt 24d ago
If like me you run models locally, it's pretty easy to run your own RAG locally also using a Vector Database like Qdrant for persistence, and a middle-layer like Mem0 for realtime retrial and updates. I documented the set-up steps here: https://leadprompt.sh/a/739-Building-an-Infinite-Memory-Local-AI-Stack-on-Fedora https://leadprompt.sh/a/739-Building-an-Infinite-Memory-Loca...
- bityard 23d agoThanks for the nice article. If you're looking for feedback, I'd suggest adding a short demo at the end. It would be nice to see you send it a prompt that says, "hey, remember this" and then tell it to recall that memory. Or show what the memories look like on their way to the model. Are the memories added to the context on every turn or only once per conversation?
- saltysalt 21d agoThanks for the feedback, and that's a great idea I should have done that! They are extracted and added each turn, all handled by the same middleware proxy that also handles the retrievals. I put the full code for that (memory_proxy.py) in the article.