6 ms·
We just held a workshop about this a few weeks ago: https://red.ht/llmappdev https://red.ht/llmappdev We created a simple chatbot using local models with Ollama
by sroecker 3y ago
We just held a workshop about this a few weeks ago: https://red.ht/llmappdev https://red.ht/llmappdev
We created a simple chatbot using local models with Ollama (llamacpp), LlamaIndex and streamlit.
Have a look at the streamlit folder, it's super easy.
I used this simple example to teach about RAG, the importance of the system prompt and prompt injection.
The notebook folder has a few more examples, local models can even do natural language SQL querying now.
- 3abiton 3y agoLlamaindex has so mucu potential. Any benchmarks on performance compared to fine-tuning?
- sroecker 3y agoYou probably don't need fine-tuning, at least if it's just new content (and no new instructions). It may even be detrimental, since LLMs are als good at forgetting: https://twitter.com/abacaj/status/1739015011748499772 https://twitter.com/abacaj/status/1739015011748499772
- agilob 3y agolooks very promising, do you plan to keep this single repo up to date as new things are released?
- sroecker 3y agoGood question, as you can see I haven't touched it for a month. I wanted to show what's possible then with open source and (open) local models and there's already so much new stuff out there. I'll probably fix some things this week and then either update it or start from scratch. Guided generation, structured extraction, function calling and multi-modal are things I wanted to add and chainlit looks interesting.