5 ms·
i run a home media server, can't wait to be able to add my own LLM service. it's just a matter of time for it to be something i can install over the weekend wit
by mattwad 2y ago
i run a home media server, can't wait to be able to add my own LLM service. it's just a matter of time for it to be something i can install over the weekend with proper hardware
- proaralyst 2y agoHave you tried https://ollama.com/ https://ollama.com/ ? You may find you already can
- gryfft 2y agogit clone https://github.com/ggerganov/llama.cpp cd llama.cpp make ./server -m models/7B/ggml-model.gguf -c 2048 I don't think it'll take you the whole weekend :)