4 ms·
Hi, I've been trying to expose the llama.cpp as an API, you mentioned that found llamacpphtmld to solve this problem, can you share the link to this project? co
by sert_121 3y ago
Hi, I've been trying to expose the llama.cpp as an API, you mentioned that found llamacpphtmld to solve this problem, can you share the link to this project? couldn't find it over github. Thanks!
- stavros 3y agoIs it not this? https://git.ivysaur.me/code.ivysaur.me/llamacpphtmld https://git.ivysaur.me/code.ivysaur.me/llamacpphtmld
- mab122 3y agoHi I was also looking into this and I am now using: https://github.com/abetlen/llama-cpp-python https://github.com/abetlen/llama-cpp-python It tries to be compatible with openAI API. I managed to run AutoGPT using it (however context window is too small to be useful and even if I set it to 2048 (max) I had to tweak AutoGPT context maximum as 1024 for it to work - probably some additional wrapping or something)