6 ms·
Big fan of LiteLLM Proxy and LiteLLM Python SDK to connect to various local models. Might be helpful here as well
by marcklingen 2y ago
Big fan of LiteLLM Proxy and LiteLLM Python SDK to connect to various local models. Might be helpful here as well
- prasoonds 2y agoThanks Marc! We'll check them out. So far, with our limited experience with local models, we'd simply been thinking to use Ollama so thanks for the heads up!
- windexh8er 2y agoAdding some love for local model support. A lot of security use cases are demanding local models. Would love to see local support soon as this looks interesting. I agree that I think starting out with Ollama is probably hits 80% of what people would be looking for out of the box.