13 ms·
Switching out an LLM? What do you mean by this? Sure some models can run locally but in a company with lots do people they might not be willing to spend to self
by rhipitr 2mo ago
Switching out an LLM? What do you mean by this? Sure some models can run locally but in a company with lots do people they might not be willing to spend to self host a larger model that requires beefier hardware to host, plus all the complexity to scale that out to a bit internal user-base
- KetoManx64 2mo agoMost of the AI companies have OpenAi compatible API's, so you just get a subscription from another provider and change the URL that your LLM Agent Harness uses to talk to the AI. I use OpenRoutet which lets you switch between providers (Anthropic, ChatGPT, Z-AI) whenever you want. Sometimes I'll have two different models from different providers evaluate each other's answers.