8 ms·

How does it compare to Bifrost?

• josephh 5d ago
  I'm always confused by LLM proxies that claim to support tool calling. Even for Bifrost that claims to be doing it, at least when I was checking it out, I found out that while it injects the list of MCP tools that's available on the proxy-side, it doesn't actually make the call on client's behalf, and clients get confused by it (response returns MCP call request whose tool doesn't exist on the client-side).
• robertclaus 5d ago
  LiteLLM is basically Bifrost in the Python ecosystem.
  • gcgbarbosa 5d ago
    Yeah, but bifrost seems tighter and claims to use way fewer resources
    • yujonglee 5d ago
      hey - litellm will migrate its core to Rust soon. please follow this issue if you're interested! https://github.com/BerriAI/litellm/issues/31263 https://github.com/BerriAI/litellm/issues/31263
    • yujonglee 5d ago
      also if you had resource issues with your production deployment, keen to hear more details
      • testycool 5d ago
        I don't have specifics since it's been months, but LiteLLM on a hetzner server, deployed through Coolify was using 1-4 GB (cannot exactly, and dont know why either). Bifrost feels very light, though. Will try LiteLLM if it's in Rust. Would also appreciate a slicker aesthetic. I got used to Bifrost shadcn with 0 border radius.
        • yujonglee 5d ago
          - We recently migrated to shadcn, which also comes with dark mode support. - You can subscribe to the issue or email me at yujong at berri.ai I’m happy to provide any support if you’re willing to try out the initial version.
      • deleted 5d ago
        [deleted]