detente18

5 ms·

1.
by detente18 3mo ago
1. I think rust is the better long term solution over go. Especially for a gateway. You can see other proxy's (e.g. cloudflare's pingora - https://blog.cloudflare.com/how-we-built-pingora-the-proxy-t... ) doing the
2.
by detente18 4mo ago
this is pretty interesting. nice work!
3.
Show HN: Lite-Harness – Self-Hosted Cursor Agents (Use Claude Code/OpenCode)(github.com)
6 points by detente18 4mo ago|0 comments
4.
by detente18 6mo ago
Update 2 (03/25/2026): - We will be holding a townhall on Friday to review the incident and share next steps ( https://lnkd.in/gsbTdCe7 ) - We can confirm a bad version of Trivy security scanner ran in our CI/C
5.
by detente18 6mo ago
I deleted it, to be safe.
6.
by detente18 6mo ago
It was the PYPI_PUBLISH token which was in our github project as an env var, that got sent to trivvy. We have deleted all our pypi publishing tokens. Our accounts had 2fa, so it's a bad token here. We're reviewing our accounts, to
7.
by detente18 6mo ago
Update: - Impacted versions (v1.82.7, v1.82.8) have been deleted from PyPI - All maintainer accounts have been changed - All keys for github, docker, circle ci, pip have been deleted We are still scanning our project to see if there's
8.
by detente18 6mo ago
LiteLLM maintainer here, this is still an evolving situation, but here's what we know so far: 1. Looks like this originated from the trivvy used in our ci/cd - https://github.com/search?q=repo%3ABerriAI%2Flitellm%2
9.
LiteLLM (YC W23) is hiring a back end engineer(ycombinator.com)
1 points by detente18 1y ago
10.
by detente18 2y ago
Should we expect the current API ( https://docs.goodfire.ai/introduction ) to be good for prod or is it for testing only? I see that the sampling API is OpenAI-compatible (nice!). Considering if we can add a native integratio
11.
by detente18 2y ago
I wish you could filter by startup name. Would be curious to know where we (litellm) ended up
12.
by detente18 2y ago
Saw you just support claude for now. If your backend code is in Python, happy to add support for other models (OpenAI/Gemini/etc.) via a consistent API interface like LiteLLM - https://docs.litellm.ai/docs/#li
13.
by detente18 2y ago
Was looking at adding this to litellm - https://docs.litellm.ai/docs/providers but the website crashed on me while reviewing the docs. Is this stable yet?
14.
by detente18 2y ago
Your changelog is neat - is this custom built or via some embeddable tool?
15.
by detente18 2y ago
Interesting - how do you use LiteLLM + OpenRouter here? I would assume they're alternatives
16.
by detente18 2y ago
Just saw `from litellm import completion` code Thank you for using litellm + all your issues on our repo! @Ey7NFZ3P0nzAe - litellm maintainer
17.
by detente18 2y ago
It's interesting to see how short-lived some of these were (e.g. HumanEval).
18.
by detente18 2y ago
Interesting — does your backend server use Python? I couldn't find much about it on your site. It would be great to see this tested with more commercial LLMs (O1 / Amazon Nova, / Llama 3.2 / etc.). If you're open to
19.
by detente18 2y ago
+1 on the github issues. It's very useful to have this on the litellm docs
20.
by detente18 2y ago
nice! thanks for using litellm! - maintainer :)
21.
by detente18 3y ago
Hey, are you just looking for litellm - https://github.com/BerriAI/litellm context - i'm the repo maintainer
22.
by detente18 3y ago
LiteLLM proxy (100+ LLMs in OpenAI format) is exactly compatible with the OpenAI endpoint. Here's how to call it with the openai sdk: ``` import openai client = openai.OpenAI( api_key="anything", # proxy key - if set
23.
by detente18 3y ago
Here's the template I'm using - https://github.com/BerriAI/litellm/blob/5ca8b23e22139a4f49bd... Anything I'm doing incorrectly?
24.
by detente18 3y ago
nice - is this using LiteLLM for making the completion calls?
25.
Interesting Post about Binance(hackernoon.com)
1 points by detente18 3y ago|0 comments
26.
by detente18 3y ago
*Update*: for those asking how to run self-hosted llama2, we just added the ollama integration. Here's the tutorial - https://github.com/BerriAI/litellm/blob/main/cookbook/liteLL...
27.
by detente18 3y ago
Any reason you're doing that vs. using Lambda Labs / Replicate / together.ai / Banana.dev, etc. There's a lot of good model deployment platforms that would make it easy to call your model behind a hosted endpoint --
28.
by detente18 3y ago
You're probably calling openai from a backend server. If you're making the call because a user asked a question -> your endpoint received it -> did some processing -> called openai -> returned the response, why would y
29.
by detente18 3y ago
If you'd like to add any of these ideas as notebooks to our cookbook - https://github.com/BerriAI/litellm/tree/main/cookbook , we'd love the contribution!
30.
by detente18 3y ago
Those are really cool use-cases. I wrote a bit about our initial motivation for LiteLLM here - https://hackernoon.com/litellm-call-every-llm-api-like-its-o... . tldr; reliable model switching involved multiple 100-line if&#x

More ›