5 ms·
I enjoy the Qwen models a lot, but building things on top of them with OpenRouter has been painful. OpenRouter does a lot of great work and I really enjoy bein
by ddtaylor 23d ago
I enjoy the Qwen models a lot, but building things on top of them with OpenRouter has been painful.
OpenRouter does a lot of great work and I really enjoy being able to use different models so easily. I like when a provider is phasing out an older model that still works for my needs and the price is much lower. It seems like such a good win-win.
However, the problem is that many Qwen models have almost no capacity or is so flaky you literally have to just litter your code with a blacklist/whitelist of providers. OpenRouter has some attempts to solve this, but they don't work. In fact, OpenRouter has a lot of really cool stuff that is documented, but if you read the code it's not yet implemented or isn't actually there yet, which is a shame.
I tried to get in contact with them at OpenRouter about this and I was interested in working with them in the past, but it's difficult to get in touch with the right people and they are growing very fast. I expect being acquired by Stripe will accelerate those problems in some ways. I have no doubt they will resolve all of these issues eventually and scaling that much that quickly is really hard, so kudos to them, but the road has been pretty lame and taken some wind out of my sails.
- irthomasthomas 23d agoOpenrouter was pretty great before prompt caching became common. Now it is extremely expensive for most individual workflows, unless you spend a lot of work customizing router preferences, and then you still get a worse cache hit rate than using the provider directly. I only keep $5-$10 in OR for occasional testing.
- dackdel 23d agodidnt stripe acquire open router? so i assumed its sunset.
- copperx 22d agoWhy is caching affected when not using the provider directly?
- irthomasthomas 21d agoI don't know. But take a look at https://openrouter.ai/deepseek/deepseek-v4-pro-0813#pricing https://openrouter.ai/deepseek/deepseek-v4-pro-0813#pricing for instance, where the deepseek provider shows an 85% cache hit rate, while the same one on zenmux is 98%.
- polski-g 22d agoUse the "only" header?
- irthomasthomas 21d agoSomething is off with them so that even using a locked provider does not deliver the same cache rate. See https://openrouter.ai/deepseek/deepseek-v4-pro-0813#pricing https://openrouter.ai/deepseek/deepseek-v4-pro-0813#pricing for instance where deepseek has 85% cache hit rate, and the same one on zenmux is 98%.
- geek_at 23d agoThe best solution to this for me is to self host litellm or a different router and use model aliases. For example I have a model called "coding" and when a new good model comes out I just switch the backend without needing to change the alias or the key in my projects (opencode, etc). I have a few of them even a smart router called "agents" which will use local models but if it thinks the request might require higher reasoning it's routing to a different model
- try-working 23d agoI built a router that lets you route between local and cloud models. Link in my profile.
- embedding-shape 23d agoYeah, I also built my own "router" for this: if (process.env.LOCAL_MODEL { http('localhost:3000/v1/completions')... } else { http('api.openrouter.ai/v1/completions')... }
- nvme0n1p1 23d agoBrilliant! I'd like to invest $10M. Who should I make the check out to?
- npn 23d agoI'm confused? Can you just define some presets and call them instead? With preset you can pinpoint a lot of things, especially the providers
- ljlolel 23d ago[dead]
- runeblaze 23d agogenuine question — how has fireworks or baseten or $reputable_inference_provider worked for your use cases? most production workload probably works fine with one of these and another set as fallback, at least so i think
- vietvu 22d agoWho use openrouter for serious usage? Caching would be really bad unless you use preset to pin to single provider.