adchurch

16 ms·

1.
Weave (YC W25) is hiring ML, AI, product, & design engineers(jobs.ashbyhq.com)
1 points by adchurch 16d ago
2.
by adchurch 3mo ago
Effectively yes (based on cost though, not raw token count)
3.
by adchurch 3mo ago
We trained a model to select which LLM to call at any given turn, based on lots of agent traces
4.
by adchurch 3mo ago
Yes the open source models are very good, that’s a big part of what makes this router save so much money in practice! There definitely are some things they still don’t handle well though where you do want a frontier model
5.
by adchurch 3mo ago
Yes we can route to Gemini models too and we handle all the translation complexity there!
6.
by adchurch 3mo ago
We welcome the competition :)
7.
by adchurch 3mo ago
Yep exactly
8.
by adchurch 3mo ago
Yes because it's a model explicitly trained to make model selections! Opus probably doesn't have a great idea of when to send a task to DeepSeek vs. to Sonnet, for example.
9.
by adchurch 3mo ago
We haven't experimented with routing to local LLMs much. Technically they benefit from the cache too although it's more a question of latency than cost. But tbh I haven't seen great results in the wild from working with local
10.
by adchurch 3mo ago
I think the key detail here is that we use embeddings of the prompt + previous context in order to decide where to route the request, and if one model is getting stuck we can course correct and move to a different model. So: we can reason
11.
by adchurch 3mo ago
We consider the cost of missing the cache when making each routing decision after the initial one. Discussed in a bit more depth here: https://news.ycombinator.com/item?id=48689448
12.
by adchurch 3mo ago
Good questions. From what I can tell, vLLM semantic router is more optimized for one-off prompt/response workflows rather than agentic coding (I don't think it's cache aware). As another commenter ( https://news.yco
13.
by adchurch 3mo ago
Cool, interested to see your approach when you do launch! I think it's a really interesting problem
14.
by adchurch 3mo ago
Great question! Our main product quantifies engineering productivity & quality so I think we're uniquely qualified to answer this - our velocity has only gone up and our quality (bugs introduced, code turnover) has not budged per o
15.
by adchurch 3mo ago
Oh interesting, didn't know Cursor did that! Totally makes sense though, routing subagents is def the easiest win, no need to have any cache awareness.
16.
by adchurch 3mo ago
If you have a Claude sub with subsidized usage we use that. If not you pay API prices.
17.
by adchurch 3mo ago
Really appreciate the thoughtful feedback! 1. Agree it's important, fwiw the proxy model doesn't blow this up though - only incurs a 1 time cost when switching models and we're aware of that when making routing decisions 2. T
18.
by adchurch 3mo ago
Appreciate the kind words! Lmk if you have any feedback on it from using!
19.
by adchurch 3mo ago
I would argue they do not have a good incentive to build this and make it better. Why would Anthropic route Claude Code traffic to DeepSeek (at 20% of the cost)?
20.
by adchurch 3mo ago
Very important consideration, addressed it in another thread ( https://news.ycombinator.com/item?id=48689448 ). tl;dr we built this to be cache aware for exactly this reason
21.
by adchurch 3mo ago
When we started building this we did it as an experiment and we thought the same thing might be true (cache misses would make the whole thing pointless). This turned out not to be true! I think there are 3 reasons intuitively: 1. Small mode
22.
by adchurch 3mo ago
Fair enough, not meant to be marketing just a statement of fact. Would have turned me off too 18 months ago but times change...
23.
by adchurch 3mo ago
It's a real concern! We take this stuff super seriously ( https://trust.mycroft.io/weave ) and tbh most of our customers opt for the hosted version because it's much simpler on their end + they're already trust
24.
by adchurch 3mo ago
Yep cache awareness is super important, mentioned this in another thread here: ( https://news.ycombinator.com/item?id=48689448 ) But intuitively I think it makes sense that a model can learn what model to route things to if i
25.
by adchurch 3mo ago
I guess delivering business value is always #1, I just meant it's the biggest problem they're trying to solve. Here's a recent example that was public: https://fortune.com/2026/05/26/uber-coo-
26.
by adchurch 3mo ago
The choice on the first turn is super important for this reason! But if a user prompt sends the convo in a very different direction then often it does make sense to reroute at that point.
27.
by adchurch 3mo ago
Nice, let me know any feedback you have from trying it out!
28.
by adchurch 3mo ago
Very interesting - curious how you've used it yourself so far? I can imagine one use case would be having e.g. GPT 5.5 review Opus 4.8's work?
29.
by adchurch 3mo ago
Yep 100%, mentioned this in another thread ( https://news.ycombinator.com/item?id=48689448 ) but tl;dr we build the router to be cache aware
30.
by adchurch 3mo ago
Indeed it is :) I skipped over talking about all the RL machinery, network design, reward function design, state representations, etc. because really the intuition is that we tell the model when it accomplishes its goal, and then it learns

More ›