7 ms·

> As things settle down and commoditize, the value of switching on a dime diminishes as people lock into their favorite models I can imagine just the opposite

> As things settle down and commoditize, the value of switching on a dime diminishes as people lock into their favorite models I can imagine just the opposite outcome from the same scenario: as people settle into their favorite but commoditized models, competition for marginal inference cost will take over. A company like OpenRouter that promises the cheapest tokens by the minute becomes essential on the low-cost margin. I think that OpenRouter and equivalents get pushed out of the market only if the froth calms down (as you posit) and winning models stay proprietary, perhaps with their own unique API surfaces.

drob518 28d ago
> A company like OpenRouter that promises the cheapest tokens by the minute becomes essential on the low-cost margin. Okay, I can see that, but if their value is just cost optimization, their ability to mark up the tokens becomes increasingly difficult as well. Or, people will build a router themselves to avoid paying the markup, possibly with reduced features, but someone will open source it. Heck, Claude or GPT can probably one-shot it these days. Either way, I think the whole OpenRouter model is going to struggle unless the market stays frothy.
- dannyw 28d ago
  First, cost optimization directly contributes to how much markup they can charge. Say on average they deliver savings of 20%, that is excess value OR/Stripe can mostly capture. Also, you don't need to mark up tokens much if you're a commodity volume business. Think of Costco and their margins & membership fees. Not everything has to be high margin, not everything has to be a SaaS subscription.
  - drob518 27d ago
    Well, it depends on how they deliver that savings. If it’s through something unique and defensible, something that only they (with some moat) can do, then yes, they can argue for a fraction of the savings. But if prices just go lower because things are commodities, then they are going to be viewed as middleman markup on top of already low prices. In that case, their value add is considerably lower.

drob518 28d ago

> A company like OpenRouter that promises the cheapest tokens by the minute becomes essential on the low-cost margin. Okay, I can see that, but if their value is just cost optimization, their ability to mark up the tokens becomes increasingly difficult as well. Or, people will build a router themselves to avoid paying the markup, possibly with reduced features, but someone will open source it. Heck, Claude or GPT can probably one-shot it these days. Either way, I think the whole OpenRouter model is going to struggle unless the market stays frothy.

dannyw 28d ago
First, cost optimization directly contributes to how much markup they can charge. Say on average they deliver savings of 20%, that is excess value OR/Stripe can mostly capture. Also, you don't need to mark up tokens much if you're a commodity volume business. Think of Costco and their margins & membership fees. Not everything has to be high margin, not everything has to be a SaaS subscription.
- drob518 27d ago
  Well, it depends on how they deliver that savings. If it’s through something unique and defensible, something that only they (with some moat) can do, then yes, they can argue for a fraction of the savings. But if prices just go lower because things are commodities, then they are going to be viewed as middleman markup on top of already low prices. In that case, their value add is considerably lower.