34 ms·
I've operated a top ~20 LLM service for over 2 years, very comfortably profitably with ads. As for the pure costs you can measure the cost of getting an LLM ans
by etaioinshrdlu 1y ago
I've operated a top ~20 LLM service for over 2 years, very comfortably profitably with ads. As for the pure costs you can measure the cost of getting an LLM answer from say, OpenAI, and the equivalent search query from Bing/Google/Exa will cost over 10x more...
- clarinificator 1y agoProfitably covering R&D or profitably using the subsidized models?
- guappa 1y agoHe was doing neither. He was using a 3rd party API and has no idea what it costs them to actually run it.
- johnecheck 1y agoSo you don't have any real info on the costs. The question is what OpenAI's profit margin is here, not yours. The theory is that these costs are subsidized by a flow of money from VCs and big tech as they race. How cheap is inference, really? What about 'thinking' inference? What are the prices going to be once growth starts to slow and investors start demanding returns on their billions?
- jsnell 1y agoEvery indication we have is that pay-per-token APIs are not subsidized or even break-even, but have very high margins. The market dynamics are such that subsidizing those APIs wouldn't make much sense. The unprofitability of the frontier labs is mostly due to them not monetizing the majority of their consumer traffic at all.
- etaioinshrdlu 1y agoIt would be profitable even if we self-hosted the LLMs, which we've done. The only thing subsidized is the training costs. So maybe people will one day stop training AI models.
- throwawayoldie 1y agoSo you're not running an LLM, you're running a service built on top of a subsidized API.
- deleted 1y ago[deleted]