12 ms·

Does anyone know how they are going to make money and turn a profit one day?

thornewolf 2y ago
LLM inference is getting cheaper year over year. It often loses money now, it may eventually stop losing money when it gets cheap enough to run. - But surely the race to the bottom will continue? Maybe, but they do offer a consumer subscription that can diverge from actual serving costs. /speculation
- lasermike026 2y ago
  I'm working with models and the costs are ridiculous. $7000 card and 800 watts later for my small projects and I can't imagine how they can make money in the next 5 to 10 years. I need to do more research on hardware approaching that reduces costs and power consumption. I just started experimenting with llama.cpp and I'm mildly impressed.
- Palmik 2y ago
  Looking at API providers like Together that host open source models like Llama 70b and running these models in production myself, they have healthy margins (and their inference stack is much better optimized).
sigmar 2y ago
relatedly: is claude3.5-haiku being delivered above their cost, after they quadrupled the price? Though it wouldn't ensure profitability since they're spending so much on training. I'm sure with inference-use growing, they're hoping that eventually total_expenses(inference) grows to be much much larger than total_expenses(training)
km144 2y ago
Same as the big tech companies, probably make all of their products worse in service to advertising. AI-generated advertising prompted by personal data could be extremely good at getting people to buy things if tuned appropriately.
- lucianbr 2y ago
  Well. If you're using AI instead of a search engine, they could make the AI respond with product placement more or less subtle. But if you're using AI for example to generate code as an aid in programming, how's that going to work? Or any other generative thing, like making images, 3d models, music, articles or documents... I can't imagine inserting ads into those would not destroy the usefulness instantly. My guess is they don't know themselves. The plan is to get market shre now, and figure it out later. Which may or may not turn out well.
danny_codes 2y ago
That's the neat part
uptownfunk 2y ago
Cost of inference will tend to the the same as cost of a Google search. It is infra that will come down to negligible and almost free. Then as others have said it will tend to freemium (pay to have no ads). And additional value added services as they continue to evolve up the food chain (ai powered sales, marketing, etc)
staticman2 2y ago
<Sarcasm> They'll invent AGI, put 50% of workers out of a job, then presumably have the AGI build some really good robots to protect them from the ensuing riots. </sarcasm>

LLM inference is getting cheaper year over year. It often loses money now, it may eventually stop losing money when it gets cheap enough to run. - But surely the race to the bottom will continue? Maybe, but they do offer a consumer subscription that can diverge from actual serving costs. /speculation

lasermike026 2y ago
I'm working with models and the costs are ridiculous. $7000 card and 800 watts later for my small projects and I can't imagine how they can make money in the next 5 to 10 years. I need to do more research on hardware approaching that reduces costs and power consumption. I just started experimenting with llama.cpp and I'm mildly impressed.
Palmik 2y ago
Looking at API providers like Together that host open source models like Llama 70b and running these models in production myself, they have healthy margins (and their inference stack is much better optimized).

sigmar 2y ago

relatedly: is claude3.5-haiku being delivered above their cost, after they quadrupled the price? Though it wouldn't ensure profitability since they're spending so much on training. I'm sure with inference-use growing, they're hoping that eventually total_expenses(inference) grows to be much much larger than total_expenses(training)

km144 2y ago

Same as the big tech companies, probably make all of their products worse in service to advertising. AI-generated advertising prompted by personal data could be extremely good at getting people to buy things if tuned appropriately.

lucianbr 2y ago
Well. If you're using AI instead of a search engine, they could make the AI respond with product placement more or less subtle. But if you're using AI for example to generate code as an aid in programming, how's that going to work? Or any other generative thing, like making images, 3d models, music, articles or documents... I can't imagine inserting ads into those would not destroy the usefulness instantly. My guess is they don't know themselves. The plan is to get market shre now, and figure it out later. Which may or may not turn out well.

danny_codes 2y ago

That's the neat part

uptownfunk 2y ago

Cost of inference will tend to the the same as cost of a Google search. It is infra that will come down to negligible and almost free. Then as others have said it will tend to freemium (pay to have no ads). And additional value added services as they continue to evolve up the food chain (ai powered sales, marketing, etc)

staticman2 2y ago

<Sarcasm> They'll invent AGI, put 50% of workers out of a job, then presumably have the AGI build some really good robots to protect them from the ensuing riots. </sarcasm>