5 ms·

Why would having more costs and less income allow them to pass savings on to the end user?

by goos 2mo ago

rjh29 2mo ago
They already invested in the massive datacentres of GPUs sitting idle. They have fewer users so they can deliver more inference per user - more thinking, larger models.
- mvdtnz 2mo ago
  So where are these mythical savings coming from? You're saying they have spent more per user therefore can charge each user less or something? I'm not following.
  - spacebanana7 2mo ago
    The (optimistic?) take is that xAI is genuinely better at building datacenters at scale than anyone else, and the freedom to use Nat Gas as the primary energy source allows them to have lower marginal costs. The (pessimistic?) take is that they have loads of idle GPUs and want to get some revenue out of them rather than none. Compare this to OpenAI/Anthropic where every token used by a consumer has to compete with enterprise spenders, and there’s not enough to go around for everyone.
    - rjh29 2mo ago
      It's also sensible for them to provide a cheap, intelligent model to users if they have capacity, then once they built a user base, tighten the screws. All the other AI providers have done that.
  - collinmcnulty 2mo ago
    It’s basically a clearance sale, is the theory.
- loeg 2mo ago
  Don't they just rent them out to the frontier AI shops? They're not sitting idle.
  - stavros 2mo ago
    They do, yes.
  - londons_explore 2mo ago
    There are various difficulties with renting GPUs, especially if your setup is very custom. The competitor would have to port their training systems to your specific network architecture, system design, rdma Vs ethernet vs infiniband Vs nvlink etc. Getting it running might not be too hard, but getting it running efficiently and making good use of all those flops will require considerable human effort and wall time. Add that to the fact most frontier labs seem to have a single huge training run - and to my knowledge nobody has figured out how to distribute that training run between data centers effectively.
    - rjh29 2mo ago
      Surely rented GPUs would be used for inference, which runs anywhere and needs to scale much larger than a training run.
    - loeg 2mo ago
      I'm not saying this as a hypothetical -- I believe xAI is already renting their GPU farm out to frontier labs. Whatever logistical challenges exist, enough of them seem to have been overcome.
    - solumunus 2mo ago
      They’ve said they’re renting them out.
re-thc 2mo ago
More like they have a less focus on margins and more on cost recovery.
- gabriel-uribe 2mo ago
  Definitely. They had insanely low rates on TTS up until a month or two ago ($4.20/1M) for example, which they only recently started increasing. As their models get more competitive I'm sure prices will catch up.
parsimo2010 2mo ago
“We lose money on every rack, but we make up for it in volume!” - Elon Musk, probably

Why would having more costs and less income allow them to pass savings on to the end user?

by goos 2mo ago

Why would having more costs and less income allow them to pass savings on to the end user?

rjh29 2mo ago

They already invested in the massive datacentres of GPUs sitting idle. They have fewer users so they can deliver more inference per user - more thinking, larger models.

mvdtnz 2mo ago
So where are these mythical savings coming from? You're saying they have spent more per user therefore can charge each user less or something? I'm not following.
- spacebanana7 2mo ago
  The (optimistic?) take is that xAI is genuinely better at building datacenters at scale than anyone else, and the freedom to use Nat Gas as the primary energy source allows them to have lower marginal costs. The (pessimistic?) take is that they have loads of idle GPUs and want to get some revenue out of them rather than none. Compare this to OpenAI/Anthropic where every token used by a consumer has to compete with enterprise spenders, and there’s not enough to go around for everyone.
  - rjh29 2mo ago
    It's also sensible for them to provide a cheap, intelligent model to users if they have capacity, then once they built a user base, tighten the screws. All the other AI providers have done that.
- collinmcnulty 2mo ago
  It’s basically a clearance sale, is the theory.
loeg 2mo ago
Don't they just rent them out to the frontier AI shops? They're not sitting idle.
- stavros 2mo ago
  They do, yes.
- londons_explore 2mo ago
  There are various difficulties with renting GPUs, especially if your setup is very custom. The competitor would have to port their training systems to your specific network architecture, system design, rdma Vs ethernet vs infiniband Vs nvlink etc. Getting it running might not be too hard, but getting it running efficiently and making good use of all those flops will require considerable human effort and wall time. Add that to the fact most frontier labs seem to have a single huge training run - and to my knowledge nobody has figured out how to distribute that training run between data centers effectively.
  - rjh29 2mo ago
    Surely rented GPUs would be used for inference, which runs anywhere and needs to scale much larger than a training run.
  - loeg 2mo ago
    I'm not saying this as a hypothetical -- I believe xAI is already renting their GPU farm out to frontier labs. Whatever logistical challenges exist, enough of them seem to have been overcome.
  - solumunus 2mo ago
    They’ve said they’re renting them out.

re-thc 2mo ago

More like they have a less focus on margins and more on cost recovery.

gabriel-uribe 2mo ago
Definitely. They had insanely low rates on TTS up until a month or two ago ($4.20/1M) for example, which they only recently started increasing. As their models get more competitive I'm sure prices will catch up.

parsimo2010 2mo ago

“We lose money on every rack, but we make up for it in volume!” - Elon Musk, probably