6 ms·

Subscriptions are, and will likely remain, the best deal in town. Unfortunately, larger companies aren't able to do that. When your monthly token costs are in t

by timfsu 23d ago

Subscriptions are, and will likely remain, the best deal in town. Unfortunately, larger companies aren't able to do that. When your monthly token costs are in the $5-10k range, the local inference starts to look a lot more attractive

ltbarcly3 23d ago
In the case where you pay for tokens without a subscription, the analysis is still very much not in favor of buying hardware. The assumption previously used was that you can run a Sol level model on an M6 or whatever hardware $20k gives you. That is not true, it was an assumption made to show that even giving your own hardware every reasonable advantage it still loses. Lets compare buying tokens of the best model you might run on your own hardware (still being unrealistic in favor of your own hardware) vs that same class of model on the market. I think one of the best models you might be able to run is GLM 5.4, but lets just look at chinese models generally: $20k workstation, best case: $15k M5 Ultra 512GB, 36-month amortization, ~$440/mo. Runs a GLM-5.3-class model at ~30 tok/s. Saturated 24/7 it produces roughly 58M output tokens/month. Buying those tokens: DeepSeek V4 Pro @ $0.87/M $50 Kimi K2.6 @ $4.00/M $232 GLM-5.3 @ $4.40/M $255 Kimi K3 @ $15.00/M $870 (does not fit on the box) The economics can never work in your favor for buying your own hardware here, unless you can utilize it or sell excess capacity and you have access to nearly free electricity. The reason is someone else can buy the same hardware at scale (or realistically more efficient hardware), park it somewhere with very cheap electricity, and sell tokens. They can get very high utilization that you are not likely to get. And keep in mind I am giving 'your own hardware' no overhead or maintenance cost, despite your condition that it's in a large corporate environment. In reality corporate IT would make it almost impossible to set up and your would need huge lead times to buy the hardware and get it installed.
- maherbeg 23d ago
  Thank you for being explicit with the math! So yes, at that speed for sure. But if the speed goes up? or the ability to batch at the same speed goes up? The economics start to shift. The gap is much closer, and you'd end up with a box you can still use or sell later. Subscription pricing is still the best though!
  - ltbarcly3 23d ago
    As speed goes up the cost / Mtoken will necessarily go down at roughly the same ratio so it will wash out. The still use hardware or sell hardware value is factored in to the amortized monthly cost, it assumes a 3 year markdown, and does not factor in the cost of money which should almost cancel the resale value in the end, which I think is quite accurate (any residual cost on a graphics card after 3 years is so small compared to the current price it should be discounted and in included there). Where you might win by owning your own hardware: - Hardware costs go up, and thus api costs go up. You've locked in your pricing. - Chinese/Open models become illegal/hard to access the way we do now. OpenAI and Anthropic are trying very hard to build a regulatory capture scheme to do this. I think they will be unsuccessful because China just won't participate.
- mrkstu 23d ago
  Leaving out that apparently high RAM Mac's apparently no longer lose value over time...
  - ltbarcly3 23d ago
    There is a critical shortage right now, if the AI datacenter boom slows the prices could crash very quickly.
- throwdbaaway 22d ago
  > $20k workstation, best case: $15k M5 Ultra 512GB, 36-month amortization, ~$440/mo. Runs a GLM-5.3-class model at ~30 tok/s. Saturated 24/7 it produces roughly 58M output tokens/month. For agentic coding, ~90% of the cost comes from cached input tokens. This cost increases quadratically with the session length. If sessions go near 1M context, the number of cached input tokens can easily exceed 1B in a day. GLM-5.3 @ $0.26/M x 1000 = $260/day This is the math to use.
swat535 23d ago
Don't subscriptions have limits and resets? Making them not very usable for an ongoing operation?

by timfsu 23d ago

ltbarcly3 23d ago

In the case where you pay for tokens without a subscription, the analysis is still very much not in favor of buying hardware. The assumption previously used was that you can run a Sol level model on an M6 or whatever hardware $20k gives you. That is not true, it was an assumption made to show that even giving your own hardware every reasonable advantage it still loses. Lets compare buying tokens of the best model you might run on your own hardware (still being unrealistic in favor of your own hardware) vs that same class of model on the market. I think one of the best models you might be able to run is GLM 5.4, but lets just look at chinese models generally: $20k workstation, best case: $15k M5 Ultra 512GB, 36-month amortization, ~$440/mo. Runs a GLM-5.3-class model at ~30 tok/s. Saturated 24/7 it produces roughly 58M output tokens/month. Buying those tokens: DeepSeek V4 Pro @ $0.87/M $50 Kimi K2.6 @ $4.00/M $232 GLM-5.3 @ $4.40/M $255 Kimi K3 @ $15.00/M $870 (does not fit on the box) The economics can never work in your favor for buying your own hardware here, unless you can utilize it or sell excess capacity and you have access to nearly free electricity. The reason is someone else can buy the same hardware at scale (or realistically more efficient hardware), park it somewhere with very cheap electricity, and sell tokens. They can get very high utilization that you are not likely to get. And keep in mind I am giving 'your own hardware' no overhead or maintenance cost, despite your condition that it's in a large corporate environment. In reality corporate IT would make it almost impossible to set up and your would need huge lead times to buy the hardware and get it installed.

maherbeg 23d ago
Thank you for being explicit with the math! So yes, at that speed for sure. But if the speed goes up? or the ability to batch at the same speed goes up? The economics start to shift. The gap is much closer, and you'd end up with a box you can still use or sell later. Subscription pricing is still the best though!
- ltbarcly3 23d ago
  As speed goes up the cost / Mtoken will necessarily go down at roughly the same ratio so it will wash out. The still use hardware or sell hardware value is factored in to the amortized monthly cost, it assumes a 3 year markdown, and does not factor in the cost of money which should almost cancel the resale value in the end, which I think is quite accurate (any residual cost on a graphics card after 3 years is so small compared to the current price it should be discounted and in included there). Where you might win by owning your own hardware: - Hardware costs go up, and thus api costs go up. You've locked in your pricing. - Chinese/Open models become illegal/hard to access the way we do now. OpenAI and Anthropic are trying very hard to build a regulatory capture scheme to do this. I think they will be unsuccessful because China just won't participate.
mrkstu 23d ago
Leaving out that apparently high RAM Mac's apparently no longer lose value over time...
- ltbarcly3 23d ago
  There is a critical shortage right now, if the AI datacenter boom slows the prices could crash very quickly.
throwdbaaway 22d ago
> $20k workstation, best case: $15k M5 Ultra 512GB, 36-month amortization, ~$440/mo. Runs a GLM-5.3-class model at ~30 tok/s. Saturated 24/7 it produces roughly 58M output tokens/month. For agentic coding, ~90% of the cost comes from cached input tokens. This cost increases quadratically with the session length. If sessions go near 1M context, the number of cached input tokens can easily exceed 1B in a day. GLM-5.3 @ $0.26/M x 1000 = $260/day This is the math to use.

swat535 23d ago

Don't subscriptions have limits and resets? Making them not very usable for an ongoing operation?