6 ms·
Problem is that might go away or get nerfed.
by zamalek 1mo ago
Problem is that might go away or get nerfed.
- ody4242 1mo agothen you switch provider, it's not a monopoly
- formerly_proven 1mo agoIf that happens you can still buy hardware later with almost certainly more (tok/s)/$ and better capabilities to run newer models more efficiently (remember native MXFP4?). Right now basically every generation of accelerator is adding new capabilities. These aren't yearly DirectX 9.0c-compatible GPU performance bumps. As an individual, for average privacy needs (e.g. open source or at-home coding and automation), it's pretty much complete nonsense financially to self-host LLMs currently or select hardware now based on the capability to do so, and pay thousands of bucks extra.