7 ms·
How and why do they get nerfed? To save money?
by d_tr 7d ago
How and why do they get nerfed? To save money?
- varispeed 7d agoYes. They save on compute and customer has to use more tokens to achieve their goal which means more profit.
- cliche 7d agoProfit? I thought these companies were making a massive loss
- johnfn 7d agoModels do not get nerfed. There has never been evidence of this. This would be trivial to prove if it were true, and such a proof would be a huge story and scandal to a news market hungry for a shred of a signal on AI's downfall. This is the "your iPhone is listening to you and serving ads based on what you say" of the 2020s.
- esafak 7d agoYes, they can, through quantization. Many providers of open source models openly serve quantized versions; check openrouter.
- 4chandaily 7d ago> This is the "your iPhone is listening to you and serving ads based on what you say" of the 2020s Of course, it turned out that this wasn't actually completely BS. We just were accusing the wrong vendor. Not disagreeing with you on models.
- johnfn 7d agoI'd be happy to read a source as I am fairly confident that audio transcription -> facebook (or other) ads has never been true.
- 4chandaily 7d agoI was referring to LG specifically. Here is a link from https://news.ycombinator.com/item?id=49592375 https://news.ycombinator.com/item?id=49592375 three days ago.
- johnfn 7d agoWow totally missed this, thanks for sharing
- ricardobeat 7d agoOne theory is that they start serving at full precision, then quantize to save on costs as adoption grows. It's kind of a conspiracy theory atm, but I have definitely felt it - I had a large project done on Opus 4.8 release day, a week later it was struggling to complete partial tasks in the same area.