6 ms·

How exactly will models get cheaper?

• himata4113 1mo ago
  Compare the performance of a 980 and a 5050 and I am sure that will answer your question. Also models baked into the silicon are able to achieve efficiency that is simply impossible to achieve with programmable circuits, there is a general slowdown in the raw capabilities that transformers can achieve and agentic tool use is simply an amplifier that will reach a wall eventually. It wouldn't surprise me if we saw within 5 to 10 years accelerator cards that you're able to purchase and plug into via usb-c that are able to achieve thousands of tok/s as well as api costs going down to what we already see with subscriptions today. There has been quite a lot of off-ramping going on where people feel satisfied with the performance they're getting out of the models and simply staying there instead of using SOTA.
  • andrekandre 1mo ago
    > accelerator cards that you're able to purchase and plug into via usb-c that are able to achieve thousands of tok/s how do you update that baked-in model for things that have happened in the last say 2 months? if i'm a programmer for example, even being a couple months old is a huge annoyance because programming languages and frameworks are changing all the time...
    • HDBaseT 1mo ago
      When was the last time you heard about someone talking about "Knowledge Cutoff" dates? OpenAI used to make a huge deal about it every release, now its not even mentioned. We give agents tools, the ability to read a man page, the ability to use web search. Knowledge cut-off is far less important than it used to be.
• deleted 1mo ago
  [deleted]