5 ms·
Easy prediction: LLMs will get shrunk down further and further until GenAI is just something that ships on a chip as part of your hardware. In the future it wil
by _doctor_love 12d ago
Easy prediction: LLMs will get shrunk down further and further until GenAI is just something that ships on a chip as part of your hardware. In the future it will seem quaint that we needed a network connection to talk to our LLM.
Adoption of open-source models to my mind is a similar step in that direction. In all cases, the goal is to become untethered from a mercurial vendor.
- honr 12d agoLike taalas.com (very recently acquired by AMD), or cerebras.ai (whole wafer is a chip)? As you said, I also think that is one of the main direction many companies (and academia) is moving to.
- wnmurphy 12d agoYeah, I'm looking forward to this actually. https://chatjimmy.ai/ https://chatjimmy.ai/ blew my mind at how fast etched model weights can be. For on-device LLMs, there's a point of diminishing returns, meaning you don't need to have the latest frontier model for most operations.
- hparadiz 12d agoI currently have a small TTS model running in the background on my machine through which my agent(s) speak to me as they work. If that can be baked into an ASIC along with a few thousand voices in every major language then it should just be a utility chip on your mobo for anything that needs it. And yes, I too, am looking forward to it.
- andriy_koval 12d agofor every small GenAI model there will be larger model or cluster of models which are smarter than small model
- hparadiz 12d agoWe're gonna start baking in models like TTS with thousands of voices available in any language as a chip on device. They just need to hit 99% accuracy and then it's a done deal.