6 ms·

These nascent inference chip efforts are reminding me of the early 3dfx / riva / mach / powervr days. Will be interesting to see if inference chips are here to

by corford 24d ago

These nascent inference chip efforts are reminding me of the early 3dfx / riva / mach / powervr days. Will be interesting to see if inference chips are here to stay and, if so, who the eventual dominant player(s) will be

ehnto 24d ago
Which in turn reminds me of Soundblaster audio cards! I suspect inference chips are closer to the GPU story than the Soundblaster story though. I remember one soundblaster card I bought came with a Lara Croft demo, that exploited the incredible immersion of real time dynamic reverb. Genuinely I think game audio took a few steps back from that heady era, the innovation in audio likely didn't sell as many cards as graphics innovations did.
- rubzah 23d ago
  I recently heard, for the first time, what Space Quest sounded like with a Roland board attached. It was mindblowingly amazing. And all most anyone ever experienced was bleep, bloop.
- noir_lord 24d ago
  On board got "good enough" and the separate cards died away. In fairness on board (depending on the board but on the whole) is pretty good.
- bayindirh 24d ago
  EAX was very powerful in its heyday, but it has died because of a thousand cuts. First we had to have the audio processor. Good EAX was available on top of the line cards, and they were not always cheap. Lower end chips got less features. Then we had to have the speaker setup to have the greatest sound, or needed to get a real 5.1 headphones, which were bulky and never provided the same fidelity. Then Microsoft changed the Windows driver model, cutting the driver's direct access to the card. All of the timing sensitive effects were gone in an instant. I remember installing the new drivers and getting literally nothing. Sound Blaster was the only card with an hardware mixer, and Microsoft didn't feel like enabling them. Mixing at the DirectX layer killed the cards. Soundblaster's very closed stance didn't help them either. None of the cards after Audigy2 worked with Linux when I had my desktop system. After my Audigy2ZS, I moved to Asus Xonar D2X. Its positional audio capabilities were nice, but I mostly bought it for its Linux support and sound quality, and that was top notch in that regards. Then sound cards became commodity. Everybody stopped making good cards. Musicians moved to audio interfaces, audiophiles moved to DACs. Just looked to the SoundBlaster website. Internal cards are very limited. One DAC, one DTS enabled 7.1 sound card for PC cinema systems, three game oriented lower end cards, nothing else.
  - thedougd 24d ago
    The DSPs that could double as a sound card and “soft” modem ruined their market in short order.
  - Razengan 23d ago
    Fucking Microsoft killed off a lot of cool shit with potential during the 1990s
    - jonathanlydall 23d ago
      As you mention the 90's you're probably referring to their embrace, extend, extinguish strategy. This situation with the sound card hardware mixing was quite different and a lot later. Windows Vista moved away from kernel mode drivers where possible to help address the issue of BSODs which were not uncommon on Windows XP. It turns out that Microsoft was incorrectly getting the blame for BSODs when it was actually the fault of buggy video and sound card drivers. While Vista was regarded as a "bad" Windows, it laid most of the foundations which allowed Windows 7 to be regarded as "really good" (by Windows standards). From a hardware perspective, by the time Windows 7 arrived pretty much all drivers had been updated for Vista and had their kinks worked out (mostly, I would still have my NVidia drivers crash on occasion, but because they were user mode now, instead of a BSOD the screen would go black for a few seconds after which my desktop would come back with a Windows pop-up saying something to the effect of "the video drivers crashed and had to be restarted", the real perpetrator now being blamed!). Windows 7 also did optimizations to make things less resource intensive and it also helped that PCs had more RAM compared to when Vista came out. On the UAC front Windows 7 was also much better, they calibrated the UAC prompts to come up less often, but what also happened is a lot of the 3rd party software which was needlessly requiring admin rights for no good reason (except that it was badly written), had finally been fixed by the time Windows 7 was released.
  - jonathanlydall 23d ago
    Creative Labs also turned itself into a brand I actively avoid for various reasons. In early 2000s for example, friend with something like a Sound Blaster Live, but couldn't use it anymore as they lost the drivers, their website only had downloads for driver updates, requiring you to still have a driver CD, so no more CD meant no way to get the drivers. They had not done very much meaningful stuff since the release of EAX and used patents to prevent anyone else from competing with them. Onboard sound cards were by and large indistinguishable from a quality perspective as Creative Labs ones, but cheaper which Creative Labs combated mostly with lawyers as opposed to upping their game. Some guy after being frustrated with a long-standing bug in drivers for their Creative Labs sound card, dug into the binaries and made a fix for it. During which they also discovered that you could simply flip a switch in the driver to unlock features only meant to be available on more expensive hardware. Creative Labs of course went straight to lawyers to shut them down. My brother bought the Creative Labs WoW headset which would have its mic get progressively softer until he would leave and re-join the call/voice chat room. They never released a driver update to fix this. By the time that Microsoft announced no more "hardware acceleration" for sound cards, I had zero sympathy for Creative Labs, I was already convinced that they made pretty shoddy hardware/software and were mostly riding on their reputation from the 90s and some patents they managed to get.
- ryukoposting 24d ago
  Eeeeeh idk about the Sound Blaster comparison. Creative earned their place in the early-mid 90s solely because they were the one company making a sound card with drivers that actually worked properly. It wasn't really the cool reverb effects or wave tables, though those were a nice bonus. It was just "I can tell my computer to make sound and it actually makes sound without days of troubleshooting." Granted, similar things could be said about 3dfx. It's was a 3D card with drivers that actually worked. And then there's the obvious "sound blasters and voodoos go in my computer, jalapeno goes in someone else's computer" thing.
wmf 24d ago
Every company is designing their own chips so the dominant players will be one level down: Broadcom, TSMC, Hynix/Samsung/Micron, etc.
- adventured 24d ago
  There is drastically more power and profit in the software ultimately. Apple is in the software first, the hardware second. Everyone at Apple has been trained to understand this for decades, and Jobs pointed it out endlessly. Apple's real moat is software (services, iOS, experience, MacOS). Windows, Office, Azure, et al. Microsoft accumulated approximately one zillion dollars in profit on the back of software. It's a vastly superior business to anything hardware has traditionally seen. Nvidia is the first true juggernaut hardware profit machine, and the AI boom in extended hardware (RAM, storage) will prove temporary (even if there is a feast during that time). Microsoft's advantage and moat was Windows-Office for decades. It was a far better business than Intel's chip biz. Google is a software company first. Every aspect of what made them and maintains them is software first, hardware second. They're a $400 billion software company. Their ad machine is software. Search is software. Facebook is software. Instagram is software. WhatsApp is software. A $200 billion software company. They're not selling hardware, they're selling ads via software, they're monetizing users that use their software. AWS is at least half software as an entity in terms of complexity, competitive advantage, et al. That's a two trillion dollar business. LLMs can run successfully with various hardware approaches. The software is the value at the end of this, regardless of the hardware under it. The sole exception so far that may be sustainable is Nvidia, and we'll see if the bottom falls out from under that margin monster (China, specialized AI chips, whatever it happens to be that cuts under them massively). Hardware always gets its margin squeezed eventually because it's a manufactured good (with inventory, fabs, etc). Software is hyper margin by default, you have to layer a lot of garbage on top of it to kill the margin. Nvidia is 33 years old, they have had a rich business for three years, that's it. The AI boom is the sole reason anything in hardware has looked great in the past 20 years. Check the margins & op income for the top 20 hardware companies, from TI to AMD to Intel to Nvidia to Micron to Sandisk to Samsung to TSMC to ASML, prior to the AI boom of the past couple years. It won't last indefinitely. And after the return to a more normal environment happens, the hyper margins in software will persist.
ignoramous 24d ago
> who the eventual dominant player(s) will be This couldn't have been easy. The team at OpenAI has worked a miracle. For example, Meta and Microsoft’s AI ASIC programs not getting off the ground despite being at it for much longer shows that cost is only one part of the equation.
- rustystump 24d ago
  I bet cost is of no issue with the capx where it is at. It is almost certainly organizational. Meta throws money at every problem and it never seems to workout for them.
thimabi 24d ago
> Will be interesting to see if inference chips are here to stay To me, the efficiency gains of inference chips are so significant that they are certainly here to stay — barring a revolution of sorts that leads to a world devoid of AI as we know it.
madaxe_again 24d ago
I’d go with “whoever owns the entire vertically integrated ecosystem” - which is where the straight GPU comparison falls flat.
conradfr 24d ago
Also the PhysX cards. Not sure how long they were useful.

by corford 24d ago

ehnto 24d ago

Which in turn reminds me of Soundblaster audio cards! I suspect inference chips are closer to the GPU story than the Soundblaster story though. I remember one soundblaster card I bought came with a Lara Croft demo, that exploited the incredible immersion of real time dynamic reverb. Genuinely I think game audio took a few steps back from that heady era, the innovation in audio likely didn't sell as many cards as graphics innovations did.

rubzah 23d ago
I recently heard, for the first time, what Space Quest sounded like with a Roland board attached. It was mindblowingly amazing. And all most anyone ever experienced was bleep, bloop.
noir_lord 24d ago
On board got "good enough" and the separate cards died away. In fairness on board (depending on the board but on the whole) is pretty good.
bayindirh 24d ago
EAX was very powerful in its heyday, but it has died because of a thousand cuts. First we had to have the audio processor. Good EAX was available on top of the line cards, and they were not always cheap. Lower end chips got less features. Then we had to have the speaker setup to have the greatest sound, or needed to get a real 5.1 headphones, which were bulky and never provided the same fidelity. Then Microsoft changed the Windows driver model, cutting the driver's direct access to the card. All of the timing sensitive effects were gone in an instant. I remember installing the new drivers and getting literally nothing. Sound Blaster was the only card with an hardware mixer, and Microsoft didn't feel like enabling them. Mixing at the DirectX layer killed the cards. Soundblaster's very closed stance didn't help them either. None of the cards after Audigy2 worked with Linux when I had my desktop system. After my Audigy2ZS, I moved to Asus Xonar D2X. Its positional audio capabilities were nice, but I mostly bought it for its Linux support and sound quality, and that was top notch in that regards. Then sound cards became commodity. Everybody stopped making good cards. Musicians moved to audio interfaces, audiophiles moved to DACs. Just looked to the SoundBlaster website. Internal cards are very limited. One DAC, one DTS enabled 7.1 sound card for PC cinema systems, three game oriented lower end cards, nothing else.
- thedougd 24d ago
  The DSPs that could double as a sound card and “soft” modem ruined their market in short order.
- Razengan 23d ago
  Fucking Microsoft killed off a lot of cool shit with potential during the 1990s
  - jonathanlydall 23d ago
    As you mention the 90's you're probably referring to their embrace, extend, extinguish strategy. This situation with the sound card hardware mixing was quite different and a lot later. Windows Vista moved away from kernel mode drivers where possible to help address the issue of BSODs which were not uncommon on Windows XP. It turns out that Microsoft was incorrectly getting the blame for BSODs when it was actually the fault of buggy video and sound card drivers. While Vista was regarded as a "bad" Windows, it laid most of the foundations which allowed Windows 7 to be regarded as "really good" (by Windows standards). From a hardware perspective, by the time Windows 7 arrived pretty much all drivers had been updated for Vista and had their kinks worked out (mostly, I would still have my NVidia drivers crash on occasion, but because they were user mode now, instead of a BSOD the screen would go black for a few seconds after which my desktop would come back with a Windows pop-up saying something to the effect of "the video drivers crashed and had to be restarted", the real perpetrator now being blamed!). Windows 7 also did optimizations to make things less resource intensive and it also helped that PCs had more RAM compared to when Vista came out. On the UAC front Windows 7 was also much better, they calibrated the UAC prompts to come up less often, but what also happened is a lot of the 3rd party software which was needlessly requiring admin rights for no good reason (except that it was badly written), had finally been fixed by the time Windows 7 was released.
- jonathanlydall 23d ago
  Creative Labs also turned itself into a brand I actively avoid for various reasons. In early 2000s for example, friend with something like a Sound Blaster Live, but couldn't use it anymore as they lost the drivers, their website only had downloads for driver updates, requiring you to still have a driver CD, so no more CD meant no way to get the drivers. They had not done very much meaningful stuff since the release of EAX and used patents to prevent anyone else from competing with them. Onboard sound cards were by and large indistinguishable from a quality perspective as Creative Labs ones, but cheaper which Creative Labs combated mostly with lawyers as opposed to upping their game. Some guy after being frustrated with a long-standing bug in drivers for their Creative Labs sound card, dug into the binaries and made a fix for it. During which they also discovered that you could simply flip a switch in the driver to unlock features only meant to be available on more expensive hardware. Creative Labs of course went straight to lawyers to shut them down. My brother bought the Creative Labs WoW headset which would have its mic get progressively softer until he would leave and re-join the call/voice chat room. They never released a driver update to fix this. By the time that Microsoft announced no more "hardware acceleration" for sound cards, I had zero sympathy for Creative Labs, I was already convinced that they made pretty shoddy hardware/software and were mostly riding on their reputation from the 90s and some patents they managed to get.
ryukoposting 24d ago
Eeeeeh idk about the Sound Blaster comparison. Creative earned their place in the early-mid 90s solely because they were the one company making a sound card with drivers that actually worked properly. It wasn't really the cool reverb effects or wave tables, though those were a nice bonus. It was just "I can tell my computer to make sound and it actually makes sound without days of troubleshooting." Granted, similar things could be said about 3dfx. It's was a 3D card with drivers that actually worked. And then there's the obvious "sound blasters and voodoos go in my computer, jalapeno goes in someone else's computer" thing.

wmf 24d ago

Every company is designing their own chips so the dominant players will be one level down: Broadcom, TSMC, Hynix/Samsung/Micron, etc.

adventured 24d ago
There is drastically more power and profit in the software ultimately. Apple is in the software first, the hardware second. Everyone at Apple has been trained to understand this for decades, and Jobs pointed it out endlessly. Apple's real moat is software (services, iOS, experience, MacOS). Windows, Office, Azure, et al. Microsoft accumulated approximately one zillion dollars in profit on the back of software. It's a vastly superior business to anything hardware has traditionally seen. Nvidia is the first true juggernaut hardware profit machine, and the AI boom in extended hardware (RAM, storage) will prove temporary (even if there is a feast during that time). Microsoft's advantage and moat was Windows-Office for decades. It was a far better business than Intel's chip biz. Google is a software company first. Every aspect of what made them and maintains them is software first, hardware second. They're a $400 billion software company. Their ad machine is software. Search is software. Facebook is software. Instagram is software. WhatsApp is software. A $200 billion software company. They're not selling hardware, they're selling ads via software, they're monetizing users that use their software. AWS is at least half software as an entity in terms of complexity, competitive advantage, et al. That's a two trillion dollar business. LLMs can run successfully with various hardware approaches. The software is the value at the end of this, regardless of the hardware under it. The sole exception so far that may be sustainable is Nvidia, and we'll see if the bottom falls out from under that margin monster (China, specialized AI chips, whatever it happens to be that cuts under them massively). Hardware always gets its margin squeezed eventually because it's a manufactured good (with inventory, fabs, etc). Software is hyper margin by default, you have to layer a lot of garbage on top of it to kill the margin. Nvidia is 33 years old, they have had a rich business for three years, that's it. The AI boom is the sole reason anything in hardware has looked great in the past 20 years. Check the margins & op income for the top 20 hardware companies, from TI to AMD to Intel to Nvidia to Micron to Sandisk to Samsung to TSMC to ASML, prior to the AI boom of the past couple years. It won't last indefinitely. And after the return to a more normal environment happens, the hyper margins in software will persist.

ignoramous 24d ago

> who the eventual dominant player(s) will be This couldn't have been easy. The team at OpenAI has worked a miracle. For example, Meta and Microsoft’s AI ASIC programs not getting off the ground despite being at it for much longer shows that cost is only one part of the equation.

rustystump 24d ago
I bet cost is of no issue with the capx where it is at. It is almost certainly organizational. Meta throws money at every problem and it never seems to workout for them.

thimabi 24d ago

> Will be interesting to see if inference chips are here to stay To me, the efficiency gains of inference chips are so significant that they are certainly here to stay — barring a revolution of sorts that leads to a world devoid of AI as we know it.

madaxe_again 24d ago

I’d go with “whoever owns the entire vertically integrated ecosystem” - which is where the straight GPU comparison falls flat.

conradfr 24d ago

Also the PhysX cards. Not sure how long they were useful.