4 ms·

I wouldn't say they suck, but they're general-purpose. Anyway, you've gotten into the core of what makes Modular's stack useful. GPUs are great, but it has beco

I wouldn't say they suck, but they're general-purpose. Anyway, you've gotten into the core of what makes Modular's stack useful. GPUs are great, but it has become obvious that they leave performance on the table, so dedicated AI accelerators get built; however, these don't have software infrastructure and kernels that have been tuned and optimised for years, like CUDA. The bet here is that you can build great hardware, and with a little bit of effort you can unlock all the necessary software for your hardware.

adastra22 26d ago
Even general purpose matrix multipliers built for large linear systems instead of 3D graphics would get you orders of magnitude improvement. That’s essential what the current generation of AI accelerators are. Dedicated hardware could improve that further. I think that counts as “kinda sucks” in the absolute sense. It’s just far faster and more efficient than any other thing you can buy off the shelf today.