7 ms·

Is this a case of of Qualcomm commoditizing their complements? Does Mojo have a potential for disrupting Nvidia's current market position? https://www.joelonso

Is this a case of of Qualcomm commoditizing their complements? Does Mojo have a potential for disrupting Nvidia's current market position? https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/ https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/

melodyogonna 27d ago
I think people underestimate the fact that Nvidia's GPUs are really good. Modular/Qualcomm's stack will provide a lot more optionality, but from what I saw in the Modcon conference on Tuesday, most of the target audience is towards other hardware vendors. At the moment, after designing a new hardware, you'd have to also build the entire software stack already existing on Cuda, ROCm, etc - this can double or triple your hardware bring up time. If you plug into Max (not Mojo), you get most of the software stack for free, so your software bring up time can go from months of engineering hours to probably weeks.
- adastra22 27d ago
  For AI? GPUs actually kinda suck. NVIDIA’s GPUs are best, and their software stack is great, but they aren’t really adapted to AI (training or inference).
  - melodyogonna 26d ago
    I wouldn't say they suck, but they're general-purpose. Anyway, you've gotten into the core of what makes Modular's stack useful. GPUs are great, but it has become obvious that they leave performance on the table, so dedicated AI accelerators get built; however, these don't have software infrastructure and kernels that have been tuned and optimised for years, like CUDA. The bet here is that you can build great hardware, and with a little bit of effort you can unlock all the necessary software for your hardware.
    - adastra22 26d ago
      Even general purpose matrix multipliers built for large linear systems instead of 3D graphics would get you orders of magnitude improvement. That’s essential what the current generation of AI accelerators are. Dedicated hardware could improve that further. I think that counts as “kinda sucks” in the absolute sense. It’s just far faster and more efficient than any other thing you can buy off the shelf today.

melodyogonna 27d ago

I think people underestimate the fact that Nvidia's GPUs are really good. Modular/Qualcomm's stack will provide a lot more optionality, but from what I saw in the Modcon conference on Tuesday, most of the target audience is towards other hardware vendors. At the moment, after designing a new hardware, you'd have to also build the entire software stack already existing on Cuda, ROCm, etc - this can double or triple your hardware bring up time. If you plug into Max (not Mojo), you get most of the software stack for free, so your software bring up time can go from months of engineering hours to probably weeks.

adastra22 27d ago
For AI? GPUs actually kinda suck. NVIDIA’s GPUs are best, and their software stack is great, but they aren’t really adapted to AI (training or inference).
- melodyogonna 26d ago
  I wouldn't say they suck, but they're general-purpose. Anyway, you've gotten into the core of what makes Modular's stack useful. GPUs are great, but it has become obvious that they leave performance on the table, so dedicated AI accelerators get built; however, these don't have software infrastructure and kernels that have been tuned and optimised for years, like CUDA. The bet here is that you can build great hardware, and with a little bit of effort you can unlock all the necessary software for your hardware.
  - adastra22 26d ago
    Even general purpose matrix multipliers built for large linear systems instead of 3D graphics would get you orders of magnitude improvement. That’s essential what the current generation of AI accelerators are. Dedicated hardware could improve that further. I think that counts as “kinda sucks” in the absolute sense. It’s just far faster and more efficient than any other thing you can buy off the shelf today.