5 ms·

AI will take down Nvidia’s moat. When it becomes trivial to translate CUDA/PTX to HIP, SYCL or Metal, CUDA is no longer the moat, it becomes the intermediate re

AI will take down Nvidia’s moat. When it becomes trivial to translate CUDA/PTX to HIP, SYCL or Metal, CUDA is no longer the moat, it becomes the intermediate representation.

Keyframe 4d ago
yeah yeah, "when" an often keyword with AI it seems. As Mr. E. Nigma put it - what always comes but never arrives? Meanwhile the moat deepens and it's build on inertia and laziness and Nvidia knows this really REALLY well.
- swerner 3d ago
  Oh, absolutely. Nvidia is the modern day “nobody gets fired for buying IBM”. The reason we’re still using Unix is not because it’s the best, but because it had to much inertia to let any alternative become its successor. Similarly, C and HTML are maybe the most terrible yet extremely useful languages we have.
mathisfun123 4d ago
i swear people who are outsiders here have only clickbait takes; if you've never had to ship GPU code professionally you should just not comment on these things. the source language has never been the moat. Nvidia sells to hyperscalers. Hyperscalers have armies of kernel authors who have no issue translating shaders by hand (or now with claude). Nvidia's moat is (and will remain for the foreseeable future) the entire stack. you cannot fathom the pain and misery of working on literally any other stack. if you've never debugged a GPU synchronization error or kernel panic due to some GPU firmware bug or fought absolute shit profilers hunting for perf you really have no idea what you're talking about.
- swerner 3d ago
  If that is your standard, I do have an idea what I’m talking about.
  - mathisfun123 3d ago
    Ya? do tell us about your experience that leads you to believe mere translation is the bottleneck in the market...
    - swerner 3d ago
      I’m too old to participate in internet pissing contests.
      - mathisfun123 3d ago
        this isn't a "pissing contest"? you made a speculative claim in a public forum and i'm challenging your authority to make such a claim. a "pissing contest" would be if i had said i've shipped hundreds of thousands of lines of shader code into prod and thus you clearly have no idea what you're talking about because you haven't (which is also true).
        swerner 3d ago
        I could post the GitHub URLs of all the shader code I wrote that’s running on countless GPUs right now, but what would it change? I’m still just a random guy on the internet with an opinion that happens to be different from your opinion. You can simply disagree with me, regardless of my experience (or lack thereof).
        mathisfun123 3d ago
        > You can simply disagree with me that's exactly what i did and made an argument for why i think you're wrong. in response you provided exactly zero substantive remarks other than "i've written shaders" and then accused me of pissing. also FYI it's clear from your profile that you've only worked on graphics (embree, blender, etc) and not compute. so i'll repeat: you're an outsider and you have absolutely no idea what you're talking about.
        3d ago
- swerner 3d ago
  Most modern graphics is compute. Pixar, Dreamworks, Sony, etc do not use Vulkan to render their movies. It’s CPUs or CUDA. “graphics people do not (and cannot) write CUDA kernels/shaders” is just not true at all. All it would take to verify that would be things like reading the introduction of the OptiX documentation, a small sample of SIGGRAPH GPU papers or the Blender/Cycles source code.
bayindirh 4d ago
> When it becomes trivial to translate CUDA/PTX to HIP,... ZLUDA is already doing that, no?
- swerner 3d ago
  I don't think we're at a point yet where anyone would trust ZLUDA enough to ship commercial products that rely on it. I would be delighted though, if anyone can prove me wrong.
  - bayindirh 3d ago
    No, but we can go there. This is an open source project. Anyone can put some more effort behind it and push it further. It's improving, AFAICS. Src: https://github.com/vosen/ZLUDA https://github.com/vosen/ZLUDA
    - swerner 3d ago
      Unfortunately, many of very good ideas end with “it’s open source, anyone can contribute” because very few actually do.
larodi 3d ago
trivial to translate (or transpile) - okay. trivial to understand the result - not so much. trivial to then evolve it - hm... perhaps a different story. still, it seems very likely now, that such "quick rewrites" are viable, not sure if an open approach to them is viable. a newly born open project that was LLM-derived, and not by a credible author, which spans hundreds of files no human eye has ever looked at, can only work for a closed organization, but will never be trusted by the general audience... just like that.

AI will take down Nvidia’s moat. When it becomes trivial to translate CUDA/PTX to HIP, SYCL or Metal, CUDA is no longer the moat, it becomes the intermediate representation.

Keyframe 4d ago

yeah yeah, "when" an often keyword with AI it seems. As Mr. E. Nigma put it - what always comes but never arrives? Meanwhile the moat deepens and it's build on inertia and laziness and Nvidia knows this really REALLY well.

swerner 3d ago
Oh, absolutely. Nvidia is the modern day “nobody gets fired for buying IBM”. The reason we’re still using Unix is not because it’s the best, but because it had to much inertia to let any alternative become its successor. Similarly, C and HTML are maybe the most terrible yet extremely useful languages we have.

mathisfun123 4d ago

i swear people who are outsiders here have only clickbait takes; if you've never had to ship GPU code professionally you should just not comment on these things. the source language has never been the moat. Nvidia sells to hyperscalers. Hyperscalers have armies of kernel authors who have no issue translating shaders by hand (or now with claude). Nvidia's moat is (and will remain for the foreseeable future) the entire stack. you cannot fathom the pain and misery of working on literally any other stack. if you've never debugged a GPU synchronization error or kernel panic due to some GPU firmware bug or fought absolute shit profilers hunting for perf you really have no idea what you're talking about.

swerner 3d ago
If that is your standard, I do have an idea what I’m talking about.
- mathisfun123 3d ago
  Ya? do tell us about your experience that leads you to believe mere translation is the bottleneck in the market...
  - swerner 3d ago
    I’m too old to participate in internet pissing contests.
    - mathisfun123 3d ago
      this isn't a "pissing contest"? you made a speculative claim in a public forum and i'm challenging your authority to make such a claim. a "pissing contest" would be if i had said i've shipped hundreds of thousands of lines of shader code into prod and thus you clearly have no idea what you're talking about because you haven't (which is also true).
      - swerner 3d ago
        I could post the GitHub URLs of all the shader code I wrote that’s running on countless GPUs right now, but what would it change? I’m still just a random guy on the internet with an opinion that happens to be different from your opinion. You can simply disagree with me, regardless of my experience (or lack thereof).
        mathisfun123 3d ago
        > You can simply disagree with me that's exactly what i did and made an argument for why i think you're wrong. in response you provided exactly zero substantive remarks other than "i've written shaders" and then accused me of pissing. also FYI it's clear from your profile that you've only worked on graphics (embree, blender, etc) and not compute. so i'll repeat: you're an outsider and you have absolutely no idea what you're talking about.
        3d ago
swerner 3d ago
Most modern graphics is compute. Pixar, Dreamworks, Sony, etc do not use Vulkan to render their movies. It’s CPUs or CUDA. “graphics people do not (and cannot) write CUDA kernels/shaders” is just not true at all. All it would take to verify that would be things like reading the introduction of the OptiX documentation, a small sample of SIGGRAPH GPU papers or the Blender/Cycles source code.

bayindirh 4d ago

> When it becomes trivial to translate CUDA/PTX to HIP,... ZLUDA is already doing that, no?

swerner 3d ago
I don't think we're at a point yet where anyone would trust ZLUDA enough to ship commercial products that rely on it. I would be delighted though, if anyone can prove me wrong.
- bayindirh 3d ago
  No, but we can go there. This is an open source project. Anyone can put some more effort behind it and push it further. It's improving, AFAICS. Src: https://github.com/vosen/ZLUDA https://github.com/vosen/ZLUDA
  - swerner 3d ago
    Unfortunately, many of very good ideas end with “it’s open source, anyone can contribute” because very few actually do.

larodi 3d ago

trivial to translate (or transpile) - okay. trivial to understand the result - not so much. trivial to then evolve it - hm... perhaps a different story. still, it seems very likely now, that such "quick rewrites" are viable, not sure if an open approach to them is viable. a newly born open project that was LLM-derived, and not by a credible author, which spans hundreds of files no human eye has ever looked at, can only work for a closed organization, but will never be trusted by the general audience... just like that.