11 ms·

NVidia is making boatloads of money because their driver works and they have a software library called CUDA that accelerates neural networks. Nobody expects AM

NVidia is making boatloads of money because their driver works and they have a software library called CUDA that accelerates neural networks. Nobody expects AMD to match them, but George thought they could at least write a GPU driver. If AMD can get that driver out, then George could provide a competitor to CUDA (for neutral networks only). They'd both make boatloads of money. However, AMD was less capable than expected and their drivers were too buggy to run neural networks like those needed for the MLPerf benchmark. So now, it appears that AMD, Tinybox, and investors like me won't be making boatloads of money.

DiabloD3 2y ago
Slight mistake in your description: CUDA is an out of date API that was replaced by Khronos's own official compute APIs. Khronos is a standards consortium that Nvidia is a founding member of. Although the marketing department at Nvidia still pushes for greenfield CUDA codebases, no new code should be written in it, and they should opt for open source international standards only. Khronos APIs are implemented by over 120 vendors.
- greenavocado 2y ago
  Show me the announcement deprecating CUDA
- mshockwave 2y ago
  With my past experience in Khronos, NVIDIA is indeed a member and they sent decent guys to the meetings -- but only for strategic reasons, rather than "advocating for open standards" as you described. My experiences there actually told me the opposite that they will never drop CUDA. Objectively they also have incentive to do so: fighting with 100-ish companies to ratify something is always slower than rolling out an feature in an ecosystem you have total control of.
  - pixelpoet 2y ago
    Not just that, but they get to abuse their near monopoly to strongarm many companies into using Cuda. Can't say more unfortunately...
- redox99 2y ago
  That's outright false and just wishful thinking from you.
- bee_rider 2y ago
  That’s not a mistake, you just have a different preference.
- baq 2y ago
  CUDA is a vendor lock-in mechanism and they’ll happily endorse whatever Khronos says is standard… after they lose dominance. It took 30 years for x86 and just barely so.
- KeplerBoy 2y ago
  Why do I have the feeling that CUDA will outlive whatever Khronos has proposed (do you refer to Sycl)?
  - Foobar8568 2y ago
    How may standards Khronos endorced over the years/decades on compute? From an uneducated and external view, it seems every 2-5years there is a new standard.
    - DiabloD3 2y ago
      Ultimately 2. OpenCL, and Vulkan (via it's compute shader). Sycl's job isn't that, its meant to abstract implementations of common components across different kinds of hardware, and doesn't force you into any particular style of impl. As in, I could write a component for Sycl for my GPU in OpenCL and what Sycl would abstract away from the consumer of my component would be the entire usage of OpenCL itself; but I could write a component for a DSP, and it'd use an entirely closed source SDK for that hardware and is entirely opaque, and a Sycl user could use that impl for that function of they owned that DSP (instead of a CPU-based or GPU-based impl). Also, Vulkan's compute doesn't replace OpenCL (not even in the sense that Vulkan, as a graphics API, replaces OpenGL). They're different levels of abstraction. Most Vulkan games are written almost entirely in compute shaders (ex: the powerhouse that is the Doom 2016 and Doom Eternal engines; and why they perform so fucking amazingly on paltry hardware like the original revision Xbox One, or hell, even the Switch). In addition, I almost consider DX12 a flavor of Vulkan. Same job, written largely by the same people from the same companies, but instead of being OpenGL C-dialect flavored, its D3D C++-dialect flavored, but they both have entirely equivalent APIs that often call the same driver internals and produce nearly identical MIR. Microsoft did this on purpose to reflect the nature of how modern GPUs are almost entirely software renderers, sans certain parts of the texture units.
lofaszvanitt 2y ago
Why is that, that AMD seemingly can't act like a sane individual would do?
- viraptor 2y ago
  There's a chance they (maybe specifically the lawyers) know something we don't. I mean, maybe they are absurdly incompetent in listening to feedback, while at the same time achieving technically great things in hardware. But after so many years and seeing all the AI money going to the competitor... That seems less and less likely every day.
  - orbital-decay 2y ago
    What might that be, for example? "Organizationally unable to make competent software, perfectly able to make great hardware" seems to be the common case with hardware companies, if not de facto standard. Exceptions are rare.
    - viraptor 2y ago
      Apart from "trying to implement this will cost us more in CUDA API copying lawsuits then it could earn", I don't know. But it's not just that they can't make competent software. It's that everyone tells them they should try, that it looks like a pile of money ready to pick up, that people try doing it on their own... and AMD does nothing. They're not even taking the chance to fail/succeed. Can you imagine that Lisa Su doesn't get asked about this at least once a week?
      - nextaccountic 2y ago
        They are already trying to copy CUDA API with rocm and HIP. If this is lawsuit worthy, they may already be hit with a lawsuit at any moment.
  - deleted 2y ago
    [deleted]
  - 2y ago
dralley 2y ago
>However, AMD was less capable than expected and their drivers were too buggy to run neural networks like those needed for the MLPerf benchmark. So now, it appears that AMD, Tinybox, and investors like me won't be making boatloads of money. This is where the melodrama kicks in. They reverse-reversed course less than a week later and now they're back on AMD again. AMD plans go "on hold" - March 19: https://twitter.com/__tinygrad__/status/1770151484363354195 https://twitter.com/__tinygrad__/status/1770151484363354195 AMD plans restarted - March 25: https://twitter.com/__tinygrad__/status/1772139983731831051 https://twitter.com/__tinygrad__/status/1772139983731831051
- erichocean 2y ago
  > This is where the melodrama kicks in. The complaint was that they couldn't debug the GPU. Now they can, so they're soldiering on. Seems reasonable to me.
  - dralley 2y ago
    Surely if they had done a good job communicating with AMD they could have figured that out beforehand. The repo was already public.
    - gunsle 2y ago
      He talks about in his streams how horrible the communication he’s received from AMD is. From what I listened to, it seemed like that was more of why he was giving up initially. Why do a bunch of free work for a massive company that won’t even communicate with you? Especially when he’s doing them a massive favor for next to no investment on their end?
razodactyl 2y ago
It's not just this though. AMD cards are priced better and NVIDIA is overpriced due to the gap. I would love to use AMD for my ML experiments and I would love to see healthy competition benefit this field.

DiabloD3 2y ago

Slight mistake in your description: CUDA is an out of date API that was replaced by Khronos's own official compute APIs. Khronos is a standards consortium that Nvidia is a founding member of. Although the marketing department at Nvidia still pushes for greenfield CUDA codebases, no new code should be written in it, and they should opt for open source international standards only. Khronos APIs are implemented by over 120 vendors.

greenavocado 2y ago
Show me the announcement deprecating CUDA
mshockwave 2y ago
With my past experience in Khronos, NVIDIA is indeed a member and they sent decent guys to the meetings -- but only for strategic reasons, rather than "advocating for open standards" as you described. My experiences there actually told me the opposite that they will never drop CUDA. Objectively they also have incentive to do so: fighting with 100-ish companies to ratify something is always slower than rolling out an feature in an ecosystem you have total control of.
- pixelpoet 2y ago
  Not just that, but they get to abuse their near monopoly to strongarm many companies into using Cuda. Can't say more unfortunately...
redox99 2y ago
That's outright false and just wishful thinking from you.
bee_rider 2y ago
That’s not a mistake, you just have a different preference.
baq 2y ago
CUDA is a vendor lock-in mechanism and they’ll happily endorse whatever Khronos says is standard… after they lose dominance. It took 30 years for x86 and just barely so.
KeplerBoy 2y ago
Why do I have the feeling that CUDA will outlive whatever Khronos has proposed (do you refer to Sycl)?
- Foobar8568 2y ago
  How may standards Khronos endorced over the years/decades on compute? From an uneducated and external view, it seems every 2-5years there is a new standard.
  - DiabloD3 2y ago
    Ultimately 2. OpenCL, and Vulkan (via it's compute shader). Sycl's job isn't that, its meant to abstract implementations of common components across different kinds of hardware, and doesn't force you into any particular style of impl. As in, I could write a component for Sycl for my GPU in OpenCL and what Sycl would abstract away from the consumer of my component would be the entire usage of OpenCL itself; but I could write a component for a DSP, and it'd use an entirely closed source SDK for that hardware and is entirely opaque, and a Sycl user could use that impl for that function of they owned that DSP (instead of a CPU-based or GPU-based impl). Also, Vulkan's compute doesn't replace OpenCL (not even in the sense that Vulkan, as a graphics API, replaces OpenGL). They're different levels of abstraction. Most Vulkan games are written almost entirely in compute shaders (ex: the powerhouse that is the Doom 2016 and Doom Eternal engines; and why they perform so fucking amazingly on paltry hardware like the original revision Xbox One, or hell, even the Switch). In addition, I almost consider DX12 a flavor of Vulkan. Same job, written largely by the same people from the same companies, but instead of being OpenGL C-dialect flavored, its D3D C++-dialect flavored, but they both have entirely equivalent APIs that often call the same driver internals and produce nearly identical MIR. Microsoft did this on purpose to reflect the nature of how modern GPUs are almost entirely software renderers, sans certain parts of the texture units.

lofaszvanitt 2y ago

Why is that, that AMD seemingly can't act like a sane individual would do?

viraptor 2y ago
There's a chance they (maybe specifically the lawyers) know something we don't. I mean, maybe they are absurdly incompetent in listening to feedback, while at the same time achieving technically great things in hardware. But after so many years and seeing all the AI money going to the competitor... That seems less and less likely every day.
- orbital-decay 2y ago
  What might that be, for example? "Organizationally unable to make competent software, perfectly able to make great hardware" seems to be the common case with hardware companies, if not de facto standard. Exceptions are rare.
  - viraptor 2y ago
    Apart from "trying to implement this will cost us more in CUDA API copying lawsuits then it could earn", I don't know. But it's not just that they can't make competent software. It's that everyone tells them they should try, that it looks like a pile of money ready to pick up, that people try doing it on their own... and AMD does nothing. They're not even taking the chance to fail/succeed. Can you imagine that Lisa Su doesn't get asked about this at least once a week?
    - nextaccountic 2y ago
      They are already trying to copy CUDA API with rocm and HIP. If this is lawsuit worthy, they may already be hit with a lawsuit at any moment.
- deleted 2y ago
  [deleted]
- 2y ago

dralley 2y ago

>However, AMD was less capable than expected and their drivers were too buggy to run neural networks like those needed for the MLPerf benchmark. So now, it appears that AMD, Tinybox, and investors like me won't be making boatloads of money. This is where the melodrama kicks in. They reverse-reversed course less than a week later and now they're back on AMD again. AMD plans go "on hold" - March 19: https://twitter.com/__tinygrad__/status/1770151484363354195 https://twitter.com/__tinygrad__/status/1770151484363354195 AMD plans restarted - March 25: https://twitter.com/__tinygrad__/status/1772139983731831051 https://twitter.com/__tinygrad__/status/1772139983731831051

erichocean 2y ago
> This is where the melodrama kicks in. The complaint was that they couldn't debug the GPU. Now they can, so they're soldiering on. Seems reasonable to me.
- dralley 2y ago
  Surely if they had done a good job communicating with AMD they could have figured that out beforehand. The repo was already public.
  - gunsle 2y ago
    He talks about in his streams how horrible the communication he’s received from AMD is. From what I listened to, it seemed like that was more of why he was giving up initially. Why do a bunch of free work for a massive company that won’t even communicate with you? Especially when he’s doing them a massive favor for next to no investment on their end?

razodactyl 2y ago

It's not just this though. AMD cards are priced better and NVIDIA is overpriced due to the gap. I would love to use AMD for my ML experiments and I would love to see healthy competition benefit this field.