6 ms·

Note: this is old work, and much of the team working on TVM, and MLC were from OctoAI and we have all recently joined NVIDIA.

sebmellen 2y ago
Is there no hope for AMD anymore? After George Hotz/Tinygrad gave up on AMD I feel there’s no realistic chance of using their chips to break the CUDA dominance.
- llm_trw 2y ago
  Not really. AMD is constitutionally incapable of shipping anything but mid range hardware that requires no innovation. The only reason why they are doing so well in CPUs right now is that Intel has basically destroyed itself without any outside help.
  - perching_aix 2y ago
    And I'm supposed to believe that HN is this amazing platform for technology and science discussions, totally unlike its peers...
    - zamadatix 2y ago
      The above take is worded a bit cynical but is their general approach to GPUs lately across the board e.g. https://www.techpowerup.com/326415/amd-confirms-retreat-from-the-enthusiast-gpu-segment-to-focus-on-gaining-market-share https://www.techpowerup.com/326415/amd-confirms-retreat-from... Also I'd take HN as being being an amazing platform for the overall consistency and quality of moderation. Anything beyond that depends more on who you're talking to than where at.
    - petesergeant 2y ago
      Maybe be the change you want to see and tell us what the real story is?
      - perching_aix 2y ago
        We seem to disagree on what the change in the world I'd like to see is like, which is a real shocker I'm sure. Personally, I think that's when somebody who has no real information to contribute doesn't try to pretend that they do. So thanks for the offer, but I think I'm already delivering on that realm.
        shadowgovt 2y ago
        Oh, there's basically no chance of getting that on the Internet. The Internet is a machine that highly simplifies the otherwise complex technical challenge of wide-casting ignorance. It wide-casts wisdom too, but it's an exercise for the reader to distinguish them.
    - llm_trw 2y ago
      I don't really care what you believe. Everyone whose dug deep into what AMD is doing has left in disgust if they are lucky and bankruptcy if they are not. If I can save someone else from wasting $100,000 on hardware and six months of their life then my post has done more good than the AMD marketing department ever will.
      - AnthonyMouse 2y ago
        > If I can save someone else from wasting $100,000 on hardware and six months of their life then my post has done more good than the AMD marketing department ever will. This seems like unuseful advice if you've already given up on them. You tried it and at some point in the past it wasn't ready. But by not being ready they're losing money, so they have a direct incentive to fix it. Which would take a certain amount of time, but once you've given up you no longer know if they've done it yet or not, at which point your advice would be stale. Meanwhile the people who attempt it apparently seem to get acquired by Nvidia, for some strange reason. Which implies it should be a worthwhile thing to do. If they've fixed it by now which you wouldn't know if you've stopped looking, or they fix it in the near future, you have a competitive advantage because you have access to lower cost GPUs than your rivals. If not, but you've demonstrated a serious attempt to fix it for everyone yourself, Nvidia comes to you with a sack full of money to make sure you don't finish, and then you get a sack full of money. That's win/win, so rather than nobody doing it, it seems like everybody should be doing it.
        llm_trw 2y ago
        I've tried it three times. I've seen people try it every six months for two decades now. At some point you just have to accept that AMD is not a serious company, but is a second rate copycat and there is no way to change that without firing everyone from middle management up. I'm deeply worried about stagnation in the CPU space now that they are top dog and Intel is dead in the water. Here's hoping China and Risk V save us. >Meanwhile the people who attempt it apparently seem to get acquired by Nvidia Everyone I've seen base jumping has gotten a sponsorship from redbull, ergo. everyone should basejump. Ignore the red smears around the parking lot.
  - ksec 2y ago
    Everything is comparative. AMD isn't perfect. As an Ex Shareholder I have argued they did well partly because of Intel's downfall. In terms of execution it is far from perfect. But Nvidia is a different beast. It is a bit like Apple in the late 00s where you take business, forecast, marketing, operation, software, hardware, sales etc You take any part of it and they are all industry leading. And having industry leading capability is only part of the game, having it all work together is completely another thing. And unlike Apple where they lost direction once Steve Jobs passed away and weren't sure about how to deploy capital. Jensen is still here, and they have more resources now making Nvidia even more competitive. It is often most people underestimate the magnitude of the task required, ( I like to tell the story again about an Intel GPU engineer in 2016 arguing they could take dGPU market shares by 2020, and we are now 2025 ), over estimate the capability of an organisation, under estimate the rival's speed of innovation and execution. These three thing combined is why most people are often off the estimate by an order of magnitude.
    - llm_trw 2y ago
      Yeah, no. We are in the middle of a monopoly squeeze by NVidia on the most innovative part of the economy right now. I expect the DOJ to hit them harder than they did MS in the 90s given the bullshit they are pulling and the drag on the economy they are causing. By comparison if AMD could write a driver that didn't shit itself when it had to multiply more than two matrices in a row they'd be selling cards faster than they can make them. You don't need to sell the best shovels in a gold rush to make mountains of money, but you can't sell teaspoons as premium shovels and expect people to come back.
      - shiroiushi 2y ago
        >I expect the DOJ to hit them harder than they did MS in the 90s given the bullshit they are pulling and the drag on the economy they are causing. It sounds like you're expecting extreme competence from the DOJ. Given their history with regulating big tech companies, and even worse, the incoming administration, I think this is a very unrealistic expectation.
      - kadoban 2y ago
  - lofaszvanitt 2y ago
    It had to destroy itself. These companies do not act on their own...
  - adrian_b 2y ago
    In CPUs, AMD has made many innovations that have been copied by Intel only after many years and this delay had an important contribution to Intel's downfall. The most important has been the fact that AMD has predicted correctly that big monolithic CPUs will no longer be feasible in the future CMOS fabrication technologies, so they have designed the Zen family since the beginning with a chiplet-based architecture. Intel had attempted to ridicule them, but after losing many billions they have been forced to copy this strategy. Also in the microarchitecture of their CPUs AMD has made the right choices since the beginning and then they have improved it constantly with each generation. The result is that now the latest Intel big core, Lion Cove, has a microarchitecture that is much more similar to AMD Zen 5 than to any of the previous Intel cores, because they had to do this to get a competitive core. In the distant past, AMD has also introduced a lot of innovations long before they were copied by Intel, but it is true that those had not been invented by AMD, but they had been copied by AMD from more expensive CPUs, like DEC Alpha or Cray or IBM POWER, but Intel has also copied them only after being forced by the competition with AMD.
- latchkey 2y ago
  https://x.com/dylan522p/status/1871287937268383867 https://x.com/dylan522p/status/1871287937268383867
  - krackers 2y ago
    That's almost word for word what geohotz said last year?
    - refulgentis 2y ago
      What part? I assume the part where she said there's "gaps in the software stack", because that's the only part that's attributed to her. But I must be wrong because that hasn't been in dispute or in the news in a decade, it's not a geohot discovery from last year. Hell I remember a subargument of a subargument re: this being an issue a decade ago in macOS dev (TL;Dr whether to invest in opencl)
    - arresin 2y ago
      I went through the thread. There’s an argument to be made in firing Su for being so spaced out as to miss an op for their own CUDA for free.
      - hedgehog 2y ago
        Not remotely, how did you get to that idea?
        refulgentis 2y ago
        Kids this days (shakes fist) tl;dr there's a non-unsubstantial # of people who learn a lot from geohot. I'd say about 3% of people here will be confused if you thought of him as less than a top technical expert across many comp sci fields. And he did the geohot thing recently, way tl;dr: acted like there was a scandal being covered up by AMD around drivers that was causing them to "lose" to nVidia. He then framed AMD not engaging with him on this topic as further covering-up and choosing to lose. So if you're of a certain set of experiences, you see an anodyne quote from the CEO that would have been utterly unsurprising dating back to when ATI was still a company, and you'd read it as the CEO breezily admitting in public that geohot was right about how there was malfeasance, followed by a cover up, implying extreme dereliction of duty, because she either helped or didn't realize till now. I'd argue this is partially due to stonk-ification of discussions, there was a vague, yet often communicated, sense there was something illegal happening. Idea was it was financial dereliction of duty to shareholders.
- dismalaf 2y ago
  IMO the hope shouldn't be that AMD specifically wins, rather it's best for consumers that hardware becomes commoditized and prices come down. And that's what's happening, slowly anyway. Google, Apple and Amazon all have their own AI chips, Intel has Gaudi, AMD had their thing, and the software is at least working on more than just Nvidia. Which is a win. Even if it's not perfect. I'm personally hoping that everyone piles in on a standard like SYCL.
- comex 2y ago
  Maybe from Modular (the company Chris Lattner is working for). In this recent announcement they said they had achieved competitive ML performance… on NVIDIA GPUs, but with their own custom stack completely replacing CUDA. And they’re targeting AMD next. https://www.modular.com/blog/introducing-max-24-6-a-gpu-native-generative-ai-platform https://www.modular.com/blog/introducing-max-24-6-a-gpu-nati...
  - behnamoh 2y ago
    Ah yes, the programming language (Mojo) that requires an account before I can use it...
    - melodyogonna 2y ago
      Mojo no longer requires an account to install. But that is irrelevant to the conversation because this is not about Mojo but something they call MAX. [1] 1. https://www.modular.com/max https://www.modular.com/max
- quotemstr 2y ago
  The world is bigger than AMD and Nvidia. Plenty of interesting new AI-tuned non-GPU accelerators coming online.
  - grigio 2y ago
    I hope, name some NPU who can run a 70B model..
- steeve 2y ago
  We (ZML) have AMD MI300X working just fine, in fact, faster than H100
- fweimer 2y ago
  Isn't AMD rather strong in the HPC space? Quite frankly, I have difficulty reconciling a lot of comments here with that, and my own experience as an AMD GPU user (although not for compute, and not on Windows).

sebmellen 2y ago

Is there no hope for AMD anymore? After George Hotz/Tinygrad gave up on AMD I feel there’s no realistic chance of using their chips to break the CUDA dominance.

llm_trw 2y ago
Not really. AMD is constitutionally incapable of shipping anything but mid range hardware that requires no innovation. The only reason why they are doing so well in CPUs right now is that Intel has basically destroyed itself without any outside help.
- perching_aix 2y ago
  And I'm supposed to believe that HN is this amazing platform for technology and science discussions, totally unlike its peers...
  - zamadatix 2y ago
    The above take is worded a bit cynical but is their general approach to GPUs lately across the board e.g. https://www.techpowerup.com/326415/amd-confirms-retreat-from-the-enthusiast-gpu-segment-to-focus-on-gaining-market-share https://www.techpowerup.com/326415/amd-confirms-retreat-from... Also I'd take HN as being being an amazing platform for the overall consistency and quality of moderation. Anything beyond that depends more on who you're talking to than where at.
  - petesergeant 2y ago
    Maybe be the change you want to see and tell us what the real story is?
    - perching_aix 2y ago
      We seem to disagree on what the change in the world I'd like to see is like, which is a real shocker I'm sure. Personally, I think that's when somebody who has no real information to contribute doesn't try to pretend that they do. So thanks for the offer, but I think I'm already delivering on that realm.
      - shadowgovt 2y ago
        Oh, there's basically no chance of getting that on the Internet. The Internet is a machine that highly simplifies the otherwise complex technical challenge of wide-casting ignorance. It wide-casts wisdom too, but it's an exercise for the reader to distinguish them.
  - llm_trw 2y ago
    I don't really care what you believe. Everyone whose dug deep into what AMD is doing has left in disgust if they are lucky and bankruptcy if they are not. If I can save someone else from wasting $100,000 on hardware and six months of their life then my post has done more good than the AMD marketing department ever will.
    - AnthonyMouse 2y ago
      > If I can save someone else from wasting $100,000 on hardware and six months of their life then my post has done more good than the AMD marketing department ever will. This seems like unuseful advice if you've already given up on them. You tried it and at some point in the past it wasn't ready. But by not being ready they're losing money, so they have a direct incentive to fix it. Which would take a certain amount of time, but once you've given up you no longer know if they've done it yet or not, at which point your advice would be stale. Meanwhile the people who attempt it apparently seem to get acquired by Nvidia, for some strange reason. Which implies it should be a worthwhile thing to do. If they've fixed it by now which you wouldn't know if you've stopped looking, or they fix it in the near future, you have a competitive advantage because you have access to lower cost GPUs than your rivals. If not, but you've demonstrated a serious attempt to fix it for everyone yourself, Nvidia comes to you with a sack full of money to make sure you don't finish, and then you get a sack full of money. That's win/win, so rather than nobody doing it, it seems like everybody should be doing it.
      - llm_trw 2y ago
        I've tried it three times. I've seen people try it every six months for two decades now. At some point you just have to accept that AMD is not a serious company, but is a second rate copycat and there is no way to change that without firing everyone from middle management up. I'm deeply worried about stagnation in the CPU space now that they are top dog and Intel is dead in the water. Here's hoping China and Risk V save us. >Meanwhile the people who attempt it apparently seem to get acquired by Nvidia Everyone I've seen base jumping has gotten a sponsorship from redbull, ergo. everyone should basejump. Ignore the red smears around the parking lot.
- ksec 2y ago
  Everything is comparative. AMD isn't perfect. As an Ex Shareholder I have argued they did well partly because of Intel's downfall. In terms of execution it is far from perfect. But Nvidia is a different beast. It is a bit like Apple in the late 00s where you take business, forecast, marketing, operation, software, hardware, sales etc You take any part of it and they are all industry leading. And having industry leading capability is only part of the game, having it all work together is completely another thing. And unlike Apple where they lost direction once Steve Jobs passed away and weren't sure about how to deploy capital. Jensen is still here, and they have more resources now making Nvidia even more competitive. It is often most people underestimate the magnitude of the task required, ( I like to tell the story again about an Intel GPU engineer in 2016 arguing they could take dGPU market shares by 2020, and we are now 2025 ), over estimate the capability of an organisation, under estimate the rival's speed of innovation and execution. These three thing combined is why most people are often off the estimate by an order of magnitude.
  - llm_trw 2y ago
    Yeah, no. We are in the middle of a monopoly squeeze by NVidia on the most innovative part of the economy right now. I expect the DOJ to hit them harder than they did MS in the 90s given the bullshit they are pulling and the drag on the economy they are causing. By comparison if AMD could write a driver that didn't shit itself when it had to multiply more than two matrices in a row they'd be selling cards faster than they can make them. You don't need to sell the best shovels in a gold rush to make mountains of money, but you can't sell teaspoons as premium shovels and expect people to come back.
    - shiroiushi 2y ago
      >I expect the DOJ to hit them harder than they did MS in the 90s given the bullshit they are pulling and the drag on the economy they are causing. It sounds like you're expecting extreme competence from the DOJ. Given their history with regulating big tech companies, and even worse, the incoming administration, I think this is a very unrealistic expectation.
    - kadoban 2y ago
- lofaszvanitt 2y ago
  It had to destroy itself. These companies do not act on their own...
- adrian_b 2y ago
  In CPUs, AMD has made many innovations that have been copied by Intel only after many years and this delay had an important contribution to Intel's downfall. The most important has been the fact that AMD has predicted correctly that big monolithic CPUs will no longer be feasible in the future CMOS fabrication technologies, so they have designed the Zen family since the beginning with a chiplet-based architecture. Intel had attempted to ridicule them, but after losing many billions they have been forced to copy this strategy. Also in the microarchitecture of their CPUs AMD has made the right choices since the beginning and then they have improved it constantly with each generation. The result is that now the latest Intel big core, Lion Cove, has a microarchitecture that is much more similar to AMD Zen 5 than to any of the previous Intel cores, because they had to do this to get a competitive core. In the distant past, AMD has also introduced a lot of innovations long before they were copied by Intel, but it is true that those had not been invented by AMD, but they had been copied by AMD from more expensive CPUs, like DEC Alpha or Cray or IBM POWER, but Intel has also copied them only after being forced by the competition with AMD.
latchkey 2y ago
https://x.com/dylan522p/status/1871287937268383867 https://x.com/dylan522p/status/1871287937268383867
- krackers 2y ago
  That's almost word for word what geohotz said last year?
  - refulgentis 2y ago
    What part? I assume the part where she said there's "gaps in the software stack", because that's the only part that's attributed to her. But I must be wrong because that hasn't been in dispute or in the news in a decade, it's not a geohot discovery from last year. Hell I remember a subargument of a subargument re: this being an issue a decade ago in macOS dev (TL;Dr whether to invest in opencl)
  - arresin 2y ago
    I went through the thread. There’s an argument to be made in firing Su for being so spaced out as to miss an op for their own CUDA for free.
    - hedgehog 2y ago
      Not remotely, how did you get to that idea?
      - refulgentis 2y ago
        Kids this days (shakes fist) tl;dr there's a non-unsubstantial # of people who learn a lot from geohot. I'd say about 3% of people here will be confused if you thought of him as less than a top technical expert across many comp sci fields. And he did the geohot thing recently, way tl;dr: acted like there was a scandal being covered up by AMD around drivers that was causing them to "lose" to nVidia. He then framed AMD not engaging with him on this topic as further covering-up and choosing to lose. So if you're of a certain set of experiences, you see an anodyne quote from the CEO that would have been utterly unsurprising dating back to when ATI was still a company, and you'd read it as the CEO breezily admitting in public that geohot was right about how there was malfeasance, followed by a cover up, implying extreme dereliction of duty, because she either helped or didn't realize till now. I'd argue this is partially due to stonk-ification of discussions, there was a vague, yet often communicated, sense there was something illegal happening. Idea was it was financial dereliction of duty to shareholders.
dismalaf 2y ago
IMO the hope shouldn't be that AMD specifically wins, rather it's best for consumers that hardware becomes commoditized and prices come down. And that's what's happening, slowly anyway. Google, Apple and Amazon all have their own AI chips, Intel has Gaudi, AMD had their thing, and the software is at least working on more than just Nvidia. Which is a win. Even if it's not perfect. I'm personally hoping that everyone piles in on a standard like SYCL.
comex 2y ago
Maybe from Modular (the company Chris Lattner is working for). In this recent announcement they said they had achieved competitive ML performance… on NVIDIA GPUs, but with their own custom stack completely replacing CUDA. And they’re targeting AMD next. https://www.modular.com/blog/introducing-max-24-6-a-gpu-native-generative-ai-platform https://www.modular.com/blog/introducing-max-24-6-a-gpu-nati...
- behnamoh 2y ago
  Ah yes, the programming language (Mojo) that requires an account before I can use it...
  - melodyogonna 2y ago
    Mojo no longer requires an account to install. But that is irrelevant to the conversation because this is not about Mojo but something they call MAX. [1] 1. https://www.modular.com/max https://www.modular.com/max
quotemstr 2y ago
The world is bigger than AMD and Nvidia. Plenty of interesting new AI-tuned non-GPU accelerators coming online.
- grigio 2y ago
  I hope, name some NPU who can run a 70B model..
steeve 2y ago
We (ZML) have AMD MI300X working just fine, in fact, faster than H100
fweimer 2y ago
Isn't AMD rather strong in the HPC space? Quite frankly, I have difficulty reconciling a lot of comments here with that, and my own experience as an AMD GPU user (although not for compute, and not on Windows).