5 ms·

You do know that Microsoft, Oracle, Meta are all in on this right? Heck I think it is being used to run ChatGPT 3.5 and 4 services.

• 0cf8612b2e1e 2y ago
  On the other hand, AMD has had a decade of watching CUDA eat their lunch and done basically nothing to change the situation.
  • bee_rider 2y ago
    AMD tries to compete in hardware with Intel’s CPUs and Nvidia’s GPUs. They have to slack somewhere, and software seems to be where. It isn’t any surprise that they can’t keep up on every front, but it does mean they can freely bring in partners whose core competency is software and work with them without any caveats. Not sure why they haven’t managed to execute on that yet, but the partners must be pretty motivated now, right? I’m sure they don’t love doing business at Nvidia’s leisure.
    • bobsondugnut 2y ago
      when was the last time AMD hardware was keeping up with NVIDIA? 2014?
      • 0cf8612b2e1e 2y ago
        Been a while since AMD had the top tier offering, but it has been trading blows in the middle tier segment the entire time. If you are just looking for a gamer card (ie not max AI performance), the AMD is typically cheaper and less power hungry than the equivalent Nvidia.
        • bobsondugnut 2y ago
          > the AMD is typically cheaper and less power hungry than the equivalent Nvidia cheaper is true, but less power hungry is absolutely not true, which is kind of my point.
          • dralley 2y ago
            It was true with RDNA 2. RDNA 3 regressed on this a bit, supposedly there was a hardware hiccup that prevented them from hitting frequency and voltage targets that they were hoping to reach. In any case they're only slightly behind, not crazy far behind like Intel is.
        • aurareturn 2y ago
          It’s trading blows because AMD sells their cards at lower margins in the midrange and Nvidia lets them.
          • bee_rider 2y ago
            But, the fact that Nvidia cards command higher margins also reflects their better software stack, right? Nvidia “lets them” trade blows in the midrange, or, equivalently, Nvidia is receiving the reward of their software investments: even their midrange hardware commands a premium.
      • bee_rider 2y ago
        The MI300X sounds like it is competitive, haha
        • bobsondugnut 2y ago
          competitive with H100 for inference. a 2 year old product on just one half of the ML story. H200 (and potentially B100) is the appropriate comparison based on their production in volume.
    • pjmlp 2y ago
      Hardware is useless without software to make it show off.
• softfalcon 2y ago
  I feel like people forget that AMD has huge contracts with Microsoft, Valve, Sony, etc to design consoles at scale. It's an invisible provider as most folks don't even realize their Xbox and their Playstation are both AMD. When you're providing fab designs at that scale, it makes a lot more sense to folks that companies would be willing to try a more affordable option to nVidia hardware. My bet is that AMD figures out a service-able solution for some (not all) workloads that isn't ground breaking, but affordable to the clients that want an alternative. That's usually how this goes for AMD in my experience.
  • sangnoir 2y ago
    If you read/listen to the Stratechary interview wirh Lisa Hsu, she spelled out being open ro customizing AMD hardware to meet partner's needs. So if Microsoft needs more memory bandwidth and less compute, AMD will build something just for them based on what they have now. If Meta wants 10% less power consumption (and cooling) for a 5% hit in compute, AMD will hear them out too. We'll see if that hardware customization strategy works outside of consoles.
    • rcxdude 2y ago
      It certainly helps differentiate from NVIDIA's "Don't even think about putting our chips on a PCB we haven't vetted" approach.
    • pjmlp 2y ago
      Yeah, but they will be using internal Microsoft and Meta software stacks, nothing that will dent CUDA.
  • Rinzler89 2y ago
    >I feel like people forget that AMD has huge contracts with Microsoft, Valve, Sony, etc to design consoles at scale. Nobody forget that, just that those console chips are super low margins, which is why Intel and Nvidia stopped catering to that market after the Xbox/PS3 generations and only AMD took it up because they were broke and every penny mattered to them. Nvidia did a brief stint with the Shield/Switch because they were trying to get into the Android/ARM space and also kinda gave up due to the margins.
• adabyron 2y ago
  I have read in a few places that Microsoft is using AMD for inference to run ChatGPT. If I recall they said the price/performance was better. I'm curious if that's just because they can't get enough Nvidia GPUs or if the price/performance is actually that much better.
  • atq2119 2y ago
    Most likely it really is better overall. Think of it this way: AMD is pretty good at hardware, so there's no reason to think that the raw difference in terms of flops is significant in either direction. It may go in AMD's favor sometimes and Nvidia's other times. What AMD traditionally couldn't do was software, so those AMD GPUs are sold at a discount (compared to Nvidia), giving you better price/performance if you can use them. Surely Microsoft is operating GPUs at large enough scale that they can pay a few people to paper over the software deficiencies so that they can use the AMD GPUs and still end up ahead in terms of overall price/performance.