5 ms·
AMD Instinct MI325X in Q4 2024, 288GB of HBM3E
- jauntywundrkind 2y agoSuch a weird & cruel modernity, where these releases are purely in the abstract. No, you still won't be able to buy a MI300X in Q4 2024. The enhanced edition will absolutely not be available. (I miss the old PC era where the world at large was benefiting in tandem from new things happening (or falling behind from not adapting)).
- forrestthewoods 2y ago> No, you still won't be able to buy a MI300X in Q4 2024 Why not? Because they’re sold out to hyperscalers?
- latchkey 2y agoThey are not sold out. It is just a lot more work to support retail on a novel new product, so they are focused on hyperscalers and CSP's. Don't forget that high end GPUs are US export controlled as well. They are considered weapons by the government. [See 88 Fed. Reg. 73458 (Oct. 25, 2023) and the Export administration Regulations (EAR)].
- almostgotcaught 2y ago> No, you still won't be able to buy a MI300X in Q4 2024. they're 15k - who exactly is disappointed they won't be able to buy one?
- krasin 2y agoPeople on reddit.com/r/localllama build even more expensive rigs sometimes. Everyone wants to run llama3 70B and eventually the 405B version.
- ai_what 2y agoI mean it's the price of 2 * Apple Mac Pro's. It isn't exactly unheard of. Just a few weeks ago I spoke to someone who shelled out $10k for running LLM's locally. I've seen more expensive builds as well.
- latchkey 2y agoAs someone who's buying them, that is not the price. Nor can you just buy one at a time. They are on a OAM/UBB, which is 8.
- ein0p 2y agoI’d buy a few at that price. But I can’t.
- bayindirh 2y ago> who exactly is disappointed they won't be able to buy one? HPC centers and research clusters.
- re-thc 2y ago> (I miss the old PC era where the world at large was benefiting in tandem from new things happening (or falling behind from not adapting)) Paper launches aren't anything new. It's always been a thing especially in hardware.
- wmf 2y agoWhat is kind of new is that hyperscalers sometimes get chips 1-2 years before the rest. If the new chip is 3x-4x faster that looks pretty unfair.
- re-thc 2y ago> What is kind of new is that hyperscalers sometimes get chips 1-2 years before the rest. That's not new. Replace hyperscalers with crypto miners and we've had a similar situation for a while.
- latchkey 2y agoCrypto miners no longer buy up large quantities of GPUs. That market died off when ETH switched from PoW to PoS.
- DEADMINCE 2y ago> Crypto miners no longer buy up large quantities of GPUs. That's news to crypto miners.
- latchkey 2y agoI no longer have the 150,000 GPUs I was mining with. The only ones left standing are the small ones and most of them are just using what they already purchased. There is no liquidity in the shitcoins to be able to support daily large dumps of their tokens and unless you have almost free power, there is zero profitability.
- 2y ago
- mpreda 2y agoI think that's where short-sighted financial gain leads AMD to. Where's the money? -- datacenter. So let's focus the good stuff on datacenter exclusivelly. What about "the rest" (gamers, hobbist, students)? There's no money there, let's give theme crap RDNA that we make sure can't be used for any real work; just pretent we're catering for their needs. I think their "consumer GPU" did so bad recently that AMD could just as well, you know, simply liquidate the "consumer GPU" division and stop pretending. I'm in the "consumer GPU" market myself; what AMD GPU do I buy today? -- Radeon Pro VII, launched in 2020 and the best AMD consumer GPU I can find today. It's such a divide. I could optimize my software for such powerful GPUs as the Mi300 line.. but why do that, given that probably I won't even see one such GPU in my lifetime.
- jacobgorm 2y agoI’ve filed a detailed AMD Windows driver crash bug months ago that is getting totally ignored because all their devs have been moved to working on AI.
- atq2119 2y agoThe RX 7900s are pretty good. You get 24GB of RAM in a consumer GPU. If you're interested in GenAI that's a good offering for your "gamers, students, hobbyists" category. And they announced a workstation version with 48GB: https://www.phoronix.com/news/AMD-Radeon-PRO-W7900-Dual-Slot https://www.phoronix.com/news/AMD-Radeon-PRO-W7900-Dual-Slot
- 3abiton 2y agoIndeed for me that's the only standout from this release, 48gb not too overpriced.
- deleted 2y ago[deleted]
- latchkey 2y agoThe focus is on hyperscalers and cloud service providers now. Even Groq stopped selling to retail.
- karma_pharmer 2y agoWelcome (back) to the age of the mainframe. Except we call it "cloudframe" now.
- shrubble 2y agoThe claim that the next generation would be 35x faster, felt like an "Osborne moment" to me, but if demand is robust enough...
- wmf 2y agoNvidia is doing the same thing. They announced B100 before H200 shipped and a few hours ago they started talking about R100 before B100 shipped.
- ipsum2 2y ago(Re: Osborne effect) It's going to be released in 2 years. Rarely can businesses wait that long, they're going to be ordering the MI300 now.
- latchkey 2y ago[deleted] See below, I did not understand the Osborne effect comment.
- ipsum2 2y agoYou're going to wait for the MI350 and not order any more MI300s?
- latchkey 2y agoWeird that I got downvoted on the above. I'm buying and deploying MI300x's today and will buy whatever AMD comes out with next.
- ipsum2 2y agoYou were probably downvoted because you were shilling your company, and you misunderstood the comment. "The Osborne effect is a social phenomenon of customers canceling or deferring orders for the current, soon-to-be-obsolete product as an unexpected drawback of a company's announcing a future product prematurely. It is an example of cannibalization."
- Havoc 2y ago> 35x increase in AI inference performance compared to AMD Instinct MI300 Series Even for marketing claims that’s pretty wild. Still lots of trajectory left in just scale up plan it seems
- layoric 2y agoI think there is a close limit considering most of these gains are coming from the reduced memory bandwidth consumption that comes with the smaller data types. This would line up with Nvidia’s crazy graph from yesterday where data types were specified. How much lower can these go though? 2bit? 1.58bit? 1bit? It seems that these massive gains have a very hard stop to gains that AMD and Nvidia will use to raise their stock price before it all comes to a sudden end.
- nabla9 2y agoAMD comparison: 8x AMD MI300X (192GB, 750W) GPU 8x H100 (80GB, 700W) GPU What would be the result against 8x H100 NVL (188GB, <800W) GPU ?
- DrNosferatu 2y agoIs the software stack working (for practical use)? AMD still has to prove themselves in this.
- deleted 2y ago[deleted]