7 ms·
AMD's Next GPU Is a 3D-Integrated Superchip
- 29athrowaway 3y agoIf not CUDA, then what can you use with those?
- DeathArrow 3y agoGames? Rendering? Transcoding video?
- sylware 3y agoVideo is getting a lot of direct ASIC blocks (look at the latest VPE block in AMD GPUs). I guess those chips are for the movies/video industry, online or not. Because for the "consumer", the CPU is already very efficient, and I don't think we would save interesting about of battery usage in a "real usage" perspective. I may be very wrong, but I don't watch hours and hours in a row of ultra high quality videos on a small screen, that off the AC plug, the battery is unusable in a matter of a few years anyway... it not less.
- imtringued 3y agoEncoding video in real time is expensive. You can run your blog on a 5€/month machine. You'll most likely need ten times that money for a single real time stream using nothing but software decoding, rendering and encoding. If you think, "hey I'm going to use a GPU for this" then congratulations, you've increased your cloud bill by a factor of 20.
- sylware 3y agoLastest AMD GPU does realtime encoding with AV1. It has directly an ASIC block for that. Just mmap the video engine command circular buffer, get some dma buffers, get a interrupt circular buffer, and you are good to go (I think the dma-buffer sync framwork between drivers and user space is still a WIP, and I dunno how the interrupt circular buffer is handled).
- anotherhue 3y agoCUDA is at the API level (ish, I know). There's plenty of room for new silicon that can expose different APIs. Compare to switching from x86 to ARM.
- ssijak 3y agowhy no CUDA rosetta?
- JonChesterfield 3y agoNvidia could build that but doesn't want it to exist. Outside of nvidia you'd have to reverse engineer their machine code which would be a massive undertaking. The ISA ISA is published by AMD and Intel, you could build tooling from the docs alone if you wish.
- c0n5pir4cy 3y agoAMD has released HIP and a tool called HIPIFY which kind of behaves like this but at the source level¹. Rather than try and just translate CUDA to work on AMD compute they are more focused on higher level tooling. Currently they seem to have a particular focus on AI frameworks and tools like PyTorch/Tensorflow/ONNX. They have sponsored and helped with a lot of PyTorch development for example, so PyTorch support for AMD is much better than it was this time last year². ¹(https://github.com/ROCm/HIP https://github.com/ROCm/HIP) ²(https://pytorch.org/blog/experience-power-pytorch-2.0/ https://pytorch.org/blog/experience-power-pytorch-2.0/)
- doikor 3y agoSimulating nuclear explosions, effects of decay on nuclear weapon stockpiles, etc (this is literally what the El Capitan supercomputer mentioned in the article is for)
- neverrroot 3y agoWrong reply, please ignore.
- hutzlibu 3y agoYou can also delete your wrong comments .. (within 2 h I think)
- jacquesm 3y agoUntil you commented...
- neverrroot 3y agoI couldn’t see anything about deleting, only edit. Will check out the web version on a desktop next time, maybe it’s available there.
- hutzlibu 3y agoNot sure, if you have to have more karma for it (would not make sense to me), but for me the delete button is right next to the edit button.
- mkl 3y agoWithin 2h, and if no one has replied.
- neverrroot 3y agoEverything is possible, especially what they have in mind, where they will do a custom implementation. CUDA is important for the existing ecosystem, but that doesn’t make it the only show in town.
- bayindirh 3y agoHPC admin here. ROCm, which is AMD's equivalent of CUDA. The thing is you don't have to directly interface with CUDA or ROCm. Once the framework you want to use supports these, you're done. AMD is consistently getting used on the TOP500 machines, and this gives them insane amounts of money to improve ROCm. CUDA has an ecosystem and hardware moat, but not because they're vastly superior, but because AMD both prioritized processors first, and NVIDIA played dirty (OpenCL support and performance shenanigans, anyone?). This moat is bound to be eaten away by both Intel and AMD, and compute will be commoditized. NVIDIA foresaw this and bought Mellanox and wanted ARM to be a complete black box, but it didn't work out. Ethernet consortium woke up and got agitated by the fact that only ultra-low latency fabric provider is not independent anymore, so they're started to build alternatives to Infiniband ecosystem. Interesting times are ahead. Also there's OneAPI, but I'm not very knowledgeable about it. It's a superAPI which can target many compute platforms, like OpenCL, but takes a different approach. It compiles to platform native artifacts (CUDA/ROCm/Intel/x86/custom, etc.) IIRC.
- justinclift 3y ago> they're started to build alternatives to Infiniband ecosystem. Cool, that sounds interesting. Anything you can point at? :)
- timschmidt 3y agoUltraEthernet
- justinclift 3y agoThanks. :)
- viewtransform 3y agoDiscussion on ultrafast ethernet and partners at AMD's December presentation. https://youtu.be/tfSZqjxsr0M?t=5198 https://youtu.be/tfSZqjxsr0M?t=5198
- formerly_proven 3y ago
- FeepingCreature 3y agoIs this actually a GPU? Ie. can it even render graphics and scan out to a monitor?
- Pomfers 3y agoAMD's compute oriented cards used to come with displayport output, but I haven't seen one of those in a long time. These cards are definitely GPUs in that they can handle graphical workloads, but I don't think anyone is trying to make them work for video games or the like.
- brucethemoose2 3y agoAMD still sells firepros (EG the W7900) like that.
- Dylan16807 3y agoThat's just a variant of the RX 7900. It's an entirely different architecture.
- slavik81 3y agoIIRC the graphics hardware was dropped in CDNA architectures (MI100, MI200, and MI300). For example, I don't think there are texture units.
- daemonologist 3y agoI heard a rumor somewhere that Stadia ran on MI25s - not sure if that's true but certainly there have been a lot of them floating around on eBay in the past year.
- maciejgryka 3y agoOxide and Friends just did an episode talking about this: https://oxide.computer/podcasts/oxide-and-friends/1643335 https://oxide.computer/podcasts/oxide-and-friends/1643335
- aidenn0 3y agoI just discovered On the Metal earlier this year and finished it a week ago. I'll probably get to this episode of Oxide & Friends sometime next year.
- yazaddaruvala 3y agoAnyone have insight on: Why don’t they position the SOC to have the HBM in the center with the CCDs and XCDs on the perimeter? Seems to me that would yield lower “wire length” through the interconnect for each CCD/XCD to all of the memory.
- deleted 3y ago[deleted]
- soganess 3y agocomplete and total unfounded guess... thermal expansion?
- PedroBatista 3y agoMaybe thermals? Idk
- jauntywundrkind 3y agoThe memory isn't designed to connect to a lot of different chips. So there's no reason to put it in the center. The core systems of interconnect are at the base most center-most (well, there's a passive interposer too, but it's just wires): the IOD. These intermediate connections across the chips on top of them, the IOD next to them, and the memory.
- brucethemoose2 3y agoThe XCDs need to talk to each other very quickly, way faster than the HBM, to act like a single chip. Also, the physical interconnect between the XCDs is different than the HBM.
- thrtythreeforty 3y agoHBM is designed for the wire lengths that you're talking about. The core protocol isn't so terribly different from good old DDR4. The die-to-die links, on the other hand, have extremely short wire length limits (less than a centimeter iirc for UCIe). At the physical layer, you just waste a bunch of power and area driving a high voltage to make the signal go further. Do that enough and you basically wind up with a PCIe PHY.
- mkj 3y agoHow do they physically align all the parts? Do they have some kind of self-aligning mechanism or it's done with external manipulation? (Or maybe that's TSMC's secret)
- brucethemoose2 3y agoYeah, it is TSMC, see: https://3dfabric.tsmc.com/english/dedicatedFoundry/technology/3DFabric.htm https://3dfabric.tsmc.com/english/dedicatedFoundry/technolog... Older explanation, but lots of this stuff is just now shipping: https://www.anandtech.com/show/16051/3dfabric-the-home-for-tsmc-2-5d-and-3d-stacking-roadmap https://www.anandtech.com/show/16051/3dfabric-the-home-for-t... Not that AMD doesn't deserve any credit, they have considerable multi chip experience under their belt and undoubtedly served as a guinea pig/pipe cleaner for TSMC's advanced package.
- amelius 3y agoIt doesn't sound very complicated compared to, say, aligning the masks.
- sylware 3y agoAsk ASML. The contraints all depend on the scale of the alignment required. I wonder what is the wavelength range used for their interferometers, and what kind of mecanical engines they use (probably piezo electric based engines).
- bob1029 3y agoI think the other parameters are way more difficult to get right than any optical system alignment considerations. Things like statistical process control make photolithography look less scary. You can check if you correctly patterned the wafer almost immediately. You won't know if the layer is any good until many process steps later. Maybe not for sure until EDS. Tuning the interactions between manufacturing processes is the actual secret sauce that all the manufacturers are trying to protect. How much dose on the EUV machine depends a lot on how you intend to etch the wafer. Imagine iteration cycles measured in months for changing individual floating point variables.
- JonChesterfield 3y agoI want one of these a lot. Spec https://www.amd.com/en/products/accelerators/instinct/mi300/mi300a.html https://www.amd.com/en/products/accelerators/instinct/mi300/.... It's 24 x64 cores and 228 CDNA compute units on a common 128GB of memory. Personally I want to run constraint solvers on one. The general approach of doing lots of integer/float work on the GPU and branchy work on the CPU, both hitting the same memory, feels like an order of magnitude capability improvement over the current systems.
- brucethemoose2 3y agoUnfortunately the BoM on these things is probably super high. Like abdornally high even for a datacenter GPU. AMD is coming out with a "strix halo" APU somewhat appropriate for compute. https://hothardware.com/news/amd-strix-halo-cpu-rumors https://hothardware.com/news/amd-strix-halo-cpu-rumors
- abstractcontrol 3y agoBoM?
- MR4D 3y agoBill of Materials. Also know as cost of the chip.
- pie420 3y agoBill of Materials should really only be used for things that require a list of items/labor, and aren't sold individually such as a datacenter, a building, a hardware integration project, a wedding, etc. For things that are sold individually, "cost" will suffice, and BoM is an example of incorrectly using a more complicated term for the sake of seeming smart.
- ska 3y ago> Also know as cost of the chip. As a shorthand for expected retail cost, that is terribly misapplied. Or even for internal cost. The chip would be one line item on the BOM for the entire thing you plan on shipping (but not packaged yet). Even in the case you are "just" selling a chip, the BOM is likely more complicated,and in this context primarily the manufacturing side cares about that. The COGS (cost of goods sold) is something the company as a whole will care more about - this is what it actually costs you to get it out the door. You will hear "BOM cost" referring to the elemental cost of one item on the BOM, but that's not the BOM itself. None of these are related to the retail (or wholesale) cost in a simple way, either than forming a floors on long term sustainable price. The GGG-whatever post is using this sloppily to suggest that the chips are going to be very expensive to produce, therefore the product is going to be expensive. You'll also see BOM in a materials and labor type invoice, (like when you get your car serviced) but that's not relevant here.
- pk-protect-ai 3y agoPing me when the software stack for the AMD hardware is as good as CUDA.
- FL33TW00D 3y agoThere is a truly gigantic demand for this - I expect you won't be waiting too long.
- throwup238 3y agoI’ve been waiting sixteen years.
- FL33TW00D 3y agoThere has never been more money riding on eliminating the CUDA monopoly than now.
- ShamelessC 3y agoThat’s been true for 16 years.
- bryanlarsen 3y agoIt's now a trillion dollar market (as measured by market cap). This has only been true for a few months.
- cyanydeez 3y agowell, that's with the 1000% profit on Nvidias monopoly.
- michaelt 3y agoEh, 16 years ago CUDA was the cheap option, compared to other HPC offerings. And there wasn't a parts shortage (modulo some cryptocurrency mining, but that impacted both GPU vendors) And ML models weren't so large as to make 8GB of vram sound meagre. And there weren't a bunch of venture capitalists throwing money at the work, because the state of the art models were doing uninspiring things. Like trying to tag your holiday photos, but doing it wrong because they couldn't tell a bicycle helmet and a bicycle apart.
- dharma1 3y agoPriced in on the stock yet? Looks like team red starting to nip at Nvidia’s heels again
- drexlspivey 3y agoIt’s around 12% up since their event but some of it is beta, everything is pumping
- redder23 3y agoI do not have much clue about chips but when AMD bought ATI I thought they will come out with some superchip that combines CPU and GPU into one. This is just for AI? So not for gaming PCs?
- pstuart 3y agoI imagine they'll go for the big money stuff first and it will trickle back to gamers.
- kube-system 3y ago> when AMD bought ATI I thought they will come out with some superchip that combines CPU and GPU into one. That is exactly what did happen. The year was 2006 and it was called the AMD Fusion project. AMD launched the chips, that they call "APUs", in 2011. Nowadays, this configuration is common in both AMD and Intel "CPUs".
- aappleby 3y agoThat is literally what drives the PS5 and XSX.
- orbital-decay 3y agoThat's a lot of layers with different thermal expansion coefficients and conductivity. How do they cool all this?
- bryanlarsen 3y agoThe AMD announcement spent a surprising amount of time on this issue and the ways they're dealing with it.
- flkenosad 3y agoCan't wait for n-layered chips.
- fancyfredbot 3y ago"MI300 [is] three slices of silicon high and can sling as much as 17 terabytes of data vertically between those slices" After you transfer 17 terabytes of data, it is worn out and you can't use it any more
- deleted 3y ago[deleted]
- metabagel 3y agoPer second
- fancyfredbot 3y agoProbably :-) The literal interpretation of the article is funnier though.
- opwieurposiu 3y agoOnce the ones wear down below 0.5 they start to look like zeros.
- fancyfredbot 3y agoExactly. Moving vertically against gravity gradually squashes the bits down.
- tgtweak 3y agoI think AMDs shady marketing where they claimed 1.4x over H100 is enough to just steer clear of the hype and wait for results. Summary: they cherrypicked legacy nvidia sdk's and used llama batch sizes that are not used often in production... https://twitter.com/karlfreund/status/1735078641631998271 https://twitter.com/karlfreund/status/1735078641631998271 https://developer.nvidia.com/blog/achieving-top-inference-performance-with-the-nvidia-h100-tensor-core-gpu-and-nvidia-tensorrt-llm/ https://developer.nvidia.com/blog/achieving-top-inference-pe...
- oelang 3y agoIf you're looking for fair comparisons don't ask nVidias marketing department, those guys are worse than Intel. What AMD did was a true comparison, while nvidia is applying their transformer engine which modifies & optimizes some of the computation to FP8 & they claim no measurable change in output. So yes, nvidia has some software tricks left up on their sleeve and that makes comparisons hard, but the fact remains that their best hardware can't match the mi300x in raw power. Given some time, AMD can apply the same software optimizations, or one of their partners will. I think AMD will likely hold the hardware advantage for a while, nVidia doesn't have any product that uses chiplets while AMD has been developing this technology for years. If the trend continues to have these huge AI chips, AMD has a better hand to economically scale their AI chips.
- jz391 3y agoNot my area, but isn't a lot of NVIDIA's edge over AMD precisely software? NVIDIA seem to employ a lot of software dev (for a hardware company) & made CUDA into the de facto standard for much ML work. Do you know if AMD are closing that gap?
- oelang 3y agoThey have improved their software significantly in the last year, but there is a movement that's broader than AMD that wants to get rid of CUDA. The entire industry is motivated to break the nvidia monopoly. The cloud providers, various startups & established players like intel are building their own AI solutions. Simultaneously, CUDA is rarely used directly, typically a higher level (Python) API that can target any low-level API like cuda, PTX or rocm. What AMD is lacking right now is decent support for rocm on their customer cards on all platforms. Right now if you don't have one of these MI cards or a rx7900 & you're not running linux you're not going to have a nice time. I believe the reason for this is that they have 2 different architectures, CDNA (the MI cards) and RDNA (the customer hardware).
- ooterness 3y agoDoes this mean some of the AMD-Xilinx FPGAs will start stacking DRAM on-chip?
- wtallis 3y agoThey've had FPGAs with HBM DRAM for several years.
- bawana 3y agoBut apple m1 chip has 1/6 the power budget of a 4090 and benchmarks show that they are similar. Will apple eat them? https://owehrens.com/whisper-nvidia-rtx-4090-vs-m1pro-with-mlx/ https://owehrens.com/whisper-nvidia-rtx-4090-vs-m1pro-with-m...