8 ms·
I was baffled by the comparison to the M4 Max. Does this mean that recent AMD chips will be performing at the same level, and what does that mean for on-device
by Kirth 1y ago
I was baffled by the comparison to the M4 Max. Does this mean that recent AMD chips will be performing at the same level, and what does that mean for on-device LLMs? .. or am I misunderstanding this whole ordeal?
- izacus 1y agoYes, the Strix series of AMD uses a similar architecture as M series with massive memory bandwidth and big caches. That results in significantly better performance.
- sidewndr46 1y agoIsn't this the desktop architecture that Torvalds suggested years ago?
- JonChesterfield 1y agoIt basically looks like a games console. Its not a conceptually difficult architecture, "what if the GPU and the CPU had the same memory?". Good things indeed.
- numpad0 1y agoFaster and bigger SRAM cache is as complicated of a solution as adding moar boosters to your rocket. It works, but expensive. RP2040 uses ~8x more die space as its dual CPU just for the RAM.
- sidewndr46 1y agoDo I misunderstand your message here or are you comparing this desktop machine to an embedded microcontroller from Raspberry Pi Limited?
- izacus 1y agoI don't know, but it's primarily very expensive to manufacture and hard to make expandable. You can see people in rage due to soldered RAM in this thread. There's always tradeoffs and people propose many things. Selling those things as a product is another game entirely.
- yonisto 1y agoThis how the Amiga worked 40 years ago...
- schmorptron 1y agoWill we be able to get similar bandwidth with socketed ram with CAMM / LPCAMM modules in the near future?
- topspin 1y agoMaybe, but due to the physics of signal integrity, socketed RAM will always be slower than RAM integrated onto the same PCB as whatever processing element is using it, so by the time CAMM / LPCAMM catches up, some newer integrated RAM solution will be faster yet. This is a matter of physics. It can't be "fixed." Signal integrity is why classic GPU cards have GiBs of integrated RAM chips: GPUs with non-upgradeable RAM that people have been happily buying for years now. Today, the RAM requirements of GPU and their applications has become so large that the extra, low cost, slow, socketed RAM is now a false economy. Naturally, therefore, it's being eliminated as PCs evolve into big GPUs, with one flavor or other of traditional ISA processing elements attached.
- cge 1y agoIt’s possible that Apple really did a disservice to soldered RAM by making it a key profit-increasing option for them, exploiting the inability of buyers to buy RAM elsewhere or upgrade later, but in turn making soldered RAM seem like a scam, when it does have fundamental advantages, as you point out. Going from 64 GB to 128 GB of soldered RAM on the Framework Desktop costs €470, which doesn’t seem that much more expensive than fast socketed RAM. Going from 64 GB to 128 GB on a Mac Studio costs €1000.
- topspin 1y agoAsk yourself this: what is the correct markup for delivering this nearly four years before everyone else? Because that's what Apple did, and why customers have been eagerly paying the cost. Let us all know when you've computed that answer. I'll be interested, because I have no idea how to go about it.
- tracker1 1y ago
- biehl 1y agoI think DHH compares them because they are both the latest, top-line chips. I think DHHs benchmarks show that they have different performance characteristics. But DHHs favorite benchmark favors whatever runs native linux and docker. For local LLM the higher memory bandwith of M4 Max makes it much more performant. Arstechnica has more benchmarks for non-llm things https://arstechnica.com/gadgets/2025/08/review-framework-desktop-is-a-mash-up-of-a-regular-desktop-pc-and-the-mac-studio/ https://arstechnica.com/gadgets/2025/08/review-framework-des...
- rr808 1y agoAfter the appstore fight, DHH's favorite is whatever is not Apple lol. TBF it just opened his eyes to alternatives now is happy off that platform.
- rramon 1y agoHow long until he clashes with the GPL and discovers the BSDs?
- dismalaf 1y agoWhy would that happen? The GPL doesn't conflict at all with anything 37Signals does nor the Rails ecosystem...
- rramon 1y agoNow, but after listening to podcasts with him I think he's someone who would tackle hard stuff like drivers or DSP, so called math genius level coding as soon as it becomes more accessible for him through AI assisted coding. There is a chance to build a real MacOS/iOS alternatives without a JVM abstraction layer on top like Android. The reason it didn't happen yet is the GPL firewall around the Linux kernel imo.
- oblio 1y agoWhat app store fight?
- cdavid 1y agoI was surprised at previous comparison on omarchy website, because apple m* work really well for data science work that don't require GPU. It may be explained by integer vs float performance, though I am too lazy to investigate. A weak data point, using a matrix product of N=6000 matrix by itself on numpy: - SER 8 8745, linux: 280 ms -> 1.53 Tflops (single prec) - my m2 macbook air: it is ~180ms ms -> ~2.4 Tflops (single prec) This is 2 mins of benchmarking on the computers I have. It is not apple to orange comparison (e.g. I use the numpy default blas on each platform), but not completely irrelevant to what people will do w/o much effort. And floating point is what matters for LLM, not integer computation (which is what the ruby test suite is most likely bottlenecked by)
- jychang 1y agoYou're most likely bottlenecked by memory bandwidth for a LLM. The AMD AI MAX 395+ gives you 256GB/sec. The M4 gives you 120GB/s, and the M4 Pro gives you 273GB/s. The M4 Max: 410GB/s (14‑core CPU/32‑core GPU) or 546GB/s (16‑core CPU/40‑core GPU).
- Tuna-Fish 1y agoIt's all about the memory bandwidth. Apple M chips are slower on the computation that AMD chips, but they have soldered on-package fast ram with a wide memory interface, which is very useful on workloads that handle lots of data. Strix halo has a 256-bit LPDDR5X interface, twice as wide as the typical desktop chip, roughly equal to the M4 Pro and half of that of the M4 Max.
- discordance 1y agoNot in perf/watt but perf, yes.
- jchw 1y agoDepends on the benchmark I think. In this case it's probably close. Apple is cagey when it comes to power draw or clock metrics but I believe the M4 max has been seen drawing around 50W in loaded scenarios. Meanwhile, Phoronix clocked the 395+ as drawing an average of 91 watts during their benchmarks. If the performance is ~twice as fast that should be a similar performance per watt. Needless to say it's at least not a dramatic difference the way it was when the M1 came out. edit: Though the M4 Max may be more power hungry than I'm giving it credit, but it's hard to say because I can't figure out if some of these power draw metrics from random Internet posts actually isolate the M4 itself. It looks like when the GPU is loaded it goes much, much higher. https://old.reddit.com/r/macbookpro/comments/1hkhtpp/m4_max_is_reaching_crazy_peak_212w_power/ https://old.reddit.com/r/macbookpro/comments/1hkhtpp/m4_max_...
- pengaru 1y agoIt's not baffling once you realize TSMC is the main defining factor for all these chips, Apple Silicon is simply not that special in the grand scheme of things. Why do you think TSMC's production being in Taiwan is basically a national security issue for the U.S. at this point?
- toasterlovin 1y ago> Apple Silicon is simply not that special in the grand scheme of things Apple Silicon might not be that special from an architecture perspective (although treating integrated GPUs as appropriate for workloads other than low end laptops was a break with industry trends), but it’s very special from an economic perspective. The Apple Silicon unit volumes from iPhones have financed TSMC’s rise to semiconductor process dominance and, it would appear, permanently dethroned Intel.
- MegaDeKay 1y agoApple was just the highest bidder for getting the latest TSMC process. They wouldn't have had a problem getting other customers to buy up that capacity. And Intel's missteps counted for a substantial part of the process dominance you refer to. So I'd argue that Apple isn't that special here either.
- toasterlovin 1y agoUntil Apple forced other chip makers to respond, nobody else was making high end phone processors. And their A series processors are competitive with and have transistor counts comparable to most mobile and desktop PC processors (and have for years). So the alternate reality where Apple isn't a TSMC customer means that TSMC is no longer manufacturing several hundred million high transistor count processors per year. In my opinion, it’s pretty likely TSMC isn’t able to achieve or maintain process dominance without that. Update: To give an idea of the scales involved here, Apple had iPhone revenue in 2024 of about $200B. At an average selling price of $1k, we get 200 million units. Thats a ballpark estimate, they don’t release unit volumes, AFAIK. This link from IDC[1] has the global PC market in 2024 at about 267 million units. Apple also has iPads and Macs, so their unit processor volume is roughly comparable to the entire PC market. But, and this is hugely important: every single processor that Apple ships is comparable in performance (and, thus, transistor counts) to high end PC processors. So their transistor volume probably exceeds the entire PC CPU market. And the majority of it is fabbed on TSMC’s leading process node in any given year. [1]: https://my.idc.com/getdoc.jsp?containerId=prUS53061925 https://my.idc.com/getdoc.jsp?containerId=prUS53061925
- Aurornis 1y agoAn M4 Max has double the memory bandwidth and should run away with similarly optimized benchmarks. An M4 Pro is the more appropriate comparison. I don't know why he's doing price comparisons to a Mac Studio when you can get a 64GB M4 Pro Mac Mini (the closest price/performance comparison point) for much less.
- dismalaf 1y ago> don't know why he's doing price comparisons to a Mac Studio when you can get a 64GB M4 Pro Mac Mini (the closest price/performance comparison point) for much less. Where? An M4 Pro Mac Mini is priced higher than the Framework here in Canada...
- ekianjo 1y agomacs have faster memory access so No, Macs are faster for llms