11 ms·
Memory has grown to nearly two-thirds of AI chip component costs
- cineticdaffodil 4mo agoI find it deeply ironic, that iran has blocked helium supply- while it relies on AI created slopaganda to subvert its advesary. Its one of those afterwits of history.
- Ylpertnodi 4mo ago> iran...slopaganda A US soldier i know commented that the iranian ai slop is "scary and powerful".
- ElenaDaibunny 4mo agounified memory architectures are getting more interesting for inference workloads.
- IAmGraydon 4mo agoBuilt a new machine with 64GB DDR5 and 5TB SSD in January 2025. It's sheer luck that I dodged that bullet.
- slicktux 4mo agoI bought 96GB of RAM a couple of years ago for ~$250. That same RAM now costs $1200!
- ksec 4mo agoIt is one of the thing with consumer when they remember they brought it at the absolutely lowest price point when DRAM maker were bleeding money. Those are not normal pricing. Before the pricing collapse in early 2020, 96GB DDR5 would have cost about $450 to $500. And I will need to restate again the cost of DRAM hasn't really changed much in the past 20 years. Its price just goes up and down in cycles. So in reality it is more like going from $500 to $1300. But consumer felt it was more like going from $200 to $1300. Crucial are already selling DRAM made by CXMT. And China are already throwing money at it. I doubt the memory bubble will burst in next 12-24 months. As in going back to money losing DRAM pricing. As they will all pivot to HBM or other money making products. But the bulk of lower end consumer DDR5 or LPDDR5 will goes to Chinese Foundry. Assuming they have figure out how to do them well. Which they have improved but are still so far away from industry leaders. Normally memory maker will push the next DDR standard to market just to push out Chinese competitors, I am not sure it will work the same this time around. DDR5 have plenty of other usage / demands.
- DoctorOetker 4mo ago> Crucial are already selling DRAM made by CXMT. Crucial was disestablished this year.
- voxic11 4mo agoHe probably meant Corsair which is the DRAM brand selling CXMT produced chips.
- trollbridge 4mo agoAh, the old decrucialisestablishmentarianism.
- DoctorOetker 4mo agoI found the phrasing weird myself, I quoted wikipedia
- cogman10 4mo ago> Its price just goes up and down in cycles. Historically the price has always trended downward. When I first got into computing $200 could buy you 128 MB (yes M) of ram. Really nice systems had 512 MB. That's obviously changed over the decades as process shrinks have lead to higher memory density. We should generally expect that ram will cheaper up and until the point where process shrinks stop happening. They've definitely slowed, but they haven't stopped.
- NikolaNovak 4mo agoExactly. My first computer had 48k, yes K of ram :-). My first PC has 2MB and made all my friends jealous as they had 1MB. Amiga 500 at the time had half. I am keeping a piece of paper that came with my Tex Murphy game which stated that one could get 32MB of RAM for as little as $700 (1990s dollars) which would drastically improve the game!
- ksec 4mo ago>They've definitely slowed, but they haven't stopped. Yes if you span into 40 years. But the spot price for DRAM floor was ~$2/GB in 2008 and touched that 2-3 times over the next 15 year. It wasn't until early 2020s it broke that into $1. Process shrinks happen but majority of DRAM part can't be shrinked by process any more.
- adroitboss 4mo agoI paid $279 for crucial 96gb DDR5 5600 MHz SO-DIMM ram October 22 of last year. Amazon has the same kit going for $1,048.90 right now.
- Joel_Mckay 4mo agoNice, you were lucky. =3
- burnt-resistor 4mo agoCORSAIR Vengeance 96GB (2 x 48GB) SO-DIMM DDR5 5600 CMSX48GX5M1A5600C48 Bought an extra one by accident, paid $218.99 March 2025 Goes for $1400 now. I haven't gotten around to selling it.
- bushbaba 4mo agoMakes prior assumptions that getting tens of gigs of ram is cheap thrown out the window. Would likely lead to super fast SSDs such as optain being way more valuable
- moregrist 4mo agoThe price of SSDs is similarly depressing.
- dawnerd 4mo agoI’m so mad I didn’t max out my main server when I had the chance. Used enterprise sticks were dirt cheap on eBay.
- Forgeties79 4mo agoUsed enterprise HDD’s also jacked up now. It’s absurd lol
- dawnerd 4mo agoYep mad about that too. I was about half way through upgrading my 45 drives server when they started to go up.
- Forgeties79 4mo agoBrutal. I only did a 6 drive bay and I was angry lol
- HDBaseT 4mo agoHe is talking about the company '45Drives', which produce solutions which aren't only 45 drive bays. [0] - https://www.45drives.com https://www.45drives.com
- Forgeties79 4mo agoAppreciate the clarification!
- peroxy 4mo agoJust decided to buy 8 drives for my NAS and was surprised to see nothing in stock anywhere + prices are 3-4x higher than half a year ago. Just wasted 2k eur for 8x8tb, it should be plenty enough for my NAS but I feel stupid having to waste so much money.
- 4mo ago
- IshKebab 4mo agoI bought a couple of used computers with 256 GB of DDR 4 (total) a year ago. The ram is worth more than I paid for the whole machines now.
- DarkUranium 4mo agoSomeone was selling an Epyc machine with 512GB RAM @ 500 EUR last year. I regret not buying it now ...
- giancarlostoro 4mo agoRamflation
- trollbridge 4mo agoI bought 192GB of DDR3 a year ago for literally $60 ($5 a stick). It's about $22 a stick now, so more like $350 today. What on earth is _anybody_ doing with DDR3?
- chinathrow 4mo agoBeing desperate?
- manquer 4mo agoAll memory products use many shared resources in the supply chain, so if there is high demand in one product line, others have to raise prices to compete for the resources or stop making those lines altogether. That is to say at least you were able to buy them at $350 today, with the current trajectory there will be no supply at all in few months.
- jlokier 4mo agoDemand for DDR3 is up because people who want DDR5 or DDR4 but can't afford either any more are choosing DDR3 and old DDR3-compatible systems to put it in, instead of what they really want.
- Keyframe 4mo agoAt the rate we're going, soon we're going to draw from SIMM stock.
- trollbridge 4mo agoJust to be clear, this was to go into an ancient Dell T420 NUMA system. Well over 10 years old.
- kristopolous 4mo agothere's an economic term for this: substitute good. https://en.wikipedia.org/wiki/Substitute_good https://en.wikipedia.org/wiki/Substitute_good
- zozbot234 4mo ago
- Forgeties79 4mo ago2x16gb for $105 total April of 2025. $600 for that now. Makes no sense.
- shevy-java 4mo agoMy main computer has 64GB. I bought that one in late 2022 or so. Looking at the current prices, even of the same RAM, is just insane. Those companies really need to pay us compensation damage here. The whole "free market" notion does not work when you have de-facto monopolies and mega-corporations abuse average Joe and average Jane.
- jmspring 4mo agoI just found two 4tb Samsung EVO drives - unused - while organizing my garage.
- jmspring 4mo agoI forgot to add, I paid ~500 each, Samsung for the same drive is quoting $2k on their site (maybe a new sku). These were bought 2ish years ago. Strange things are a foot at the Circle-K.
- malwrar 4mo agoLucky find! Just picked up one of those for a build, ohhhh boy was that a painful purchase. Thank god for my fortune to work in tech.
- journal 4mo agoyea, but people now have more money.
- rldjbpin 4mo agopaid a bit more than that just for a half-decent 16 gig stick recently :) i compensate by never paying for AI
- positron26 4mo agoThe algorithm advances are going to crash this so hard.
- Coffeewine 4mo agoI mean, god willing, but it'll be just as likely that we'll blissfully consume 100 million token contexts in that case.
- iamtheworstdev 4mo agoisn't there a law for that? as things become cheaper you consume more?
- loloquwowndueo 4mo agoJevons paradox. https://en.wikipedia.org/wiki/Jevons_paradox https://en.wikipedia.org/wiki/Jevons_paradox
- sidhantdhar 4mo agojevons paradox
- simonw 4mo agoJevons paradox: https://en.wikipedia.org/wiki/Jevons_paradox https://en.wikipedia.org/wiki/Jevons_paradox
- sobellian 4mo agoYou're probably thinking about jevons paradox. But you slightly mis-stated. It is the phenomenon that increasing the efficiency of resource consumption can end up increasing total consumption. As you stated it, it would merely be a property of (nearly) all demand curves. Jevons paradox only happens sometimes. It isn't a law.
- dangus 4mo agoAn example of where it stopped happening is with gasoline in developed countries. Cars having better fuel efficiency doesn’t make me drive further to the grocery store or work. Generally when someone replaces their vehicle the new one is more fuel efficient than the old one even if I bought the same car.
- oceansky 4mo agoAwful time for gamers and PC hobbyists not fully into AI.
- lacunary 4mo agoalso for ones fully into AI
- aunty_helen 4mo agoThis is 100% going to kill the home built pc market. When I started building gaming pcs, the top top card was 750$ (NZD). Now they’re 10,000 just for the gpu and another 1-2000 for ram. People used to get into gaming pcs as an affordable hobby, now it’s making general aviation look like plan B.
- johnvanommen 4mo agoYes, this will definitely renew interest in Stadia type products.
- Gigachad 4mo agoWhy? Those servers still have to pay the same price for components plus a markup for the service. In theory you can serve more gamers per GPU, but these GPUs have to be physically located in your city to have a usable latency, and that means you'll have issues with peak utilization being most users gaming at the same time of day. I just don't see the cost savings of sharing a GPU overcoming the extra expense + profit such a service would need.
- HDBaseT 4mo agoThe GPUs do not have to be "psychically located in your city" to have usable latency. Of course, less latency is always better although running a traceroute between my IP and major city (Sydney) from 1,500 km equates to about 11ms latency with optimal routing. (Real life test, traceroute via an ISP Looking Glass).
- deadbabe 4mo agoHere’s the thing, what if memory manufacturers take this opportunity to collude and basically never reduce the price of memory below the current levels since it’s too hard for a new competitor to just rise up and undercut them? Everything I hear about is how hard and risky it is to spin up a new fab. And by doing this, they ensure local LLMs never become feasible for the vast majority of people and AI companies solidify subscriptions forever.
- aurareturn 4mo agoKeeping prices at this level is precisely how one or more competitor will rise up. Making memory isn’t super hard. That’s why it is a commodity. The problem with the memory market is that up and down cycles have bankrupted the vast majority of players in the past. Now we only have 3 players left except for a few smaller ones in China. The reason memory prices can stay high for years in this mega cycle is because the 3 players will be very cautious on overbuilding. They’d rather under build, make great profit (not maximum) and reduce the risk of going bust if this suddenly ends. Same for TSMC in chips. Great opportunity for Chinese companies though. This shortage is exactly what Chinese companies need to scale.
- jazzyjackson 4mo ago> up and down cycles have bankrupted the vast majority of players in the past Exactly, so what’s the incentive for anyone to sink half a billy into building out more capacity. The existing players get to rest on their laurels and succeed whether or not the AI bubble busts.
- aurareturn 4mo agoThe incentive is that your 2 competitors will build more than you and gain market share on you if you are too conservative. Samsung, SK Hynix, and Micron all have to balance between capex spending, making as much profit as possible, and risk of bankruptcy.
- 4mo ago
- elorant 4mo agoBought a second hand Dell server a week ago. The entire rig with a 12-core CPU and 32GB DDR4 ecc RAM cost as much as I'd pay to buy 64 GB of DDR RAM alone. I hope there's an end to this absurdity soon enough otherwise the pain will affect other markets too. I read the other day that PC case sales have collapsed by more than 40%.
- nik282000 4mo agoI feel like by the time the AI bubble bursts the PC market will be irreparably damaged. Manufactures who have been making "enterprise" parts aren't going to go back to making consumer parts because there will be no market for it. And with a glut of datacenters not making any money on slop, they are going to be repurposed for saas, stuff like OnShape but for every application. Most users don't seem to care about storing everything they generate in cloud services and this could easily be sold as an alternative to owning "expensive" desktop or laptop hardware.
- dawnerd 4mo agoThey’re going to pivot to you renting desktop cloud compute instead of owning anything.
- bitwize 4mo agoEnjoy your HP laptop subscription, it's all the computer you're going to get moving forward.
- nik282000 4mo agoIt's the reason I just build a new PC, despite the insane prices, I'd rather overpay than have reasonable prices but no stock to buy. With any luck I'll get 8-10 years out of this one and by then the PC landscape will be something else entirely.
- MattDamonSpace 4mo ago“Bubble”
- MrGilbert 4mo agoI assume that memory manufacturers don’t really care where the money is coming from, as long as the "numbers go up" game is working. NVIDIA in their recent quarterly report stopped categorizing "Geforce" as a single category, and merged it into "Edge-Computing". If you are a PC Gamer or PC Enthusiast as I am, then we have some dark times ahead.
- reactordev 4mo agoDo we though? DLSS 5 changes that somewhat from a “we need powah” to “we need models”. I think the future consumer GPU market will be tuned for image and world inference while workstation cards will be tuned for image and video inference. The old way of thinking about this will come to an end when we stop looking at the render loop as the be-all-end-all… Or, we could be fucked.
- kg 4mo agoIf DLSS 5 becomes the norm it's possible that just makes things worse. The DLSS 5 demos required an entire separate card to run the model, though IIRC NVIDIA did claim it would eventually work on a single card. Given what the model is doing (yassifying the whole scene instead of just upscaling/reconstructing) it makes sense to me that it would increase compute demand instead of reduce it like previous versions of DLSS.
- reactordev 4mo agoThe demos did, but look how far we have come in just two years? Running local LLMs, running local diffusion models, running local world models (albeit, barely a scene at this point). I do believe that in 10 years time, game will be producing latents and not events they way they do now. I also hope this means that VR can finally get the fidelity it needs to really take off.
- MrGilbert 4mo agoFrom my point of view, I suppose we will enter a "Let AI generate entertainment" era. In which you just might rent everything, including games. No need for a beefy computer at home, you just need a slim endpoint: "Order yours now, for just $99.99 per month, hardware included! Order today, and you will get three months of 'Office Suite' for free, with a small additional cost of $49.99 after month 4. On a tight budget? Switch to the yearly subscription, and pay comfortably in 18 installments."
- amazingamazing 4mo agoA commodity rapidly increasing in price. What could go wrong?
- KronisLV 4mo agoI'm not moving past my DDR4 build (and the 32 GB of DDR4 2133 MHz backup chips I still have around from way back, before I got the current 3200 MHz ones) until the prices go back to being at least partially sane. This also means that CPU manufacturers are not getting my money (since the 5800X is fine for now) and I have no reason to get a new GPU either (though admittedly the B580 isn't perfect).
- johnvanommen 4mo agoWhat if this is the lowest that prices will ever be?
- mrandish 4mo agoAs Yogi Berra famously said, "It's tough to make predictions, especially about the future." But based on historical tech industry trends, a price increase in one component that's this rapid and extreme, is likely to eventually regress somewhat toward the long-term trend line - even if that trend line experiences a longer-term shift upward. As always, some interpret certain recent events as reason to conclude "but this time it's different." Occasionally they are correct. But that doesn't change the fact that it's reasonable to assume some of the recent extreme, rapid price inflation is due to shorter term market distortion. It's also pretty clear that some of the recent increase in demand represents a stable increase in the long-term trendline. The question is how much is long-term stable and how much is short-term distortion.
- KronisLV 4mo agoThen I will make my build last as long as it can, in protest of that. I do expect at least a performative price drop in the coming years, though.
- willis936 4mo agoThen I better divert all of my investment into memory maker stocks.
- daringrain32781 4mo ago[dead]
- brcmthrowaway 4mo agoAnyone invested in Micron stock?
- lostlogin 4mo agoUp 700% in a year. WallstreeetBets has been disturbingly accurate in its predictions - basically anything related to AI.
- DoctorOetker 4mo agoIt's still unclear to me: the shortage is semiconductor boules / wafers? or the shortage is semiconductor fab process step availability? As long as the discussion seems focused on memory, I'd suspect the latter, but if its really the semiconductor boules/wafers, then I'd expect the boule growers to profit, not the memory makers, who just pass on the cost. So which is it?
- AnotherGoodName 4mo agoIt’s fab capacity. Fwiw dram is different enough that fabs are not transferable between dram memory and other usages. It’s nice to think ‘wow if they made the current 10nm dram on the latest 2nm processes it’d be much faster’ but it doesn’t work that way. The specific size is needed for the capacitance. Sram can be made on fabs that make other circuitry since it’s transistor not capacitor based but is less dense. Dram is just extremely specialised.
- DoctorOetker 4mo agoI know the differences between SRAM, DRAM, ... I asked for evidence different people keep feeding me opposite stories: one insists its not fab capacity but wafer competition, with a recent article claiming HBM3E takes 3 times as much wafer area per bit than LPDDR5X. Others tell me the complete opposite: its fab capacity, not wafer shortage. Do we have citable references to ground either set of claims?
- sowbug 4mo agoI believe those are two ways of describing the same thing. If you're able to book some fab capacity, that means you get to decide what the fab does with the next wafers in the queue. From your sibling comment, I think you're interpreting the 3x HBM stat as contributing to making wafers scarce. It's more that the next wafer to be processed in a fab is especially precious, making the opportunity cost larger. The beach sand remains plentiful.
- DoctorOetker 4mo ago
- Legend2440 4mo agoI wonder why the hyperscalers aren't vertically integrating more and building their own fabs. Sure, a fab costs a billion dollars, but they're currently spending hundreds of billions of dollars purchasing chips from NVidia and others.
- epistasis 4mo agoI'm not sure if they should vertically integrate, it would probably be a better idea to directly fund the expansion of capacity, much like Apple does when they scale up a new technology for iPhones. However, that the hyperscalers and AI companies aren't doing this says a lot about their true beliefs about how much future demand AI will have. AI companies claim they will need a ton of massive expansion, but are unwilling to take on the risk of the capital needed for that expansion. I'm hearing a lot of sad whining from AI folks about how these chip makers are holding them back, but who actually has the money to finance the expansion easily? Chip makers have been through this game far longer, when Sam Altman went around claiming it was time for $7T of fabs the AI companies made it clear that they were willing to make ridiculous claims, eliminating credibility. What's needed now is for them to funnel a tiny amount of their massive piles of cash into financing fabs directly.
- energy123 4mo agoOracle is getting sold because of how much capex they're spending on new data centers in the middle of a high rates environment. It's not like they're stockpiling cash due to doubting AI.
- epistasis 4mo agoOracle had not entered into my thoughts at all; I know they do some cloud stuff but they are in a very different position than OpenAI or Anthropic or Google.
- alecco 4mo ago> [...] better idea to directly fund the expansion of capacity [...] > > However, that the hyperscalers and AI companies aren't doing this says a lot about their true beliefs about how much future demand AI will have. With what money? They have to spend the money they get on hardware ASAP else they are left behind.
- skiing_crawling 4mo agoI recently built a system at insane ddr4 prices ($2000 for 256gb). But that’s only after seeing how ddr5 prices were 3-4x that!
- preisschild 4mo agoYeah I upgraded all of my systems to DDR5 last year, so now I have to buy for ddr5 memory upgrades.
- Joel_Mckay 4mo agoHad to fork over almost $1k for a 64G DDR5 kit a few weeks back. At least AMD chips large L3 cache allows folks to get away with lower grade udimms. Also had to do an Intel build, and there was no way we were going cudimm at current prices. =3
- TheGrassyKnoll 4mo agoI wish I had figured that out a year ago. MU up ~10x, SNDK up ~37x. My crystal ball is woefully under performing.
- chvid 4mo agoTime to let ASML sell to the Chinese memory producers … or not.
- mchusma 4mo agoEverything I read seems to suggest that RAM capacity is going to grow at 20-25% a year, which just doesn't seem good enough. Even in consumer use cases, phones and laptops would benefit greatly by double the amount of RAM. And then obviously, the AI need is gigantic. I don't see it going away. I mean, it may not grow as fast as now, but I don't see it growing away either. I get why the memory makers do not want to bankrupt themselves, but it feels like there's got to be some way to push that risk off onto model providers and other people in the ecosystem to allow us to grow ram capacity more like 50% per year.
- foota 4mo agoIn theory the new futures markets for chip components would help here, since it would allow DRAM suppliers to insulate themselves from that risk.
- minraws 4mo agoI mean the biggest risk is Chinese CXML benefits and capturing markets that others are leaving hanging and then being able to compete and push out the others when costs start to normalize. As for 20-25% growth not being enough, I think it's not that far off, if we assume data center build out plans hit a wall and slow down significantly, and the AI heat starts to cool off. I don't think 20-25% may be enough in the short term but if the AI build out stops within this year, we have a massive oversupply instead of a under supply.
- zx8080 4mo agoWhat is the risk? Competition is good for consumers.
- LPisGood 4mo agoThe risk is to the business not the consumers
- bigbadfeline 4mo ago
- Traubenfuchs 4mo agoWhy did this happen so suddenly? Why were tech savy investors unable to figure this out when the datacenter craze had already started? How to explain this lag between quickly rising demand for all datacenter components besides memory?
- johnvanommen 4mo agoNine years after Google's seminal paper lit the fuse on AI, a total lack of manufacturing foresight has trapped over a trillion dollars of incoming capital in a hardware bottleneck. The entire sector is now facing a critical RAM starvation crisis where memory manufacturers are actively slow-rolling supply just to keep prices high and avoid running out entirely. This has created an unprecedented supply-and-demand distortion where desperate companies are getting rejected even at a 5x markup, and mission-critical SKUs are skyrocketing to 10x and 20x their baseline value. It is a macroeconomic squeeze at a staggering scale, and the massive venture scale opportunity lies in capturing the value created by this memory gatekeeper. From the perspective of an armchair economist, the winners will be the investors who invest in RAM wisely. The losers will likely be cash strapped SAAS companies. They’re almost completely dependent on a fleet of servers in the hyperscalers, and they’re leasing those servers and services. That leaves small SAAS companies exposed to incoming inflation in the cost of hosting.
- irthomasthomas 4mo agoA lot of words to say that Sam Altman bought up the worlds total supply of ram chips for the next few years.
- Auracle 4mo agoA dick move or just really prescient?
- regularfry 4mo agoIt's only prescient if it works out. But it's a dick move either way.
- johnvanommen 4mo agoI really don’t want to give anyone ideas, but doesn’t this make the Nvidia 5090 an unbelievably good deal right now? The VRAM in the 5090 is only made by one country in the world. The 50xx series is special, because its ram is so dependent on a single commodity. It’s not like a 4090 or a 3090; their VRAM chips have been around for years. If there’s a shortage or interruption in DDR7 VRAM, it seems like every GPU that requires it would explode in value. I hope I don’t regret posting this because I’d really like to buy one myself…
- mattmanser 4mo agoIt's gone up like 300% in cost in the last year.
- JacobAsmuth 4mo agoWhich surely is the highest it'll ever be! You're suggesting that the price will go down in the future? Would love to hear more about your thought process!
- bcrosby95 4mo agoAre you saying we're entering a period where tech increases in price instead of decreases? I guess it depends upon time horizon, but your statement isn't very specific.
- JacobAsmuth 4mo agoYeah man, obviously. RTX 5090s will almost certainly increase in price over the next two years as memory shortages get worse.
- johnvanommen 4mo agoI believe msrp is $2000 right?
- EnPissant 4mo agoThere was only a very brief time it was selling for MSRP (last fall for $2000). Even if you use that as the previous data point, it's only 200% increased.
- Lapsa 4mo ago[dead]
- I_am_tiberius 4mo agoIt seems to me the max memory you can buy in a laptop stagnated for the past 3 years or so.
- giancarlostoro 4mo agoI have always felt insulted that most laptops even offer a low 4 GB of RAM I rather take 16 GB in previous gen memory
- ffaccount2 4mo agoMy several years old laptop has 128GB of RAM, is that not enough? I admit that it's a pretty heavy one.
- grapedangackle 4mo ago[dead]
- superkuh 4mo agoAnd the max storage in pre-built computers has stagnated at 2010 levels (~1TB). This was first due to the switch to the much more expensive and much faster charge trap flash. In the 2020s it finally started to approach 2010 sizes in pre-builts but then the corporate finance wars re: fab capacity happened.
- rldjbpin 4mo agofor the most part, unless soldered down, it has been hard to find higher than dual channel (maybe quad for a massive odm gaming laptop). each stick and platform having set maximum memory capacity has put a glass ceiling for those machines. doesn't matter anyway when things are not reasonably priced. i am stuck at the same memory capacity in my personal system for the better part of two decades, partially due to the above and the current pricing today.
- gpm 4mo agoAn interesting implication of this is that AI inference and training has a path to a ~3x hardware cost reduction (and maybe ~2x total cost reduction) without any technical innovation whatsoever, we just need to wait for dram supply to meet demand (either by manufacturing scaling or just waiting for the current rate of manufacturing to fill the demand spike).
- fittingopposite 4mo agoReally wondering what this might mean for local LLMs when RAM costs plummet...
- Waterluvian 4mo agoWhat’s the lifespan/refurbishability of the capex elements like the “GPU” modules or even the DRAM soldered into them?
- jmalicki 4mo agoFor lifespan, AWS is still running a ton of T4 GPUs from 2018, that power a lot of computer vision models. A ton of these will have a long life, not all ML is about frontier LLMs.
- ck2 4mo agoif we survive the bubble bursting and there isn't a "too big to fail" bailout with public money manipulation by bought politicians we are going to have amazing cheap used hardware for a decade
- deleted 4mo ago[deleted]
- thinklanceai 4mo ago[flagged]
- hottrends 4mo ago[flagged]
- ecommerceguy 4mo agoAs models gain efficiency, will the need for ram cool?
- throwatdem12311 4mo agoThey’ll just fill up the ram with bigger models. Demand will INCREASE, not decrease.
- helterskelter 4mo agoEvery time we add capacity with almost anything, we find ways to saturate it.
- robot_jesus 4mo agoBraess's paradox for roads. When we add capacity to road networks, traffic increases even more than the capacity. https://en.wikipedia.org/wiki/Braess%27s_paradox https://en.wikipedia.org/wiki/Braess%27s_paradox
- kingstnap 4mo agoJevons paradox is at play. Right now frontier AI is very expensive which heavily suppresses demand. If you made it 10x cheaper right now you would see a truly unimaginable wave of bot slop.
- shevy-java 4mo agoI think the companies that drive up the prices here, need to pay an extra-tax to all of us. I fail to see why I now have to pay more due to the AI monster companies ruining the economy.
- proee 4mo agoMemory manufactures sit on a war chest of IP. So even if someone has excess fab capacity and wants to get into memory manufacturing, they will have to fight an uphill battle of about a zillion patents. Most memory companies have backroom deals to exchange tit-for-tat patent violations against each other. Not sure how a new memory manufacture comes into being without getting sunk from licensing costs?
- byzantinegene 4mo agochina?
- jalospinoso 4mo ago[flagged]
- notnullorvoid 4mo agoGood time to focus on more memory efficient means of training and inference. SeedLM from Apple is an interesting approach for inference memory efficiency. I'd like to see someone try and build that into training so that it's not a post training compression step.
- danborn26 4mo ago[flagged]
- blindriver 4mo agoSince January, I've been lucky and picking up various used DDR4 memory sticks for cheap-ish. I got a total of 64 GB for $180. I feel like I hit the jackpot!
- inciampati 4mo agoMemory makes computation universal.
- flykespice 4mo agoSince memory is becoming an expensive commodity, I guess the old ways of being precious on the efficient memory usage of your program (like it running on the constrained 1mb memory back then) are making a comeback. I only feel sorrow for the electron devs, they will have a hard time.
- cloudengineer94 4mo agoWith how things are going, I'm really wondering how we are gonna tackle the consumer market for things like gaming and machine learning. No doubt Cloud Gaming is in the cards for the future, only purists like myself with an RTX 5090 will pay premium for offline gaming
- weitendorf 4mo agoIn the long run cloud gaming is inevitable, it’s just more economically efficient for the cost of the hardware required to render graphics to be amortized across consumers and not sit idle when being unused by collocating them with game assets in POPs. Once enough gaming compute runs at the edge it also allows for more technically advanced games than would currently be economically feasible (but aren’t made mostly for lack of a market/adoption of cloud gaming and the resulting lack of technical know-how). So I think it will stick and probably end up winning over the holdouts, once the cost of rendering the games they want to play with consumer hardware becomes too large to stomach.
- Marsymars 4mo agoYou could make the same economic argument for any SaaS, but the margins SaaS providers look for make it so that the only time it isn't cheaper to run your own software/hardware stack in place of SaaS is when the hardware requirements are very low, not high. SaaS makes sense economically when you take into account the admin, compliance, etc. costs... and the admin costs of a Nintendo Switch are pretty low.
- willis936 4mo agoEconomic efficiency does not win the day because the free market is a myth. Cloud gaming is a technically worse solution because the latency floor is higher. It's a microeconomic disaster (rent vs buy, buy wins). The only reason it would become a thing is if the multinationals succeed in concentrating more wealth and power, which consumers aren't interested in supporting. It's a bad deal and consumers know it. They would have to be forced into it by having the consumer hardware market taken off the table (which is happening and the only possible avenue for a technical regression like cloud gaming to have a market).
- luxuryballs 4mo agoit’s fun and ironic that “having a memory” is what AI appears to lack the most in practice while at the same time it demands more computer memory than anything to run
- abhaynayar 4mo agoHow can I use this information to MY advantage? Do I started going into something to do with AI chip memory-stuff? If so, how? But just on a software level cause hardware is hard.
- Escapade5160 4mo agoAnd four-fiths the cost of a consumer PC build.
- maxnevermind 4mo agoI wonder if it is reasonable to assume the propagation of shortages further. At first it was GPUs, then RAM, then what?
- aceazzameen 4mo agoFresh water?
- Jasonwang123 4mo agoThe cost of memory should continue go up as we tend to have the AI to have context and remember lots more.
- Developer_H 4mo ago[flagged]
- deleted 4mo ago[deleted]
- SeattleAntifa 4mo ago[dead]
- emsign 4mo agoAI is choking the computing economy. Many companies will die. It's already a mass extinction event and will leave behind deserts.
- deleted 4mo ago[deleted]
- zeristor 4mo agoOr to put it another way, the prices will only come down the other side of an intense catastrophe. AI growth is locked in now, only if it were to stop will demand be abated.