4 ms·

If there's a decrease in AI ram demand, it will not be because the models get better. Models getting better will increase RAM demand, because it grows the part

If there's a decrease in AI ram demand, it will not be because the models get better. Models getting better will increase RAM demand, because it grows the part of the economy that models are useful for. Classic Jevon's Paradox.

cogman10 24d ago
I'm expecting the drop for a couple of reasons. For 1, AI datacenter builders have said that they have more equipment than they have places to put them. Leaving a ton of hardware shelved while you wait for datacenters to build out is bad business to say the least. For 2, I think we are nearing saturation for the usefulness of AI. I certainly could be wrong, but I don't really foresee there to be a bunch of new exciting usages of AI that will ultimately justify the continued buildout.
- znpy 24d ago
  > For 2, I think we are nearing saturation for the usefulness of AI. I certainly could be wrong Unless some other breakthrough in AI happens, like applying AI to some other large market. Overall i think that AI is here to stay, if anything it will become more common. Maybe in ten years we’ll all have some 256gb ai machine at home, like we most (all?) of us have AC.
  - josephg 24d ago
    I think LLMs will start being etched into custom chips. There’s already some companies experimenting with it. The benefits are enormous - you get better performance, lower memory usage and no need to load a model in ram. Companies would be able to sell local LLMs while keeping their models proprietary. If that happens, hopefully the ram price comes back to earth.
    - switchbak 24d ago
      I’d love to buy a tiny swappable coprocessor that slots in next to my cpu and can take this stuff on. Swap it out every 8 months as you can benefit from something better. Hell, sign me up for a subscription!
      - deleted 24d ago
        [deleted]
- jack_pp 24d ago
  the better AI becomes the more non-programmers will be able to use it to build apps, and because they don't know how to program themselves they will use FAR more tokens than us to achieve the same thing. As long as AI is good enough for them to build whatever they desire there will be a lot of demand imo.
  - josephg 24d ago
    I’ve been thinking about this question for months: with access to good LLMs and unlimited tokens, how many non developers will end up making software? I’m sure some people will. But most people I know don’t think about software like that. They don’t know what use cases are, or ask themselves how their life could be simpler with different software. LLMs only make software if you prompt them. Most people don’t frame the problems they have in life as “I’m missing software to solve problem X”. I think most people will never prompt LLMs to write software. Just like I don’t have a 3d printer, because I don’t think about how problems in my life could be solved with little plastic doodads.
    - jack_pp 24d ago
      I've met at least one person that has gone pretty far on a somewhat complicated project who has literally zero programming experience. he did it for fun. I also know someone that thought about making a SaaS for his specific niche he works in, he asked me if I ever used Claude then told me his project. Yes, most people will never do it because they're not the type of people that are self-motivated and courageous enough to dive into something with confidence to see how it goes. But probably, as more people build stuff and it makes headlines that X person built Y service and is making Z$$ per month and he can't even write a hello world to save his life, a lot more people will try it out.
      - skydhash 24d ago
        > Yes, most people will never do it because they're not the type of people that are self-motivated and courageous enough to dive into something with confidence to see how it goes Or they’re just not interested. Just look around to see how many things that people are doing that you have no interest into. That’s the same disinterest people have towards programming and sys admin.
- baxtr 24d ago
  Re 2: Even if that was true penetration is still very low. If all companies start using AI at scale the demand will skyrocket.
  - taurath 24d ago
    > If all companies start using AI at scale the demand will skyrocket This is why ROI is so critical, and I’d argue that picture looks a lot less optimistic for it to pan out functionality wise today than it did in March, when everyone was being forced to use it. Faced with millions of dollars in token costs and gains not materializing, the question is more about the overall usefulness of LLMs as a matter of returns. We do seem to be spending trillions of dollars to drown our public spaces in grey goo
HDThoreaun 24d ago
Yes, but a right now a lot of the RAM buy is for demand that doesnt exist yet. Big tech is projecting AI demand and making sure they have enough compute to fulfill that, so AI demand would have to increase even more than they expect, otherwise theyll build their datacenters and then slow down the amount they buy.

cogman10 24d ago

I'm expecting the drop for a couple of reasons. For 1, AI datacenter builders have said that they have more equipment than they have places to put them. Leaving a ton of hardware shelved while you wait for datacenters to build out is bad business to say the least. For 2, I think we are nearing saturation for the usefulness of AI. I certainly could be wrong, but I don't really foresee there to be a bunch of new exciting usages of AI that will ultimately justify the continued buildout.

znpy 24d ago
> For 2, I think we are nearing saturation for the usefulness of AI. I certainly could be wrong Unless some other breakthrough in AI happens, like applying AI to some other large market. Overall i think that AI is here to stay, if anything it will become more common. Maybe in ten years we’ll all have some 256gb ai machine at home, like we most (all?) of us have AC.
- josephg 24d ago
  I think LLMs will start being etched into custom chips. There’s already some companies experimenting with it. The benefits are enormous - you get better performance, lower memory usage and no need to load a model in ram. Companies would be able to sell local LLMs while keeping their models proprietary. If that happens, hopefully the ram price comes back to earth.
  - switchbak 24d ago
    I’d love to buy a tiny swappable coprocessor that slots in next to my cpu and can take this stuff on. Swap it out every 8 months as you can benefit from something better. Hell, sign me up for a subscription!
    - deleted 24d ago
      [deleted]
jack_pp 24d ago
the better AI becomes the more non-programmers will be able to use it to build apps, and because they don't know how to program themselves they will use FAR more tokens than us to achieve the same thing. As long as AI is good enough for them to build whatever they desire there will be a lot of demand imo.
- josephg 24d ago
  I’ve been thinking about this question for months: with access to good LLMs and unlimited tokens, how many non developers will end up making software? I’m sure some people will. But most people I know don’t think about software like that. They don’t know what use cases are, or ask themselves how their life could be simpler with different software. LLMs only make software if you prompt them. Most people don’t frame the problems they have in life as “I’m missing software to solve problem X”. I think most people will never prompt LLMs to write software. Just like I don’t have a 3d printer, because I don’t think about how problems in my life could be solved with little plastic doodads.
  - jack_pp 24d ago
    I've met at least one person that has gone pretty far on a somewhat complicated project who has literally zero programming experience. he did it for fun. I also know someone that thought about making a SaaS for his specific niche he works in, he asked me if I ever used Claude then told me his project. Yes, most people will never do it because they're not the type of people that are self-motivated and courageous enough to dive into something with confidence to see how it goes. But probably, as more people build stuff and it makes headlines that X person built Y service and is making Z$$ per month and he can't even write a hello world to save his life, a lot more people will try it out.
    - skydhash 24d ago
      > Yes, most people will never do it because they're not the type of people that are self-motivated and courageous enough to dive into something with confidence to see how it goes Or they’re just not interested. Just look around to see how many things that people are doing that you have no interest into. That’s the same disinterest people have towards programming and sys admin.
baxtr 24d ago
Re 2: Even if that was true penetration is still very low. If all companies start using AI at scale the demand will skyrocket.
- taurath 24d ago
  > If all companies start using AI at scale the demand will skyrocket This is why ROI is so critical, and I’d argue that picture looks a lot less optimistic for it to pan out functionality wise today than it did in March, when everyone was being forced to use it. Faced with millions of dollars in token costs and gains not materializing, the question is more about the overall usefulness of LLMs as a matter of returns. We do seem to be spending trillions of dollars to drown our public spaces in grey goo

HDThoreaun 24d ago

Yes, but a right now a lot of the RAM buy is for demand that doesnt exist yet. Big tech is projecting AI demand and making sure they have enough compute to fulfill that, so AI demand would have to increase even more than they expect, otherwise theyll build their datacenters and then slow down the amount they buy.