5 ms·
Any clues to how they plan to invest $500 billion dollars? What infrastructure are they planning that will cost that much?
by non- 2y ago
Any clues to how they plan to invest $500 billion dollars? What infrastructure are they planning that will cost that much?
- deleted 2y ago[deleted]
- MangoCoffee 2y agodata center + gpu server farm (?)
- mrandish 2y agoPlus power plants to drive the massive data centers. At large enough scale, power availability and cost is a constraint.
- burnte 2y agoThat was literally my question. Is this basically just for more datacenters, NVidia chips, and electricity with a sprinkling of engineers to run it all? If so, then that $500bn should NOT be invested in today's tech, but instead in making more powerful and power efficient chips, IMO.
- bitmasher9 2y agoI don’t know if $500bn could put anyone ahead of nvidia/tmc.
- entropicdrifter 2y agoNvidia's in on it, so presumably this is a doubling-down on Nvidia as the chip developers
- amluto 2y ago$500bn of usefully deployed engineering, mostly software, seems like it would put AMD far ahead of Nvidia. Actually usefully deploying large amounts of money is not so easy, though, and this would still go through TSMC.
- patall 2y agoHe wanted to do that, but would have needed 5T for that. Only got 100 bn so far, so this is what you get (only slightly /s)
- bdangubic 2y agoif only $500bn was enough to make more powerful and power efficient chips…
- Havoc 2y agoAdd some nuclear power and you’ve suddenly got a big bill
- burnte 2y agoNot really. Plant Vogtle in Georgia was way over budget and still was "only" $35bn. $500bn could get you 14 of those.
- kristianp 2y agoNvidia and TSMC are already working on more powerful and efficient chips, but the physical limits to scaling mean lots more power is going to be used in each new generation of chips. They might improve by offering specific features such as FP4, but Moore's law is still dead.
- pillefitz 2y agoAnd we are, running on a 20W brain.
- TrainedMonkey 2y agoI'll make a wild guess that they will be building data centers and maybe robotic labs. They are starting with 100B of committed by mostly Softbank, but probably not transacted yet, money. > building new AI infrastructure for OpenAI in the United States The carrot is probably something like - we will build enough compute to make a supper intelligence that will solve all the problems, ???, profit.
- deleted 2y ago[deleted]
- K0balt 2y agoIf we look at the processing requirements in nature, I think that the main trend in AI going forward is going to be doing more with less, not doing less with more, as the current scaling is going. Thermodynamic neural networks may also basically turn everything on its ear, especially if we figure out how to scale them like NAND flash. If anything, I would estimate that this is a space-race type effort to “win” the AI “wars”. In the short term, it might work. In the long term, it’s probably going to result in a massive glut in accelerated data center capacity. The trend of technology is towards doing better than natural processes, not doing it 100000x less efficiently. I don’t think AI will be an exception. If we look at what is -theoretically- possible using thermodynamic wells, with current model architectures, for instance, we could (theoretically) make a network that applies 1t parameters in something like 1cm2. It would use about 20watts, back of the napkin, and be able to generate a few thousand T/S. Operational thermodynamic wells have already been demonstrated en silica. There are scaling challenges, cooling requirements, etc but AFAIK no theoretical roadblocks to scaling. Obviously, the theoretical doesn’t translate to results, but it does correlate strongly with the trend. So the real question is, what can we build that can only be done if there are hundreds of millions of NVIDIA GPUs sitting around idle in ten years? Or alternatively, if those systems are depreciated and available on secondary markets? What does that look like?
- pillefitz 2y agoWhat is a thermodynamic well? Couldn't find much on it.
- lukeplato 2y agohopefully nuclear power plants
- croddin 2y agoThis could be a clue https://x.com/sama/status/1756090136935416039 https://x.com/sama/status/1756090136935416039
- deleted 2y ago[deleted]
- jppope 2y agoReasonably speaking, there is no way they can know how they plan to invest $500 billion dollars. The current generation of large language models basically use all human text thats ever been created for the parameters... not really sure where you go after than using the same tech.
- Philpax 2y agoThat's not really true - the current generation, as in "of the last three months", uses reinforcement learning to synthesize new training data for themselves: https://huggingface.co/deepseek-ai/DeepSeek-R1-Zero https://huggingface.co/deepseek-ai/DeepSeek-R1-Zero
- XorNot 2y agoRight but that's kind of the point: there's no way forward which could benefit from "moar data". In fact it's weird we need so much data now - i.e. my son in learning to talk hardly needs to have read the complete works of Shakespeare. If it's possible to produce intelligence from just ingesting text, then current tech companies have all the data they need from their initial scrapes of the internet. They don't need more. That's different to keeping models up to date on current affairs.
- throwaway4aday 2y agoThat's essentially what R1 Zero is showing: > Notably, it is the first open research to validate that reasoning capabilities of LLMs can be incentivized purely through RL, without the need for SFT.
- YetAnotherNick 2y agoO3 high compute requires 1000s of dollars to solve one medium complexity problem like ARC.
- artificialprint 2y agoLight bulbs used to be expensive too, nails as well.
- disambiguation 2y agoYachts, mansions, private jets, maybe some very expensive space heaters.
- deleted 2y ago[deleted]
- layer8 2y agoI’m more interested in how they plan to draw the rest of the damn owl.
- HarHarVeryFunny 2y agoThey are going to buy 50 $10B nuclear aircraft carriers and use them as a power source.
- paulnpace 2y agoCongress.