12 ms·

$585B on new fabs, $357B on AI data centers, and $5.8B on humanoid robots. One of those numbers is not like the others

by timerol 3mo ago

kijin 3mo ago
[dead]
b112 3mo ago
Androids (humanoid robots) will require loads of ram, and loads of model training under the current paradigm. So it sort of makes sense. At least, I see robots as the top of the pyramid.
- gavinsyancey 3mo ago
  Autonomous (non-teleoperated) humanoid robots that can do useful work in an unfamiliar environment do not exist. And nobody's close enough to making them to understand if they're possible with our current level of technology, let alone how.
  - red75prime 3mo ago
    If there's no unknown unknowns in the brain, it's most likely possible. As the universal approximation theorem and empirical results of scaling SGD+RL suggest. Whether it will be economically viable remains to be seen. The human cerebellum has a peculiar structure and 80% of the brain's neurons after all.
    - ben_w 3mo ago
      The parameter count equivalent of a human brain is not yet known, but if it was one per synapse then a full human brain replica would need about 1.5e14. We also don't yet know how to be as efficient with training examples as any living creatures' brain, and we only partially make up for this by training on so many examples it would take you a million or so years to do the same, so we'd still stuggle with something proportionally smaller-brained such as a cat. That said, remote controlled androids are going to be economically disruptive, as they make every (unlicensed) job open to outsourcing from an office in a low wage country.
    - xvilka 3mo ago
      Real neurons are orders of magnitude more complex than their artificial pseudo-approximation (it is all based on the century-old understanding of how neurons work). You can think of _individual_ biological neuron as an analog of the small artificial neural network. You can see this simple visual explanation on YouTube[1]. So we aren't even close. It doesn't mean the AI is impossible, it just means people underestimate the "computing power" of real brains, as well as that AI, even the future one might be totally different in how it works from the natural intelligence. [1] https://www.youtube.com/watch?v=hmtQPrH-gC4 https://www.youtube.com/watch?v=hmtQPrH-gC4
      - red75prime 3mo ago
        But for biological neurons to do something that can't be efficiently approximated on a digital computer (but conductive to useful information processing) they need to have unknown unknowns (well, partial unknowns like an unknown quantum algorithm will do too). We don't know the violations of the physical Church-Turing thesis that are conductive for machine learning. We don't have evidence for their existence in the brain (although, the brain would be the prime candidate for finding them as evolution works directly with the true physical laws). BTW, large ANNs don't try to model how the brain does things. They are trying to mimic what the brain does. So, using "how many transistors/artificial neurons it takes to model a biological neuron" is not a good approach. We have no evidence. We even have no solid theories how this can work (Penrose's OrchOR is "OrchOR somehow taps into mathematical knowledge somehow encoded into the structure of spacetime"). But people, for some reason, insist that there should be something there. I can't attribute it to anything else but to deeply entrenched feeling of human exceptionalism.
        formerly_proven 3mo ago
        You're talking about a philosophical debate whether the brain is computable, the other commenters are pointing out that even conservative estimates point to a brain-like NN requiring over a quadrillion parameters.
        red75prime 3mo ago
    - wqaatwt 3mo ago
      We certainly don’t need to replicate humans or overly anthropomorphise robots. Just like cars didn’t need to imitate mechanical horses.
  - p1esk 3mo ago
    We’re experiencing gpt-2 moment in robotics now. This means in about 2-3 years they will do useful work (cooking, repairs, cleaning, etc).
    - ben_w 3mo ago
      The extrapolation cannot be justified. It may be much longer or tomorrow.
      - darig 3mo ago
        [dead]
    - gavinsyancey 3mo ago
      So far all humanoid robot demos have either been either known tasks in a tightly-controlled environment, teleportation, or so poorly-functioning as to be obviously not fit for purpose. It's possible throwing more compute at the problem will work and they'll be useful in a few years. Or maybe we're actually experiencing a Markov-chain / ELIZA moment and are multiple decades away from anything actually useful.
      - cogman10 3mo ago
        I'll do you one further. Self driving cars is far from a solved problem. It's something we've been working on for the last 20 years at least. We are getting closer, some solutions are impressive (like waymo). But even those ultimately need operators in the area of the cars to solve the problem of the car getting stuck. Self driving cars are an infinitely simpler problem to solve vs a general purpose humanoid robot. You have basically 2 outputs, acceleration and steering. You have rules simple enough that I was driving at 14. With enough input, you'd think self driving could be completed in a snap. But it's not there yet. Humanoid robots which are useful will come after widespread deployment of L5 self driving cars. Since we don't have that, I have no faith that we are close to useful humanoids.
        ForOldHack 3mo ago
  - mkl 3mo ago
    Most initial work for them would be in familiar, well-controlled environments - replacing humans in existing factories. I think whether they'd be cost effective for that will remain unknown even after a few years in service though.
    - gavinsyancey 3mo ago
      Factory work that can be straightforwardly automated by robots generally already is, by special-purpose factory robots. The remaining tasks are generally: * Tasks where poorly-paid humans are cheaper than expensive factory robots. Humanoid robots are more complex / fiddly so will be more expensive than existing factory robots. No help here. * Tasks where human dexterity has an advantage over state-of-the-art robot actuators (e.g. sewing fabric panels into garments). Better robotics could help here, but the advancement needed is better actuators, not AI and a humanoid form-factor. And if you solve this you'd be better off putting your new end-effector on an existing 6dof platform. * Supervising robotic equipment and handling exceptions. But then you get to "handling poorly-specified unfamiliar tasks in the physical world" which is not currently a solved problem and there's no guarantee just throwing more compute at it will be sufficient to solve this. So far all humanoid robot demos have either been either known tasks in a tightly-controlled environment, teleportation, or so poorly-functioning as to be obviously not fit for purpose.
      - deleted 3mo ago
        [deleted]
      - cogman10 3mo ago
        Totally agree. I'd also point out that what makes this whole thing smell of being a grift is the fact that what's being chased is humanoid. Humans are not the pinnacle of dexterity or stability. And freed from biological constraints, it makes no sense why we'd use the human form as a reference. Making a humanoid robot is a hard thing to do, for sure, but it's also not particularly useful. The routines needed to balance a robot or correct for a slip are interesting to solve, but don't really make for a better robot which is more capable of doing the dishes or folding the laundry. If I'm amazon, for example, then the most useful form factor for a general purpose robot is 2 arms on a 4 wheel omnidirectional rolling platform.
  - dyauspitr 3mo ago
    We said the same thing about Waymo, that it was perpetually in the future. It took them less than a decade. The robots today are functionally capable, they don’t have the right fuzzy intelligence yet. It’s purely a data problem (lack of) and a lot of people are working on it.
    - darkwater 3mo ago
      Are you saying autonomous driving is a solved problem, even at scale? I haven't seen any Waymo in my small town in Southern Europe yet.
      - sdfsdfs34dfsdf 3mo ago
        > "small town" in "Southern Europe" I've highlighted the two main issues you are currently experiencing.
        darkwater 3mo ago
        Then it's not a solved problem.
      - OkayPhysicist 3mo ago
        From what I've seen with Waymo, autonomous driving in relatively good conditions (light to medium rain, fog, or sunny) is a solved technical problem, even in small, winding streets. Snow is the next big hurdle, and they're actively working on that in Denver and Detroit. They're being conservative with their rollout mostly because of civic issues. Most places do not have a legal framework for "what if your autonomous vehicle hits someone?" yet. Even if Waymos never were at fault for a collision with a person, you can always have cases like the one in Georgia back in October where a bicyclist wasn't looking where they were going and rammed into one. The shaky legal ground is a pretty big impediment right now, and that's in the US where we have much laxer laws about corporations killing people.
        darkwater 3mo ago
  - b112 3mo ago
    https://www.1x.tech/ https://www.1x.tech/
    - RealityVoid 3mo ago
      I have been following them since they were Halodi robotics. They're cool, but nowhere near the level of autonomy you need. My theory is that before household robots become a thing we should have self driving cars be a common occurrence, since that is a much much simpler problem.
- Dylan16807 3mo ago
  If you're running a massive model for logic you're probably better off not putting it in the robot. And it'll be a long time before there's enough robots to make up a significant share of usage. More basic movement control doesn't need loads of ram as far as I know.
  - b112 3mo ago
    The same logic for why self-driving cars can't be cloud based, applies for robots. Something cannot be in the middle of a delicate operation and then "oops!", no network, it just stops. The larger the context window, the better with models. Having a few TB of RAM would be exceptionally helpful. All this just made me realise something however. Having your robot dormant and charging, is a bit of a waste. You could have robots dormant, but its compute in use to act as a compute node. If the distribution of robots is similar world-wide, we'd need a fraction of the datacenters we have now. Using such nodes for training purposes would be beyond advantageous. And the company which can slice up the work and having training done in batches would get the big bucks. And actually, with consumer facing products soon all laden with extra ram and gpu for local compute, that applies there too. Imagine leasing out idle time on your desktop or even laptop for cash. There may be a market here, especially with the cost of new datacenters. Any company able to securely package compute without risking data safety is going to make a mint. Anyone have any ideas?
    - Dylan16807 3mo ago
      > The same logic for why self-driving cars can't be cloud based, applies for robots. Something cannot be in the middle of a delicate operation and then "oops!", no network, it just stops. I don't think you understood my post. The equivalent of self-driving is the movement control I was talking about. Self-driving cars don't have high level logic, except for route planning. Which often is offloaded to the cloud. An extra 30 milliseconds on understanding your speech is nothing. > Imagine leasing out idle time on your desktop or even laptop for cash. https://vast.ai/ https://vast.ai/
      - b112 3mo ago
- ForOldHack 3mo ago
  So... like vertical markets? Ram for the data centers, data centers for the robots, robots for the Humanoid? or... the other way around? Robots to build data centers, and robots to build Ram. I see... tech giants and R & D at the top of the pyramid, and I have seen Gumee, the fab that started all this Korean ram prodution.
hobofan 3mo ago
I don't have the historic numbers at hand, but I would assume that for each of those categories this is a similar proportional increase, so it's similarly notable to mention the increase in spending on humanoid robots.
aenis 3mo ago
How can you spend more on a still early stage tech, well in the depths of R&D?

$585B on new fabs, $357B on AI data centers, and $5.8B on humanoid robots. One of those numbers is not like the others

by timerol 3mo ago

$585B on new fabs, $357B on AI data centers, and $5.8B on humanoid robots. One of those numbers is not like the others

kijin 3mo ago

[dead]

b112 3mo ago

Androids (humanoid robots) will require loads of ram, and loads of model training under the current paradigm. So it sort of makes sense. At least, I see robots as the top of the pyramid.

gavinsyancey 3mo ago
Autonomous (non-teleoperated) humanoid robots that can do useful work in an unfamiliar environment do not exist. And nobody's close enough to making them to understand if they're possible with our current level of technology, let alone how.
- red75prime 3mo ago
  If there's no unknown unknowns in the brain, it's most likely possible. As the universal approximation theorem and empirical results of scaling SGD+RL suggest. Whether it will be economically viable remains to be seen. The human cerebellum has a peculiar structure and 80% of the brain's neurons after all.
  - ben_w 3mo ago
    The parameter count equivalent of a human brain is not yet known, but if it was one per synapse then a full human brain replica would need about 1.5e14. We also don't yet know how to be as efficient with training examples as any living creatures' brain, and we only partially make up for this by training on so many examples it would take you a million or so years to do the same, so we'd still stuggle with something proportionally smaller-brained such as a cat. That said, remote controlled androids are going to be economically disruptive, as they make every (unlicensed) job open to outsourcing from an office in a low wage country.
  - xvilka 3mo ago
    Real neurons are orders of magnitude more complex than their artificial pseudo-approximation (it is all based on the century-old understanding of how neurons work). You can think of _individual_ biological neuron as an analog of the small artificial neural network. You can see this simple visual explanation on YouTube[1]. So we aren't even close. It doesn't mean the AI is impossible, it just means people underestimate the "computing power" of real brains, as well as that AI, even the future one might be totally different in how it works from the natural intelligence. [1] https://www.youtube.com/watch?v=hmtQPrH-gC4 https://www.youtube.com/watch?v=hmtQPrH-gC4
    - red75prime 3mo ago
      But for biological neurons to do something that can't be efficiently approximated on a digital computer (but conductive to useful information processing) they need to have unknown unknowns (well, partial unknowns like an unknown quantum algorithm will do too). We don't know the violations of the physical Church-Turing thesis that are conductive for machine learning. We don't have evidence for their existence in the brain (although, the brain would be the prime candidate for finding them as evolution works directly with the true physical laws). BTW, large ANNs don't try to model how the brain does things. They are trying to mimic what the brain does. So, using "how many transistors/artificial neurons it takes to model a biological neuron" is not a good approach. We have no evidence. We even have no solid theories how this can work (Penrose's OrchOR is "OrchOR somehow taps into mathematical knowledge somehow encoded into the structure of spacetime"). But people, for some reason, insist that there should be something there. I can't attribute it to anything else but to deeply entrenched feeling of human exceptionalism.
      - formerly_proven 3mo ago
        You're talking about a philosophical debate whether the brain is computable, the other commenters are pointing out that even conservative estimates point to a brain-like NN requiring over a quadrillion parameters.
        red75prime 3mo ago
  - wqaatwt 3mo ago
    We certainly don’t need to replicate humans or overly anthropomorphise robots. Just like cars didn’t need to imitate mechanical horses.
- p1esk 3mo ago
  We’re experiencing gpt-2 moment in robotics now. This means in about 2-3 years they will do useful work (cooking, repairs, cleaning, etc).
  - ben_w 3mo ago
    The extrapolation cannot be justified. It may be much longer or tomorrow.
    - darig 3mo ago
      [dead]
  - gavinsyancey 3mo ago
    So far all humanoid robot demos have either been either known tasks in a tightly-controlled environment, teleportation, or so poorly-functioning as to be obviously not fit for purpose. It's possible throwing more compute at the problem will work and they'll be useful in a few years. Or maybe we're actually experiencing a Markov-chain / ELIZA moment and are multiple decades away from anything actually useful.
    - cogman10 3mo ago
      I'll do you one further. Self driving cars is far from a solved problem. It's something we've been working on for the last 20 years at least. We are getting closer, some solutions are impressive (like waymo). But even those ultimately need operators in the area of the cars to solve the problem of the car getting stuck. Self driving cars are an infinitely simpler problem to solve vs a general purpose humanoid robot. You have basically 2 outputs, acceleration and steering. You have rules simple enough that I was driving at 14. With enough input, you'd think self driving could be completed in a snap. But it's not there yet. Humanoid robots which are useful will come after widespread deployment of L5 self driving cars. Since we don't have that, I have no faith that we are close to useful humanoids.
      - ForOldHack 3mo ago
- mkl 3mo ago
  Most initial work for them would be in familiar, well-controlled environments - replacing humans in existing factories. I think whether they'd be cost effective for that will remain unknown even after a few years in service though.
  - gavinsyancey 3mo ago
    Factory work that can be straightforwardly automated by robots generally already is, by special-purpose factory robots. The remaining tasks are generally: * Tasks where poorly-paid humans are cheaper than expensive factory robots. Humanoid robots are more complex / fiddly so will be more expensive than existing factory robots. No help here. * Tasks where human dexterity has an advantage over state-of-the-art robot actuators (e.g. sewing fabric panels into garments). Better robotics could help here, but the advancement needed is better actuators, not AI and a humanoid form-factor. And if you solve this you'd be better off putting your new end-effector on an existing 6dof platform. * Supervising robotic equipment and handling exceptions. But then you get to "handling poorly-specified unfamiliar tasks in the physical world" which is not currently a solved problem and there's no guarantee just throwing more compute at it will be sufficient to solve this. So far all humanoid robot demos have either been either known tasks in a tightly-controlled environment, teleportation, or so poorly-functioning as to be obviously not fit for purpose.
    - deleted 3mo ago
      [deleted]
    - cogman10 3mo ago
      Totally agree. I'd also point out that what makes this whole thing smell of being a grift is the fact that what's being chased is humanoid. Humans are not the pinnacle of dexterity or stability. And freed from biological constraints, it makes no sense why we'd use the human form as a reference. Making a humanoid robot is a hard thing to do, for sure, but it's also not particularly useful. The routines needed to balance a robot or correct for a slip are interesting to solve, but don't really make for a better robot which is more capable of doing the dishes or folding the laundry. If I'm amazon, for example, then the most useful form factor for a general purpose robot is 2 arms on a 4 wheel omnidirectional rolling platform.
- dyauspitr 3mo ago
  We said the same thing about Waymo, that it was perpetually in the future. It took them less than a decade. The robots today are functionally capable, they don’t have the right fuzzy intelligence yet. It’s purely a data problem (lack of) and a lot of people are working on it.
  - darkwater 3mo ago
    Are you saying autonomous driving is a solved problem, even at scale? I haven't seen any Waymo in my small town in Southern Europe yet.
    - sdfsdfs34dfsdf 3mo ago
      > "small town" in "Southern Europe" I've highlighted the two main issues you are currently experiencing.
      - darkwater 3mo ago
        Then it's not a solved problem.
    - OkayPhysicist 3mo ago
      From what I've seen with Waymo, autonomous driving in relatively good conditions (light to medium rain, fog, or sunny) is a solved technical problem, even in small, winding streets. Snow is the next big hurdle, and they're actively working on that in Denver and Detroit. They're being conservative with their rollout mostly because of civic issues. Most places do not have a legal framework for "what if your autonomous vehicle hits someone?" yet. Even if Waymos never were at fault for a collision with a person, you can always have cases like the one in Georgia back in October where a bicyclist wasn't looking where they were going and rammed into one. The shaky legal ground is a pretty big impediment right now, and that's in the US where we have much laxer laws about corporations killing people.
      - darkwater 3mo ago
- b112 3mo ago
  https://www.1x.tech/ https://www.1x.tech/
  - RealityVoid 3mo ago
    I have been following them since they were Halodi robotics. They're cool, but nowhere near the level of autonomy you need. My theory is that before household robots become a thing we should have self driving cars be a common occurrence, since that is a much much simpler problem.
Dylan16807 3mo ago
If you're running a massive model for logic you're probably better off not putting it in the robot. And it'll be a long time before there's enough robots to make up a significant share of usage. More basic movement control doesn't need loads of ram as far as I know.
- b112 3mo ago
  The same logic for why self-driving cars can't be cloud based, applies for robots. Something cannot be in the middle of a delicate operation and then "oops!", no network, it just stops. The larger the context window, the better with models. Having a few TB of RAM would be exceptionally helpful. All this just made me realise something however. Having your robot dormant and charging, is a bit of a waste. You could have robots dormant, but its compute in use to act as a compute node. If the distribution of robots is similar world-wide, we'd need a fraction of the datacenters we have now. Using such nodes for training purposes would be beyond advantageous. And the company which can slice up the work and having training done in batches would get the big bucks. And actually, with consumer facing products soon all laden with extra ram and gpu for local compute, that applies there too. Imagine leasing out idle time on your desktop or even laptop for cash. There may be a market here, especially with the cost of new datacenters. Any company able to securely package compute without risking data safety is going to make a mint. Anyone have any ideas?
  - Dylan16807 3mo ago
    > The same logic for why self-driving cars can't be cloud based, applies for robots. Something cannot be in the middle of a delicate operation and then "oops!", no network, it just stops. I don't think you understood my post. The equivalent of self-driving is the movement control I was talking about. Self-driving cars don't have high level logic, except for route planning. Which often is offloaded to the cloud. An extra 30 milliseconds on understanding your speech is nothing. > Imagine leasing out idle time on your desktop or even laptop for cash. https://vast.ai/ https://vast.ai/
    - b112 3mo ago
ForOldHack 3mo ago
So... like vertical markets? Ram for the data centers, data centers for the robots, robots for the Humanoid? or... the other way around? Robots to build data centers, and robots to build Ram. I see... tech giants and R & D at the top of the pyramid, and I have seen Gumee, the fab that started all this Korean ram prodution.

hobofan 3mo ago

I don't have the historic numbers at hand, but I would assume that for each of those categories this is a similar proportional increase, so it's similarly notable to mention the increase in spending on humanoid robots.

aenis 3mo ago

How can you spend more on a still early stage tech, well in the depths of R&D?