5 ms·

Generally I don’t think anyone is arguing about the for now part. I don’t think it’s crazy to extrapolate out a few years and ask what kind of danger we’ll be i

Generally I don’t think anyone is arguing about the for now part. I don’t think it’s crazy to extrapolate out a few years and ask what kind of danger we’ll be in then. A team of 10,000 agents just solved the Navier Stokes problem (sans bad behavior by the researchers). Even 1 year ago that would have been unimaginable. What happens to this risk view as: 1. Robotics begin rolling out more broadly across the world. 2. Labs start automating more and more of the physical process of running science as expectations of natural science advances begin to mount. 3. Economic pressure between the labs continues to ramp up and the pressure to continuously improve forces quicker and quicker model releases than a team of human scientists can effectively evaluate outside of automated means. No one knows what pre-conditions are for us to hit the point of no return nor how quickly it will come. If all is required is a sufficiently advanced cyber model we may not be far off. If it requires incredibly complex biological knowledge and access to certain lab supplies we likely have a bit longer. Yes this is guess work and we need more evidence of the dangers but at the same time we need evidence of safety. While you may disagree with the risk level, I think it is easy to see the consequence if these labs achieve their stated goal. At this point it seems a political solution is the only way to enforce caution.

skybrian 10d ago
The mathematics research results are certainly impressive, but I don't see what that has to do with robotics. Waymo is getting somewhere, but it's been a long slog. There doesn't seem to be much progress on, say, package delivery. For more, see: https://secondthoughts.ai/p/14-reasons-robotics-is-hard https://secondthoughts.ai/p/14-reasons-robotics-is-hard
- frrt 10d ago
  That’s because being 99.9999% good isn’t better than 98% + human does the rest + human is liable for screw ups - highly important in edge case scenario’s. It’s more economical. Technology can only generalise+verify so much. E.g automobile production - humans do the QA / touches.
- rurban 10d ago
  Didn't you follow the robotics advances in the Ukrainian - Russian battlefield? There is now a death zone of 50km, only controlled by drones and automatic weapons.
  - red-iron-pine 9d ago
    50km is just the "very likely to get hit" part of the gray zone ISR drones and other tools can go as far as 200km, albeit with worsening visibility. in most cases everything meaningful in that 200k is seen, and the only limitation is range of the drones, artillery, missiles, and potential countermeasures.
  - theamk 9d ago
    Those drones can only stay up for a few hours at most. Then they need refueling and relaunch, with plenty of human-only steps. Even if you have terminator-style AI uprising, all you need to do to be safe from those drones is to wait a day. (The automated weapons can probably last longer, but they still need electricity, and there are still a lot of human steps there)
    - 1attice 5d ago
      Ah yes, everyone knows that an AI that plugs itself in is impossible via like the CAP theorem or something
  - rcxdude 10d ago
    The drones are obviously very important but they only work in the context of everything else on and around the battlefield, especially people. You aren't holding the line with drones alone, and on the whole they are not autonomous.
    - rurban 10d ago
      They are holding the lines with drones alone nowadays. That's why it's called the deathzone, no humans survive. Some drones have FPV operators, some weapons use AI only already.
paul7986 10d ago
Job destruction signals... - Uber is lobbying cities to slow down Waymo rollouts https://www.hcamag.com/us/specialization/transformation/uber-tries-to-slow-driverless-cars-by-siding-with-union/588561 https://www.hcamag.com/us/specialization/transformation/uber...... - 23,000 information sector jobs were lost https://www.axios.com/2026/09/08/jobs-media-software-information https://www.axios.com/2026/09/08/jobs-media-software-informa... - If you have been laid from your info sector/digital creation job you are now competing with 100s of thousands looking for their next such job where Ai can do a lot of the tasks these workers did/do. It's a shitshow for those unemployed looking for their next info sector/digital asset creation job. You are better off doing welding building out the Ai data centers if you want long term properous financial stable employment.
ethbr1 9d ago
> 1. Robotics begin rolling out more broadly across the world. To borrow the Strangelove quip, this is not only necessary but essential. Broader physical robotic deployments are the glue that connects {cheap synthetic intelligence} to {physical outcomes}. The former is a newer phenomenon, in the LLMs-can-approximate-higher-intelligence-well sense, so robotics deployments haven't caught up yet (outside of the highest ROI dark factory scenarios). But it's too economically-valuable not to come.
- LogicFailsMe 9d ago
  They'd also have to be self-assembling from the ground up or rely on human slaves. They'd need to understand the world much better. And they'd have to be incredibly more power efficient. These all remain hard problems right now. That said, you could swarm people with cheap drones (the predicted slaughterbots) that were assembled by robots that eventually break down I guess, but how is that any worse than a crazy human with a gun or what nation states do to people they deem enemies already? The worst these things seem capable of is an attack on the infrastructure and shutting down the Internet. Perhaps this is a great time to insure that our infrastructure is capable of operating autonomously indefinitely and hardening it against attack? But unfortunately, no VC or politician will touch that until something bad happens. So broadly, a better educated Blake Lemoine IMO. What's disturbing to me is how fast and far some junior minion from the frontier labs was platformed all the way to the Wall Street Journal over a tweet that sounds exactly like every other anti-AI tweet. I share concerns about irresponsible deployments of AI, and we need technological literacy in the leadership, but America hates technocracy and prefers populism so buckle up.
  - ethbr1 9d ago
    > They'd also have to be self-assembling from the ground up or rely on human slaves. They'd need to understand the world much better. And they'd have to be incredibly more power efficient. All robotics needs to drive deployment at scale is (1) useful intelligence & (2) performing novel valuable functions. Both of these are satisfied by current tech (LLM + sensors + robotics) and the existing simplest use cases (assembly lines and repetitive tasks). LLMs get you a moderately-intelligent, mostly-reliable orchestration layer of intelligence on top of deterministic hard robotics. And that's enough for a lot of failure-tolerant, feedback-looped cases. E.g. reading 20 dials in a factory throughout the day or monitoring a production line visually, then figuring out what to do about a problem, then effecting that plan. It'll be deployed in heavily utilized spaces at first, but will eventually trickle down throughout the economy. And that will be when 'control over robots' starts to be an existential threat, for denial of critical services at the minimum. And all of this will inevitably roll out as long as ROI is positive, because that's the way the world works.

skybrian 10d ago

The mathematics research results are certainly impressive, but I don't see what that has to do with robotics. Waymo is getting somewhere, but it's been a long slog. There doesn't seem to be much progress on, say, package delivery. For more, see: https://secondthoughts.ai/p/14-reasons-robotics-is-hard https://secondthoughts.ai/p/14-reasons-robotics-is-hard

frrt 10d ago
That’s because being 99.9999% good isn’t better than 98% + human does the rest + human is liable for screw ups - highly important in edge case scenario’s. It’s more economical. Technology can only generalise+verify so much. E.g automobile production - humans do the QA / touches.
rurban 10d ago
Didn't you follow the robotics advances in the Ukrainian - Russian battlefield? There is now a death zone of 50km, only controlled by drones and automatic weapons.
- red-iron-pine 9d ago
  50km is just the "very likely to get hit" part of the gray zone ISR drones and other tools can go as far as 200km, albeit with worsening visibility. in most cases everything meaningful in that 200k is seen, and the only limitation is range of the drones, artillery, missiles, and potential countermeasures.
- theamk 9d ago
  Those drones can only stay up for a few hours at most. Then they need refueling and relaunch, with plenty of human-only steps. Even if you have terminator-style AI uprising, all you need to do to be safe from those drones is to wait a day. (The automated weapons can probably last longer, but they still need electricity, and there are still a lot of human steps there)
  - 1attice 5d ago
    Ah yes, everyone knows that an AI that plugs itself in is impossible via like the CAP theorem or something
- rcxdude 10d ago
  The drones are obviously very important but they only work in the context of everything else on and around the battlefield, especially people. You aren't holding the line with drones alone, and on the whole they are not autonomous.
  - rurban 10d ago
    They are holding the lines with drones alone nowadays. That's why it's called the deathzone, no humans survive. Some drones have FPV operators, some weapons use AI only already.

paul7986 10d ago

Job destruction signals... - Uber is lobbying cities to slow down Waymo rollouts https://www.hcamag.com/us/specialization/transformation/uber-tries-to-slow-driverless-cars-by-siding-with-union/588561 https://www.hcamag.com/us/specialization/transformation/uber...... - 23,000 information sector jobs were lost https://www.axios.com/2026/09/08/jobs-media-software-information https://www.axios.com/2026/09/08/jobs-media-software-informa... - If you have been laid from your info sector/digital creation job you are now competing with 100s of thousands looking for their next such job where Ai can do a lot of the tasks these workers did/do. It's a shitshow for those unemployed looking for their next info sector/digital asset creation job. You are better off doing welding building out the Ai data centers if you want long term properous financial stable employment.

ethbr1 9d ago

> 1. Robotics begin rolling out more broadly across the world. To borrow the Strangelove quip, this is not only necessary but essential. Broader physical robotic deployments are the glue that connects {cheap synthetic intelligence} to {physical outcomes}. The former is a newer phenomenon, in the LLMs-can-approximate-higher-intelligence-well sense, so robotics deployments haven't caught up yet (outside of the highest ROI dark factory scenarios). But it's too economically-valuable not to come.

LogicFailsMe 9d ago
They'd also have to be self-assembling from the ground up or rely on human slaves. They'd need to understand the world much better. And they'd have to be incredibly more power efficient. These all remain hard problems right now. That said, you could swarm people with cheap drones (the predicted slaughterbots) that were assembled by robots that eventually break down I guess, but how is that any worse than a crazy human with a gun or what nation states do to people they deem enemies already? The worst these things seem capable of is an attack on the infrastructure and shutting down the Internet. Perhaps this is a great time to insure that our infrastructure is capable of operating autonomously indefinitely and hardening it against attack? But unfortunately, no VC or politician will touch that until something bad happens. So broadly, a better educated Blake Lemoine IMO. What's disturbing to me is how fast and far some junior minion from the frontier labs was platformed all the way to the Wall Street Journal over a tweet that sounds exactly like every other anti-AI tweet. I share concerns about irresponsible deployments of AI, and we need technological literacy in the leadership, but America hates technocracy and prefers populism so buckle up.
- ethbr1 9d ago
  > They'd also have to be self-assembling from the ground up or rely on human slaves. They'd need to understand the world much better. And they'd have to be incredibly more power efficient. All robotics needs to drive deployment at scale is (1) useful intelligence & (2) performing novel valuable functions. Both of these are satisfied by current tech (LLM + sensors + robotics) and the existing simplest use cases (assembly lines and repetitive tasks). LLMs get you a moderately-intelligent, mostly-reliable orchestration layer of intelligence on top of deterministic hard robotics. And that's enough for a lot of failure-tolerant, feedback-looped cases. E.g. reading 20 dials in a factory throughout the day or monitoring a production line visually, then figuring out what to do about a problem, then effecting that plan. It'll be deployed in heavily utilized spaces at first, but will eventually trickle down throughout the economy. And that will be when 'control over robots' starts to be an existential threat, for denial of critical services at the minimum. And all of this will inevitably roll out as long as ROI is positive, because that's the way the world works.