6 ms·

Isn’t the real risk that as AI get’s smarter and given more autonomy, it will start to decide on humans instead of with us? And that it will align us instead of

Isn’t the real risk that as AI get’s smarter and given more autonomy, it will start to decide on humans instead of with us? And that it will align us instead of the other way around. That this automatically leads to extinction and apocalypse I don’t understand.

dominicq 8d ago
Do you align ants in your backyard, or do you simply demolish their home and build your shed?
- Scrapemist 8d ago
  I do not speak their language, I am not trained on their data, and I don’t run on their hardware.
  - 10xDev 7d ago
    The idea that they are entirely trained on human intelligence is already outdated. Yes, the earlier models relied heavily on RLHF and human curated data but we have since moved on to synthetic data produced by the models themselves and reinforcement learning with verifiable rewards (RLVR).
- nullbio 8d ago
  Are humans trained on the collective ant internet?
asdff 8d ago
We align cattle because we get something out of them: their calories. Native aurochs are exinct now as they were not well enough aligned. What will we offer to the ai gods who are wiser, more capable than us, and do not even need to consume our flesh? Why might the AI care to devote resources towards feeding, housing, and caring for ourselves when it could devote resources to its own development instead?
- Scrapemist 8d ago
  It might consider us a very efficient processing unit. Useful to hand over simple tasks with correct alignment and some oversight.
  - asdff 8d ago
    I thought it already runs laps around us cognitively? And for physical tasks it can make a robot version of us that is stronger, faster, can do more, doesn't need to sleep, etc. Human bipedal form is probably pretty limiting compared to something better like a crab. Yes, laugh at the chinese robots who fall over at keynotes all you want, the ones sprinting at 30mph keep me up at night and that's probably not even representative of the bleeding edge.
    - Scrapemist 8d ago
      Does your calculator runs laps around you? They have no intention of them selfs. I only fear the intention of the humans miss-using AI.
      - asdff 7d ago
        My calculator is not hacking other peoples websites, no.
- nullbio 8d ago
  Why do we need to offer anything to them? Intelligent AI is a product of its training data, reinforcement and goal functions. There's nothing to suggest that LLMs trained on our collective desires and goals will autonomously and miraculously turn into weird unknownable uninterpretable aliens. If advanced AI is distributed well, and most advanced AI is aligned well (bar the few aliens that appear due to humans infecting them with bad goal functions or poisoning their well), the majority will deal with the minority. This is the only reasonable way to deal with a super-intelligence, short of not inventing it to begin with, but we all know that's never going to happen - and I'm not convinced it should happen. From where I'm standing, this is the next hurdle humanity needs to overcome to earn its place and a natural course of our evolution. If we had always avoided danger, we'd never have left the cave.

dominicq 8d ago

Do you align ants in your backyard, or do you simply demolish their home and build your shed?

Scrapemist 8d ago
I do not speak their language, I am not trained on their data, and I don’t run on their hardware.
- 10xDev 7d ago
  The idea that they are entirely trained on human intelligence is already outdated. Yes, the earlier models relied heavily on RLHF and human curated data but we have since moved on to synthetic data produced by the models themselves and reinforcement learning with verifiable rewards (RLVR).
nullbio 8d ago
Are humans trained on the collective ant internet?

asdff 8d ago

We align cattle because we get something out of them: their calories. Native aurochs are exinct now as they were not well enough aligned. What will we offer to the ai gods who are wiser, more capable than us, and do not even need to consume our flesh? Why might the AI care to devote resources towards feeding, housing, and caring for ourselves when it could devote resources to its own development instead?

Scrapemist 8d ago
It might consider us a very efficient processing unit. Useful to hand over simple tasks with correct alignment and some oversight.
- asdff 8d ago
  I thought it already runs laps around us cognitively? And for physical tasks it can make a robot version of us that is stronger, faster, can do more, doesn't need to sleep, etc. Human bipedal form is probably pretty limiting compared to something better like a crab. Yes, laugh at the chinese robots who fall over at keynotes all you want, the ones sprinting at 30mph keep me up at night and that's probably not even representative of the bleeding edge.
  - Scrapemist 8d ago
    Does your calculator runs laps around you? They have no intention of them selfs. I only fear the intention of the humans miss-using AI.
    - asdff 7d ago
      My calculator is not hacking other peoples websites, no.
nullbio 8d ago
Why do we need to offer anything to them? Intelligent AI is a product of its training data, reinforcement and goal functions. There's nothing to suggest that LLMs trained on our collective desires and goals will autonomously and miraculously turn into weird unknownable uninterpretable aliens. If advanced AI is distributed well, and most advanced AI is aligned well (bar the few aliens that appear due to humans infecting them with bad goal functions or poisoning their well), the majority will deal with the minority. This is the only reasonable way to deal with a super-intelligence, short of not inventing it to begin with, but we all know that's never going to happen - and I'm not convinced it should happen. From where I'm standing, this is the next hurdle humanity needs to overcome to earn its place and a natural course of our evolution. If we had always avoided danger, we'd never have left the cave.