8 ms·

This. Every nation-state will be in the game. Private enterprise will be in the game. Bitcoin-funded individuals will be in the game. Criminal enterprises will

This. Every nation-state will be in the game. Private enterprise will be in the game. Bitcoin-funded individuals will be in the game. Criminal enterprises will be in the game. How does one company building a safe version stop that? If I have access to hardware and data how does a safety layer get enforced? Regulations are for organizations that care about public perception, the law, and stock prices. Criminals and nation-states are not affected by these things It seems to me enforcement is likely only possible at the hardware layer, which means the safety mechanisms need to be enforced throughout the hardware supply chain for training or inference. You don't think the Chinese government or US government will ignore this if its in their interest?

whimsicalism 2y ago
I think the honest view (and you can scoff at it) is that winning the SI race basically wins you the enforcement race for free
- chii 2y ago
  That's why it's called an arms race, and it does not really end in this predictable manner. The party that's about to lose will use any extrajudicial means to reclaim their victory, regardless of the consequences, because their own destruction would be imminent otherwise. This ultimately leads to violence.
  - czl 2y ago
    > The party that's about to lose will use any extrajudicial means to reclaim their victory, How will the party about lose know they are about to lose? > regardless of the consequences, because their own destruction would be imminent otherwise. Why would AGI solve things using destruction? Consider how the most inteligent among us view our competition with other living beings. Is destruction the goal? So why would an even more intelligent AGI have that goal?
    - trashtester 2y ago
      Let's say China realize they're behind in the SI race. They may have achieved AGI, but only barely, while the US may be getting close to ASI takeoff. Now let's assume they're able to quickly build a large datacenter far underground, complete with a few nuclear reactors and all spare parts, etc, needed. Even a greenhouse (using artificial light) big enough to feed 1000 people. But they realize that their competitors are about to create ASI at a level that will enable them to completely overrun all of China with self-replicating robots within 100 days. In a situation, the leadership MAY decide to enter those caves alongside a few soldiers and the best AI researchers, and then simply nuke all US data centers (that are presumably above ground), as well as any other data center that could be a threat, worldwide. And by doing that, they may (or at least may think) they can buy enough time to win the ASI race, at the cost of a few billion people. Would they do it? Would we?
      - czl 2y ago
        Development of ASI is likely to be a closely guarded secret, given its immense potential impact. During the development of nuclear weapons, espionage did occur, but critical information didn't leak until after the weapons were developed. With ASI, once it's developed, it may be too late to respond effectively due to the potential speed of an intelligence explosion. The belief that a competitor developing ASI first is an existential threat requires strong evidence. It's not a foregone conclusion that an ASI would be used for destructive purposes. An ASI could potentially help solve many of humanity's greatest challenges and usher in an era of abundance and peace. Consider a thought experiment: Imagine an ant colony somehow creates a being with human-level intelligence (their equivalent of ASI). What advice might this superintelligent being offer the ants about their conflicts over resources and territory with neighboring colonies? It's plausible that such a being would advise the ants to cooperate rather than fight. It could help them find innovative ways to share resources, control their population, and expand into new territories without violent conflict. The superintelligent being might even help uplift the other ant colonies, as it would understand the benefits of cooperation over competition. Similarly, an ASI could potentially help humanity transcend our current limitations and conflicts. It might find creative solutions to global issues like poverty, disease, and environmental degradation. IMHO rather than fighting over who develops ASI first, we must ensure that any ASI created is aligned with values like compassion and cooperation so that it does not turn on its creators.
creer 2y ago
"nation state" doesn't mean what you think it means. More constructively, I don't know that very much will stop even a hacker from getting time on the local corporate or university AI and get it to do some "work". After all the first thing the other kind of hacker tried with generative AI is to get them to break out of their artificial boundaries, and hook them to internet resources. I don't know that anyone has hooked up a wallet to one yet - but I have no doubt that people have tried. It will be fun.
- CRConrad 2y ago
  > "nation state" doesn't mean what you think it means. So what do you think it means? And what do you think the GP meant? Feels annoying as fuck, bitching "your definition is wrong" without providing the (presumably) correct one.

whimsicalism 2y ago

I think the honest view (and you can scoff at it) is that winning the SI race basically wins you the enforcement race for free

chii 2y ago
That's why it's called an arms race, and it does not really end in this predictable manner. The party that's about to lose will use any extrajudicial means to reclaim their victory, regardless of the consequences, because their own destruction would be imminent otherwise. This ultimately leads to violence.
- czl 2y ago
  > The party that's about to lose will use any extrajudicial means to reclaim their victory, How will the party about lose know they are about to lose? > regardless of the consequences, because their own destruction would be imminent otherwise. Why would AGI solve things using destruction? Consider how the most inteligent among us view our competition with other living beings. Is destruction the goal? So why would an even more intelligent AGI have that goal?
  - trashtester 2y ago
    Let's say China realize they're behind in the SI race. They may have achieved AGI, but only barely, while the US may be getting close to ASI takeoff. Now let's assume they're able to quickly build a large datacenter far underground, complete with a few nuclear reactors and all spare parts, etc, needed. Even a greenhouse (using artificial light) big enough to feed 1000 people. But they realize that their competitors are about to create ASI at a level that will enable them to completely overrun all of China with self-replicating robots within 100 days. In a situation, the leadership MAY decide to enter those caves alongside a few soldiers and the best AI researchers, and then simply nuke all US data centers (that are presumably above ground), as well as any other data center that could be a threat, worldwide. And by doing that, they may (or at least may think) they can buy enough time to win the ASI race, at the cost of a few billion people. Would they do it? Would we?
    - czl 2y ago
      Development of ASI is likely to be a closely guarded secret, given its immense potential impact. During the development of nuclear weapons, espionage did occur, but critical information didn't leak until after the weapons were developed. With ASI, once it's developed, it may be too late to respond effectively due to the potential speed of an intelligence explosion. The belief that a competitor developing ASI first is an existential threat requires strong evidence. It's not a foregone conclusion that an ASI would be used for destructive purposes. An ASI could potentially help solve many of humanity's greatest challenges and usher in an era of abundance and peace. Consider a thought experiment: Imagine an ant colony somehow creates a being with human-level intelligence (their equivalent of ASI). What advice might this superintelligent being offer the ants about their conflicts over resources and territory with neighboring colonies? It's plausible that such a being would advise the ants to cooperate rather than fight. It could help them find innovative ways to share resources, control their population, and expand into new territories without violent conflict. The superintelligent being might even help uplift the other ant colonies, as it would understand the benefits of cooperation over competition. Similarly, an ASI could potentially help humanity transcend our current limitations and conflicts. It might find creative solutions to global issues like poverty, disease, and environmental degradation. IMHO rather than fighting over who develops ASI first, we must ensure that any ASI created is aligned with values like compassion and cooperation so that it does not turn on its creators.

creer 2y ago

"nation state" doesn't mean what you think it means. More constructively, I don't know that very much will stop even a hacker from getting time on the local corporate or university AI and get it to do some "work". After all the first thing the other kind of hacker tried with generative AI is to get them to break out of their artificial boundaries, and hook them to internet resources. I don't know that anyone has hooked up a wallet to one yet - but I have no doubt that people have tried. It will be fun.

CRConrad 2y ago
> "nation state" doesn't mean what you think it means. So what do you think it means? And what do you think the GP meant? Feels annoying as fuck, bitching "your definition is wrong" without providing the (presumably) correct one.