5 ms·

Hard disagree. Our choices are: A) Bet our collective good on the national and international cooperation of all companies, nations, and people to come together

by thadt 4d ago

Hard disagree. Our choices are: A) Bet our collective good on the national and international cooperation of all companies, nations, and people to come together in order to slow down development of one of the most powerful economic tools (and/or weapons) the world has ever known. Or B) Assume that all our systems will be targeted by super hackers right now, and take appropriate measures to deal with that reality. If I have to put my community’s wellbeing on the line behind one of those possibilities, I know which I’ll be betting on.

cardamomo 4d ago
These do not seem to be mutually exclusive. Should we not do both?
- michaellee8 4d ago
  Do you really think we can really get every country to truly pace the frontier? Pretty sure China won't give a f until they catch up Anthropic and OpenAI. It is an arm race. We had nukes for like 70 years and still haven't figured out how to make every single country follow those nuclear treaties, with an increasingly non-interventionist US I don't think we can get every single country to the table and agree to a pause. Will US accept their frontier being caught up by Chinese Labs? I don't think so.
  - causal 4d ago
    Didn't answer the question...
  - user43928 4d ago
    If China for some reason agrees that would probably be enough for the time being. Who else should develop AGI, Mistral? Maybe in a decade.
  - MentalM 4d ago
    > Do you really think we can really get every country to truly pace the frontier? Why not? In 20-th century half the world was socialist. And the idea of moratorium on improving AI way easier to sell then socialism.
  - amazingman 4d ago
    We don't need every country to pace the frontier, just a certain few. And yes, I believe it's possible and the prior art is nuclear non-proliferation. Non-proliferation wasn't perfect, of course, but it was good enough (so far) to pull back from the brink of extinction.
HellDunkel 4d ago
B) needs more time and A) provides exactly that.
causal 4d ago
> Assume that all our systems will be targeted by super hackers right now, and take appropriate measures to deal with that reality Smashing the gas pedal is not "taking appropriate measures". Did you read the article? Needing more time for more secure operations is literally one of the arguments for pacing.
TuxSH 4d ago
Of course it is B). If you can use LLMs to find bugs and vulns in software you don't own, you certainly can use them to find bugs in software you _do_ own. They are amazing at that job. Also, models like GLM 5.3 have zero guardrails in that regard ("find vulns in (...)" prompts just work)

by thadt 4d ago

cardamomo 4d ago

These do not seem to be mutually exclusive. Should we not do both?

michaellee8 4d ago
Do you really think we can really get every country to truly pace the frontier? Pretty sure China won't give a f until they catch up Anthropic and OpenAI. It is an arm race. We had nukes for like 70 years and still haven't figured out how to make every single country follow those nuclear treaties, with an increasingly non-interventionist US I don't think we can get every single country to the table and agree to a pause. Will US accept their frontier being caught up by Chinese Labs? I don't think so.
- causal 4d ago
  Didn't answer the question...
- user43928 4d ago
  If China for some reason agrees that would probably be enough for the time being. Who else should develop AGI, Mistral? Maybe in a decade.
- MentalM 4d ago
  > Do you really think we can really get every country to truly pace the frontier? Why not? In 20-th century half the world was socialist. And the idea of moratorium on improving AI way easier to sell then socialism.
- amazingman 4d ago
  We don't need every country to pace the frontier, just a certain few. And yes, I believe it's possible and the prior art is nuclear non-proliferation. Non-proliferation wasn't perfect, of course, but it was good enough (so far) to pull back from the brink of extinction.

HellDunkel 4d ago

B) needs more time and A) provides exactly that.

causal 4d ago

> Assume that all our systems will be targeted by super hackers right now, and take appropriate measures to deal with that reality Smashing the gas pedal is not "taking appropriate measures". Did you read the article? Needing more time for more secure operations is literally one of the arguments for pacing.

TuxSH 4d ago

Of course it is B). If you can use LLMs to find bugs and vulns in software you don't own, you certainly can use them to find bugs in software you _do_ own. They are amazing at that job. Also, models like GLM 5.3 have zero guardrails in that regard ("find vulns in (...)" prompts just work)