8 ms·
The problem is that even if the majority of labs decide to slow down, there will be one lab, somewhere, with dark money moving forward. The military wouldn't le
by glimshe 4d ago
The problem is that even if the majority of labs decide to slow down, there will be one lab, somewhere, with dark money moving forward. The military wouldn't let that chance pass by.
- JumpCrisscross 4d ago> there will be one lab, somewhere, with dark money moving forward If this is really an existential threat, that's better than fifty of them.
- nba456_ 4d agoNo it's not. It's worse. Do you have any idea how much power that one lab would have?
- henry2023 4d agoHow is worse?
- jacquesm 4d agoIf they manage to leap ahead they can effectively kill all competition, present and future and you'll be at their mercy.
- henry2023 4d agoSo the solution is to make the company that can’t avoid sub-AGI models from going rouge to try RSI? Also, AI performance has proven to be a function of power and compute, and right now most of the compute lies between two companies rushing towards RSI not because they are afraid someone is going to do this before but because they are the ones who wants to build it.
- UncleMeat 4d agoIf the position is that one of these models will go out of control and destroy humanity then why would having two such models be safer than one?
- jacquesm 4d agoGiven that the arrival of this stuff has been compared to nuclear weapons already the question is whether or not the same mechanisms that kept us from annihilating the planet would hold there too. I am not sure that it will. But a balance duopoly tends to be better than a monopoly and if there are multiple poles then so much the better. I'm past the point where I believe that any of these a-holes is in it 'for the good of humanity', their behavior so far can be extrapolated safely to what they will be like when they are in the control of what effectively would be the largest private arsenal ever created. For a while I thought the AI doomers were off their meds and too paranoid but I'm coming around to the fact that they may have a point: we are rapidly approaching a point of no return, if we are not already past it and the people in charge are exactly the wrong people to be in charge at such a point in time.
- UncleMeat 4d agoHow specifically would a balanced duopoly be safer?
- bigbadfeline 4d ago> If the position is that one of these models will go out of control and destroy humanity then why would having two such models be safer than one? Models don't go out of control, they generate totally harmless tokens. The people directing the models go out of control. The owners of the single powerful model are seriously tempted to go rouge and they're going to do it, sooner or later. Two or more competing groups force the development of defensive capabilities which require strong models too. In addition, offensive capabilities in possession of different competing groups introduce the option of MAD (Mutually Assured Destruction) which is a form of defense that doesn't require defensive capabilities. Worked with nukes.
- JoshTriplett 4d agoThey will if it would be against international treaties to do otherwise, mutually enforced by every other world power.
- dgellow 4d agoWe don’t even need something that powerful, you can control the hardware manufacturers and distributors. If you control GPUs, HBM, and the hardware needed to cool all that down and provide the power, that’s enough to make it very hard to scale up the risk. You cannot really hide an AI datacenter consuming GW of power for too long.
- nba456_ 4d agoExcept no one country controls all that. And if you put limits on it, other countries will speed up doing it themselves.
- JoshTriplett 4d agoIf you stop going "nobody will do anything actually effective", and start going "how do we do something actually successful", any number of successful ideas start readily arising. It is possible to stop. Not easy, but possible.
- koolala 4d agoLike a war or something? I can't think of a successful non-bad way for everyone to stop besides a treaty.
- malfist 4d agoOutside of the current nonsense in Iran, we've never had to go to war to prevent nuclear arms. We've never had to go to war to ban weapons in space. These things are possible. Hard but possible
- susuususj 4d ago[dead]
- threatofrain 4d agoAnd what framework would there be for international cooperation? Without that element, this would just be self-destruction in a global race.
- pllbnk 4d agoI think they will all conveniently "decide" to slow down because models are becoming crazy expensive, both per token and how much tokens they need to do anything meaningful. The difference between Sol and Astra is 150% pricing increase per token. The next one could be, I don't know, Orion, at $150 and it won't do anything meaningfully different than Astra. It's quite safe to say that they have ingested all human knowledge and because models are inherently just statistical generators, they are not expanding it, just rehashing what we already know anyway.