5 ms·

So the argument is basically: This technology is too dangerous so only _we_ should have access to it. We’re the good guys and only we can ensure a safe use of t

So the argument is basically: This technology is too dangerous so only _we_ should have access to it. We’re the good guys and only we can ensure a safe use of this technology. Quis custodiet ipsos custodes?

Nevermark 2mo ago
> To summarize my and Anthropic’s position, we have not and are not advocating for a ban on open-weights models as a category. We should instead focus on keeping powerful chips out of authoritarian hands, stopping industrial-scale distillation, and requiring safety testing of all sufficiently capable models, open and closed.
- fwn 2mo ago
  Demanding "required safety testing" is demanding a ban. Otherwise the testing would be inconsequential, right? I'm sure he didn't mean just a "lobotomized to be worse than Anthropic products" badge for the test-passing models. If a ban is the implied consequence of failing his "safety" tests, that means that Anthropic was and currently is advocating for a ban on some open-weight models.
  - Nevermark 2mo ago
    Are you replying to the article or me? I have not stated any position. The quote is Anthropic's and was relevant to the comment it followed. My views: I find testing of SOTA models problematic. I find not testing of SOTA models problematic. Neither view on testing is without merit. The right way forward is unlikely to be as simple as either of those, but some carved out balance between them. And it is likely to change over time.
    - fwn 2mo ago
      Yes, to clarify: I was not arguing against you but against the statement you quoted, made by Dario. Your quote was very relevant, as it highlights the foundational lack of intellectual honesty behind the whole Anthropic statement.
      - Nevermark 2mo ago
        I felt like the grudging "if we must" nature of his acceptance of open weights came through honestly. :) It is clear he isn't a champion for them.
  - yadaeno 2mo ago
    It is not demanding a ban at all. Any sane regulation of AI will involve safety testing. This seems like common sense.
    - jwitthuhn 2mo ago
      If models that fail the safety testing are banned, then asking for that testing is asking for a ban. That you believe any sane regulation would involve such a ban doesn't stop it from being a ban.
      - yadaeno 2mo ago
        You are taking a hardline libertarian position. We safety test cars, roads, bridges, pharmaceutical. What happens when the unvetted AI model starts teaching people how to make biological weapons, or convincing kids to kill themselves?
        pastel8739 2mo ago
        The parent is, to my reading, arguing that Anthropic is indeed advocating for a ban. I do not see them making any judgement on whether that ban is justified or not. I don’t think that claim is hardline libertarian.
- mjorgers 2mo ago
  Maybe I should’ve expanded a bit on my comment. I didn’t mean “we” as in Anthropic directly—even though they certainly are advocating for restrictions on how to train AI systems by proposing mandatory safety training—I mean the argument that American AI labs are somehow more responsible than their Chinese counterparts. It’s especially jarring when just last week OpenAI—an American company—accidentally hacked Hugginface when performing safety testing on an upcoming model [1]. If they have the ability to turn off all guardrails when testing out their models—or when selling them to the military—then the safety training is only there for show. If they can pick and choose who should have access to their most powerful model, surely they are trying to act as the world police? [1] https://openai.com/index/hugging-face-model-evaluation-security-incident/ https://openai.com/index/hugging-face-model-evaluation-secur...
MrCheeze 2mo ago
That is in fact Anthropic's entire reason for existence, the belief that AGI is too dangerous to be controlled by OpenAI/Sam Altman. It naturally follows that it would also be too dangerous to be in the hands of literally everyone on earth.
- Cookingboy 2mo ago
  >the belief that AGI is too dangerous to be controlled by OpenAI/Sam Altman. I agree with that assessment. But the Dario's jump went from "AGI should not be controlled by OpenAI/Sam Altman" to "AGI shoudl be controlled by Anthropic/Dario", which is definitely a better scenario for him, but not the rest of the world. >It naturally follows that it would also be too dangerous to be in the hands of literally everyone on earth. In fact, you can argue that in a world where all countries have nuclear weapons is actually a better scenario than a world where nuclear weapons are owned by 1 or 2 American billionaires/trillionaires, no matter if those people believe they are the "good guys".
  - ianm218 2mo ago
    Isn’t it plausibly better if AI is controlled by Dario than Altman even if neither situation is palatable? It feels like people look at the two and dislike both of them so they consider them the same. People who know him frequently refer to Altman as a literal psychopath unprompted. People who don’t know Dario just find him arrogant.
    - Torien 2mo ago
      The comments about vance mean both are bad enough. Any toady shouldn't be controlling Ai
      - ianm218 2mo ago
        Sure nobody should. Still the comment says it wouldn't be better for the world when I would rather someone whose arrogant over a literal psycopath? And that comment just mentions that it's on the US government's radar I don't think it is fair to call him a toady.
orbital-decay 2mo ago
There's more to it than just safety: in Dario's psychopathic mind, obtaining advanced AI will lead to massive military advantage, hence "we" must do it first to prevent "them" attacking "us". See where it's going? Suppose it happened, then what? By Dario's logic there are two possible continuations: 1. start a war by attacking first and 2. do nothing and waste the entire effort because "they" will obtain the same capability a bit later.
dools 2mo ago
“I don’t know about you, but I don’t want to live in a world where someone makes the world a better place better than we do” - Gavin Belson
nullbio 2mo ago
Yeah, which is ironic considering their track record too. The most recent one is every shared Claude chat has been indexed by Google and is searchable on the search results. Then there was the RCE in their CLI that they had for a year and was silently patched without any sort of disclosure.
throwaway27448 2mo ago
Not to mention it's simply farcical America of all places would be considered responsible. You could not name another country on earth who could wreak more damage (although thank god this is waning by the day with Trump destroying the empire in charge)

Nevermark 2mo ago

> To summarize my and Anthropic’s position, we have not and are not advocating for a ban on open-weights models as a category. We should instead focus on keeping powerful chips out of authoritarian hands, stopping industrial-scale distillation, and requiring safety testing of all sufficiently capable models, open and closed.

fwn 2mo ago
Demanding "required safety testing" is demanding a ban. Otherwise the testing would be inconsequential, right? I'm sure he didn't mean just a "lobotomized to be worse than Anthropic products" badge for the test-passing models. If a ban is the implied consequence of failing his "safety" tests, that means that Anthropic was and currently is advocating for a ban on some open-weight models.
- Nevermark 2mo ago
  Are you replying to the article or me? I have not stated any position. The quote is Anthropic's and was relevant to the comment it followed. My views: I find testing of SOTA models problematic. I find not testing of SOTA models problematic. Neither view on testing is without merit. The right way forward is unlikely to be as simple as either of those, but some carved out balance between them. And it is likely to change over time.
  - fwn 2mo ago
    Yes, to clarify: I was not arguing against you but against the statement you quoted, made by Dario. Your quote was very relevant, as it highlights the foundational lack of intellectual honesty behind the whole Anthropic statement.
    - Nevermark 2mo ago
      I felt like the grudging "if we must" nature of his acceptance of open weights came through honestly. :) It is clear he isn't a champion for them.
- yadaeno 2mo ago
  It is not demanding a ban at all. Any sane regulation of AI will involve safety testing. This seems like common sense.
  - jwitthuhn 2mo ago
    If models that fail the safety testing are banned, then asking for that testing is asking for a ban. That you believe any sane regulation would involve such a ban doesn't stop it from being a ban.
    - yadaeno 2mo ago
      You are taking a hardline libertarian position. We safety test cars, roads, bridges, pharmaceutical. What happens when the unvetted AI model starts teaching people how to make biological weapons, or convincing kids to kill themselves?
      - pastel8739 2mo ago
        The parent is, to my reading, arguing that Anthropic is indeed advocating for a ban. I do not see them making any judgement on whether that ban is justified or not. I don’t think that claim is hardline libertarian.
mjorgers 2mo ago
Maybe I should’ve expanded a bit on my comment. I didn’t mean “we” as in Anthropic directly—even though they certainly are advocating for restrictions on how to train AI systems by proposing mandatory safety training—I mean the argument that American AI labs are somehow more responsible than their Chinese counterparts. It’s especially jarring when just last week OpenAI—an American company—accidentally hacked Hugginface when performing safety testing on an upcoming model [1]. If they have the ability to turn off all guardrails when testing out their models—or when selling them to the military—then the safety training is only there for show. If they can pick and choose who should have access to their most powerful model, surely they are trying to act as the world police? [1] https://openai.com/index/hugging-face-model-evaluation-security-incident/ https://openai.com/index/hugging-face-model-evaluation-secur...

MrCheeze 2mo ago

That is in fact Anthropic's entire reason for existence, the belief that AGI is too dangerous to be controlled by OpenAI/Sam Altman. It naturally follows that it would also be too dangerous to be in the hands of literally everyone on earth.

Cookingboy 2mo ago
>the belief that AGI is too dangerous to be controlled by OpenAI/Sam Altman. I agree with that assessment. But the Dario's jump went from "AGI should not be controlled by OpenAI/Sam Altman" to "AGI shoudl be controlled by Anthropic/Dario", which is definitely a better scenario for him, but not the rest of the world. >It naturally follows that it would also be too dangerous to be in the hands of literally everyone on earth. In fact, you can argue that in a world where all countries have nuclear weapons is actually a better scenario than a world where nuclear weapons are owned by 1 or 2 American billionaires/trillionaires, no matter if those people believe they are the "good guys".
- ianm218 2mo ago
  Isn’t it plausibly better if AI is controlled by Dario than Altman even if neither situation is palatable? It feels like people look at the two and dislike both of them so they consider them the same. People who know him frequently refer to Altman as a literal psychopath unprompted. People who don’t know Dario just find him arrogant.
  - Torien 2mo ago
    The comments about vance mean both are bad enough. Any toady shouldn't be controlling Ai
    - ianm218 2mo ago
      Sure nobody should. Still the comment says it wouldn't be better for the world when I would rather someone whose arrogant over a literal psycopath? And that comment just mentions that it's on the US government's radar I don't think it is fair to call him a toady.

orbital-decay 2mo ago

There's more to it than just safety: in Dario's psychopathic mind, obtaining advanced AI will lead to massive military advantage, hence "we" must do it first to prevent "them" attacking "us". See where it's going? Suppose it happened, then what? By Dario's logic there are two possible continuations: 1. start a war by attacking first and 2. do nothing and waste the entire effort because "they" will obtain the same capability a bit later.

dools 2mo ago

“I don’t know about you, but I don’t want to live in a world where someone makes the world a better place better than we do” - Gavin Belson

nullbio 2mo ago

Yeah, which is ironic considering their track record too. The most recent one is every shared Claude chat has been indexed by Google and is searchable on the search results. Then there was the RCE in their CLI that they had for a year and was silently patched without any sort of disclosure.

throwaway27448 2mo ago

Not to mention it's simply farcical America of all places would be considered responsible. You could not name another country on earth who could wreak more damage (although thank god this is waning by the day with Trump destroying the empire in charge)