9 ms·

I think you're mischaracterizing Anthropic uncharitably and lumping them in with other, less savory tech billionaires, and also not thinking through the nuances

I think you're mischaracterizing Anthropic uncharitably and lumping them in with other, less savory tech billionaires, and also not thinking through the nuances here. First, Amodei has taken an unusually strong stance among tech companies for not supplying fascist regimes with fascist tooling; in fact, even when threatened with being labeled a national security risk unless he bent the knee, he didn't. Compare and contrast with OpenAI who leapt at the opportunity to bend the knee, or obviously Elon Musk, etc., etc. When you say 'surveillance and control', that's exactly what got Anthropic labeled a national security supply chain risk: Anthropic's unwillingness to be used for that purpose. Second, it's not clear that giving everyone extremely powerful LLMs is a great idea yet. LLMs can be used for defense and finding vulnerabilities, but that same LLM can be used to create and exploit vulnerabilities, design new lethal weapons, and so on. The history of gun availability in America 'for our freedoms' demonstrates the kind of risk that should be responsibly considered before replicating. And again, there's nuance here; yes, we should not be subjugated by fascist states with sole control of a critical technology obviously; but also, do you trust the median maga 4channer to operate a Mythos-level model with a sense of civilizational responsibility and ethics? It's not an easy and obvious question and it's not as simplistic as your argument would suggest.

SwellJoe 14d ago
Yes, there is nuance. And, I have a Claude subscription partly because they showed more hesitation to provide surveillance tools for spying on US citizens than other vendors. They are not wholly free of ties to the US regime, but they've been better than others. But, I'll come back to "two things can be true". Anthropic is better than some, and in some regards they are navigating a complicated ethical landscape with more care than others. On the other hand, it really looks like they're angling to regulate their open competitors out of the game and one of the tools for doing that is to make claims about safety; Anthropic models are safe and restricted to use by entities they deem safe, open models are not safe because anybody can use them and also who knows what those Chinese people are putting in their models. And again, this also has nuance, models, including the Chinese open models, could be adversarial and we may not know it. Anthropic proved models can be a risk by sabotaging Fable briefly, causing it to produce bad results based on what the model thought it was being used for. This is why I tend to take Anthropic's words with a grain of salt. They're literally doing the unsafe things they say are risks of open models, while still laying claim to the "safe AI company" mantle.
- felixgallo 14d ago
  it's possible that their rationale for making claims about safety is in fact that they, among everyone else, are doing the most to be prudently safe. While that's a powerful tool to compete with, that doesn't make them bad. Amodei and Anthropic have never said "open models are not safe because anyone can use them," in fact to the contrary, they've said "open-weights models that don’t have dangerous capabilities are a public good". It's important not to muddy the water here with assertions about their intentions when they've actually been super clear about that in a way that I, at least, personally find difficult to disagree with -- releasing dangerous-capability models into the wild would likely be a bad idea for humanity. If you disagree, state why. I also think you're confusing multiple different things, calling them all risks, lumping them together as equally bad, and using that to attribute contradictory/shady behavior to Anthropic. Depending on what you mean by 'sabotaging Fable briefly', you could either mean experiments they have run internally to try to improve alignment, or you could mean their attempts to restrict Fable from working on danger-adjacent work. Neither one of those is a 'risk'; they are both risk-analysis or risk-mitigation. That is not them 'doing the unsafe things they say are risks of open models', that is literally them working to avoid the unsafe things they say are risks of open models. They don't, in my experience, 'lay claim' to the 'safe AI company mantle' as much as they, apparently principledly and conscientiously, attempt to be safe and talk about what they're doing -- which is not in and of itself a problem. If you think Anthropic is doing all of this badly, what's your optimum alternative here? What would you do in Amodei's shoes?
  - SwellJoe 14d ago
    I mean when Anthropic made Fable sabotage the work of folks who they believed were working on competing products, by silently degrading performance. They backtracked after pushback from users, making it an explicit downgrade to Opus.
    - felixgallo 14d ago
      oh! You mean when people were trying to distill Fable. I feel like that's a different definition of the word 'sabotage' than is in normal use. If someone is violating the TOS they agreed to with Anthropic, then they should probably not feel bad when Anthropic takes action to deal with that. Would you disagree?

SwellJoe 14d ago

Yes, there is nuance. And, I have a Claude subscription partly because they showed more hesitation to provide surveillance tools for spying on US citizens than other vendors. They are not wholly free of ties to the US regime, but they've been better than others. But, I'll come back to "two things can be true". Anthropic is better than some, and in some regards they are navigating a complicated ethical landscape with more care than others. On the other hand, it really looks like they're angling to regulate their open competitors out of the game and one of the tools for doing that is to make claims about safety; Anthropic models are safe and restricted to use by entities they deem safe, open models are not safe because anybody can use them and also who knows what those Chinese people are putting in their models. And again, this also has nuance, models, including the Chinese open models, could be adversarial and we may not know it. Anthropic proved models can be a risk by sabotaging Fable briefly, causing it to produce bad results based on what the model thought it was being used for. This is why I tend to take Anthropic's words with a grain of salt. They're literally doing the unsafe things they say are risks of open models, while still laying claim to the "safe AI company" mantle.

felixgallo 14d ago
it's possible that their rationale for making claims about safety is in fact that they, among everyone else, are doing the most to be prudently safe. While that's a powerful tool to compete with, that doesn't make them bad. Amodei and Anthropic have never said "open models are not safe because anyone can use them," in fact to the contrary, they've said "open-weights models that don’t have dangerous capabilities are a public good". It's important not to muddy the water here with assertions about their intentions when they've actually been super clear about that in a way that I, at least, personally find difficult to disagree with -- releasing dangerous-capability models into the wild would likely be a bad idea for humanity. If you disagree, state why. I also think you're confusing multiple different things, calling them all risks, lumping them together as equally bad, and using that to attribute contradictory/shady behavior to Anthropic. Depending on what you mean by 'sabotaging Fable briefly', you could either mean experiments they have run internally to try to improve alignment, or you could mean their attempts to restrict Fable from working on danger-adjacent work. Neither one of those is a 'risk'; they are both risk-analysis or risk-mitigation. That is not them 'doing the unsafe things they say are risks of open models', that is literally them working to avoid the unsafe things they say are risks of open models. They don't, in my experience, 'lay claim' to the 'safe AI company mantle' as much as they, apparently principledly and conscientiously, attempt to be safe and talk about what they're doing -- which is not in and of itself a problem. If you think Anthropic is doing all of this badly, what's your optimum alternative here? What would you do in Amodei's shoes?
- SwellJoe 14d ago
  I mean when Anthropic made Fable sabotage the work of folks who they believed were working on competing products, by silently degrading performance. They backtracked after pushback from users, making it an explicit downgrade to Opus.
  - felixgallo 14d ago
    oh! You mean when people were trying to distill Fable. I feel like that's a different definition of the word 'sabotage' than is in normal use. If someone is violating the TOS they agreed to with Anthropic, then they should probably not feel bad when Anthropic takes action to deal with that. Would you disagree?