9 ms·
At what point do we stop engaging with Anthropic’s leadership in good faith and acknowledge their track record, - no open weights - can’t use claude to resear
by cuuupid 5d ago
At what point do we stop engaging with Anthropic’s leadership in good faith and acknowledge their track record,
- no open weights
- can’t use claude to research AI
- train on everyone else’s IP and sell it back to them
- 8 regulatory capture attempts and counting
- so controlling they are the only US company blacklisted by the US government
This is not effective altruism / rationalism gone wild, it’s just monopolistic anti-competitive business practices masquerading as ethics, and they’ll continue getting away with this until we look past their sensationalism and hit them with antitrust.
Altman gets so much hate but OpenAI has been a far better steward (on 3/5 above at least) than Anthropic!
- SwellJoe 4d agoI mostly agree with you, but this one: "- so controlling they are the only US company blacklisted by the US government" They were blacklisted by the US government because they didn't bribe Trump. While all the rest of the tech elite lined up to siphon a few million dollars through various Trump grifts, Anthropic sat out the inauguration, the ballroom, etc. Then, when they mildly pushed back on surveilling US citizens with their technology, the administration banned them on that pretext. If there's any evidence Anthropic is one of the better actors in an industry full of bad actors, that's it. They have nothing to be ashamed of in that particular series of events, as far as I can tell.
- daveguy 4d agoWait... Which of the 2 did Altman not do where Amodei did? I mean, maybe not quite 8 attempts at regulatory capture. But come on. And the only reason OpenAI didn't get blacklisted is they rolled over for the worst applications.
- isthispermanent 4d ago100%, rules for thee not for me
- causal 5d agoThe reasoning in Dario's letter here can be correct regardless. But yes, I think Anthropic has done real harm to coordinated AI alignment by being such a controlling and sneaky actor.
- busymom0 5d ago> can’t use claude to research AI What's this about? Where's this rule?
- matheusmoreira 5d agoIn the system cards. Anthropic will literally make Claude sabotage you silently instead of downgrading you to Opus if you try to use Fable for AI research.
- user43928 5d agoDidn't they walk that one back eventually?
- matheusmoreira 5d agoWho knows? It's trivial to "walk back" claims that we can't even verify are happening in the first place. They cannot be trusted.
- Ey7NFZ3P0nzAe 4d agoDon't they just fallback to opus non-silently?
- matheusmoreira 4d agoOnly if you're doing cybersecurity or biology stuff. If you're a possible competitor, they will invisibly fuck with Fable's weights instead to deliberately sabotage you.
- zahlman 4d agoI find it implausible that they simultaneously are so worried about these recursively self-improving systems that they don't properly understand, but can freely "invisibly fuck with the weights" on that level.
- deleted 5d ago[deleted]
- bpodgursky 5d agoDo you have any familiarity with Anthropic at all? At no point did they ever say open weights are a good idea. Their entire thesis is AI IS VERY DANGEROUS AND WE MUST DO IT RIGHT. You can hate it, but everything they do is consistent with this thesis, and everything they say is consistent with their actions! You just want them to want different things.
- softwaredoug 5d agoIt reads like “AI is dangerous, only we should be allowed to make money from it”
- encyclopedism 4d agoExactly He doesn't need anyones permission to do so, go ahead no one is stopping you! If you feel so strongly about it, lead by example. Perhaps others will follow, maybe even China. Regardless backup your sentiment with actions!
- GeneralMayhem 4d agoThere is a specific, unilateral action that he is taking described in the article. And the "why are they developing AI if they think it's so dangerous" argument is neither new nor persuasive. They're developing it because (1) they think the potential benefits are as great as the potential risks, and they know that if they aren't one of the actors on the frontier (2) they won't be able to propose meaningful solutions and (3) their opinions won't be taken seriously. Amodei is in rooms with powerful people to propose these things because Anthropic is successfully developing frontier models. The heads of various NGOs and advocacy groups that are concerned with AI safety but not themselves working on those systems are... nowhere, writing pamphlets and blogs that not you nor I nor anyone in Congress will ever read.
- encyclopedism 4d agoI'll leave this here: https://x.com/DavidSacks/status/2098973625252708460 https://x.com/DavidSacks/status/2098973625252708460
- epihelix 5d ago> At what point do we stop engaging with Anthropic’s leadership in good faith About two years ago? I would also note that Dario's post appears to be LLM written. Maybe... maybe... he's read so much Claudeish that it's all he can speak now himself. But I wonder if he's becoming a bit of a meat proxy. (It's funny, I thought "pace the frontier" was going to mean something similar to "patrolling the frontier". But no, it's pace as in speed of change - everyone must slow down, right now (unless it's Anthropic, but you know we're the good guys in this, right? We're going to get someone to audit our desks!) "Pacing the frontier" feels so LLM.) ok, he doesn't actually say this. But he is getting the desks audited...
- Nition 4d agoFor what it's worth, I didn't get AI-written vibes from it, and Pangram also flags it as 100% human-written.
- roncesvalles 4d ago"Write a blog article about <topic prompt>. Keep rewriting it until it passes Pangram as 100% human-written. <pangram_tool.md>"
- zahlman 4d agoPangram's algorithms are not open-source, so I wouldn't expect this to work very well.
- aarondong 4d agoIf Dario is using an unreleased internal model to write, then Pangram wouldn't be able to flag it I suspect. Pangram needs enough public info about the patterns in generated text.
- abustamam 4d ago> But I wonder if he's becoming a bit of a meat proxy. A joke I recently encountered: --- A CEO proudly announced he'd bought an AI system designed to identify the company's most replaceable employee. It spent the night analyzing five years of emails, meetings, salaries, performance reviews, and productivity data. The next morning, it came back with one name: The CEO. The IT guy was fired for installing defective software.
- jvanderbot 5d agoExactly right. When local models and startup labs can distill / learn / accelerate open models for local use by startups - the rational response by incumbents is to call LLMs doomsday machines that cannot be trusted in the hands of normies.
- TacticalCoder 5d ago> At what point do we stop engaging with Anthropic’s leadership in good faith and acknowledge their track record, ... It's all lies as usual. This announcement has got nothing to do with alignment and pacing the "frontier": all models are getting very close in capabilities and they want to hide that they're not way ahead anymore (say compared to the Chinese or compared to the Geminis) by pretending to slow down due to "alignment" or whatever. We know it's not an announcement made in good faith: reading between the lines they're saying "China is more than catching up, so let's pretend we need to slow down to explain our lack of lead".
- barrrrald 5d agoI have many issues with Anthropic, but I will say that their actions are fully consistent with a group of people who earnestly believe that AI is extremely dangerous In fact, I would say that the Occam’s razor explanation is not that they are seeking regulatory capture, but that they earnestly believe in the x-risk, and that they are the most thoughtful and capable people to address it You may disagree, you may think that they are delusional or have a God complex. Those are valid opinions. But I don’t believe that this is all a elaborate ruse for commercial gain.
- j-bos 5d agoWhy not both?
- deepwoods 5d agoI agree with this take. Regardless of whether you think they're right, most everyone at Anthropic is acting in good faith. If this is an intentional media campaign it is a remarkably sloppy one. It has been effective because if you tell people they're going to die, they tend to pay attention - think of the grip the 2011 Harold Camping rapture prediction had on our collective psyche, or the 2012 apocalypse. But the messaging is inconsistent, the target audience is unclear, the stated goals are muddy, the whole thing is packaged in dense SF-speak, it's just a mess from a comms perspective. That doesn't suggest to me that this is a concerted effort to enable regulatory capture. Perhaps there are some cynics among the executives and the investors who are happy it's happening to the extent it brings about regulatory capture, but nobody seems to be pulling the strings.
- ElProlactin 4d ago> Regardless of whether you think they're right, most everyone at Anthropic is acting in good faith. The road to hell is paved with good intentions. Dario Amodei is a 40-something dude who is obviously very, very intelligent. But intelligence is not wisdom and his "essay" here can easily be read as someone who opened a can of worms and doesn't know (or can't accept) that he won't be able to put the worms back in. But he's going to try because he believes he owes it to humanity to try. It's hubris in its most basic form, even if the guy who has it looks nice and wears shawl-collar sweaters.
- aubanel 4d agoYou can disagree with Anthropic leadership, but all the points you mentioned can reconcile very well with them thinking in good faith "advanced AI is too dangerous to be left in all hands" Except the "train on everyone else IP" which can be said of all AI companies.
- RobertDeNiro 4d agoProblem with Anthropic is that they benefited massively from open source and now are completely against it. It’s hypocrisy at its worst. Why do they get to benefit, but not everyone else?
- sigmarule 4d agoIf this is going to factor into my reasoning about the topic at hand, it will surely be a secondary or tertiary consideration at best. Same goes for much of the list in the root comment. We are discussing the possibility of severe adverse societal/world/human impacts from AI, why should open vs closed source/weights, copyright violations, anti-competitive moves, corporate hypocrisy, etc be so heavily weighted in the discussion? Hell, multiple things in OP’s list are quite literally moves a good-faith (wrt the topic at hand, genuine vs non-genuine concern) actor in such a position would do: closed weights means bad actors can’t abliterate, regulatory control, getting blacklisted by Hegseth’s DOW because they LIMITED acceptable use cases… I’m not pro- or anti-Anthropic at all, but it has started to annoy me how biased and shaky reasoning gets about this topic on here, it feels like people arguing to fit their existing opinions into today’s hubbubery rather than measured analysis of said hubbubery, which is why I come here. Question: consider the scenario where RSI is reached internally and internal tests struggle to align or control/contain the model; what are the differences in observables vs what we’re observing today?
- thayne 4d agoIMO, if the model is too dangerous for everyone to have access to, then no one should. I do not trust a for-profit company to be good stewards of such technology. Nor do I trust the government, any government.
- vlyan 4d agoI can't imagine a grown adult genuinely believing that an American company funded with over $100 billion of venture capital values the best interests of mankind over the best interests of its investors. while not everyone may recognize all that self-serving chutzpah as regulatory capture efforts, I think everyone can tell they're being bullshitted. some just pretend to suspend their disbelief when the blatant lies they're told align with the values they hold. just fucking imagine McDonalds running a public awareness campaign about the harms of fast food, urging the public and legislators to regulate the dangerously unsafe technology of combining carbs with grease, insisting that no one except Ronald McDonald himself can be trusted to steward it responsibly.
- sigmarule 4d agoThis thinking feels off for this context. First, there is a counterexample to your point, or a class of counterexamples: scenarios where the company believes the harm to mankind could propagate to harm their ability to sell to mankind. Which is interesting, of course, considering the “AI may destroy humanity” arguments this topic is centered on are prototypical members of this category. To be fair, one could argue this counterexample is not actually one, but rather a degenerate case where the good of the shareholders and humanity collapse and converge - in either case, though, the implication here is the same. It could certainly be the case they are playing these concerns up for more effective use as tactics in unrelated corporate strategies. But failing to consider the possibility of genuine concern being behind this behavior is childish.
- Avicebron 4d agoIf they wanted to be taken seriously about their desire to help humanity not just themselves they would be talking about universal healthcare, actionable plans for UBI (or whatever they think would be how people retain agency and dignity post-work), only building sustainable net-zero data centers... But no. I see this comment come up again and again and it's really the childish view to hold. Their proximity to wealth and potential future wealth is warping their thinking.
- alansaber 4d agoAnthropic's sense of ethics is completely aligned with its economic incentives. Shocking, right.
- sigmarule 4d ago> so controlling they are the only US company blacklisted by the US government Extremely bad example for the topic at hand.
- intunderflow 4d ago> so controlling they are the only US company blacklisted by the US government Very proud that they refused to sell AI for mass domestic surveillance and lethal autonomous weapons. People who criticise that in my view are showing their true colours.
- maciejzj 4d agoHad they issued an initiative on slowing down when they had the best model in the game, it would have been both more (anyhow?) effective and less dubious.
- Gigachad 4d ago> so controlling they are the only US company blacklisted by the US government Their control here was refusing to make fully automated killing machines. They simply required someone to have to press the button.
- deleted 4d ago[deleted]
- MrBuddyCasino 4d ago"The most common question people ask me is “Why do they do it?” That is a hard question to answer, but today I will answer it. [...] The first and most persistent oddity I ran into was the Effective Altruists’ willingness to lie, then admit to lying, then lie again." https://x.com/brianchau57/status/2098035812365463637 https://x.com/brianchau57/status/2098035812365463637 https://news.ycombinator.com/item?id=49648088 https://news.ycombinator.com/item?id=49648088
- mschuster91 4d ago> Altman gets so much hate but OpenAI has been a far better steward (on 3/5 above at least) than Anthropic! Altman single-handedly pulled off the most daring market manipulation heist in the last few decades [1]. Under any normal circumstances he'd have been arrested and prosecuted. The entire AI industry is filled to the brim with conmen, scammers and grifters, but Sam Altman is the most disgusting of all of them. [1] https://www.mooreslawisdead.com/post/sam-altman-s-dirty-dram-deal https://www.mooreslawisdead.com/post/sam-altman-s-dirty-dram...
- deleted 4d ago[deleted]
- Dlemlo 4d agoIn good faith? In what context? When? Who? Its a company, it sells a product, people use it.
- toss1 4d agoOn the topic of regulatory capture, there is this rumor that Google has achieved Recursive Self Improvement, and the other three (OpenAI/Anthropic/X) are just trying to get the regulators to slow down Google so they can catch up. Yes, what is in the labs is almost by definition ahead of what we can see, but I've come to take all of these announcements of impending doom as perverse marketing stunts — it is so powerful Disaster-XYZ is coming — so if it's that power you must invest... [0] https://wccftech.com/a-wild-rumor-says-google-is-on-the-verge-of-achieving-rsi-as-anthropic-openai-and-elon-musk-all-call-for-pacing-ai-in-what-might-be-a-cynical-attempt-at-regulatory-capture/ https://wccftech.com/a-wild-rumor-says-google-is-on-the-verg...