8 ms·

David Sacks: OpenAI and Anthropic Don't Need Regulations to Pace Frontier Models

330 points by kolanos 4d ago

chinathrow 4d ago
You know what rubs me is that in Darios blog post he talks about "abundance", but what he really means is that Anthropic will bath in endless pools of hard curreny while the rest of the world will try to stay just barely afloat/alive.
pembrook 4d ago
I don't agree with Sacks on a lot but he's 100% correct here. I heavily use frontier models daily, and quite frankly their capabilities are just not very relevant outside of a narrow subset of software engineering tasks and the production of generic white collar "deliverables." We're many years into $10s of billions of dollars being thrown at coding as a problem space specifically, and its one of the areas with the MOST training data available, and still...I can't get a frontier model to execute simple front-end UI tasks beyond the level of a visually autistic intern. Fable and Astra-class models were the first ones trained on Blackwell GPUs (the latest and greatest) and from what I've heard from folks on the inside, the expectation was that throwing this much extra compute at the problem would lead to greater gains. It has not. And the next gen GPUs are not going to be the same jump that h100 clusters to Blackwell was. I think a lot of Anthropic/OpenAI employees are true believers here (who doesn't want to imagine they're having a large impact?) and have deluded themselves into believing they're building a god in their science fiction fantasy world. Unfortunately the type of people working at these orgs don't really have much understanding of how the world works outside of their extremely narrow specialization. When I hear them throwing out silly statistics like "8% GDP growth," "50% of white collar workers unemployed by next year," and "10% chance of human annihilation," I cringe so hard my eyebrows hit my chin. Here's a real statistic: 82% of german companies still use fax machines. I seem to recall Waymo told a 60 minutes reporter during an early report on self driving cars: "your daughter will never need to learn to drive." Well, over a decade has passed since then, and we're still without driverless cars in 99% of places.
levocardia 4d ago
Is it 100% AI-generated? checks Pangram yep, it is. NEXT!
yawnxyz 4d ago
If they want something that looks more like FDA sanction clinical trial proceedings for these products, then what they have to also realize is that they will need a third party to audit every single thing for every single product release and you get something like ERoom's law where every success and model gets more and more expensive release and probably billions of dollars in multiple years release so I don't know if this actually helps them
deagle50 4d ago
I'll go a step further, they aren't just pretending, they are blackmailing the country. The sane response would be "we don't negotiate with terrorists, and you will face criminal and financial penalties for damages caused."
- matheusmoreira 4d ago
  Couldn't agree more. Better yet: the AI labs get straight up nationalized.
  - gr_norm 4d ago
    No. That kills competition. I want everyone to be competing on the open playing field they are right now, with losses socialized as minimally as possible. There's nothing fundamentally special about LLM companies that would warrant nationalizing them.
  - underlipton 4d ago
    I'll do you one better: each state gets a direct share. So, not nationalized in the sense of getting put behind a faceless bureaucracy forever; states collectively have control and benefit directly from success.
- preommr 4d ago
  > and you will face criminal and financial penalties for damages caused. I don't understand why this isn't talked about more. We don't need a slowdown. Just double down on prosecuting crimes. Let the companies take the risk. If they feel confident and they're right they win market share. If they don't feel confident, they can not risk it and slow down. If they feel confident and they get it wrong, they should be sued to kingdom come. That alone should disincentivize reckleses behavior like letting models run crazy with large amounts of compute which leads to them escaping sandboxes. This performative "our internal modles are so crazy powerful" song and dance is getting old, especially with no real concrete explanations.
  - deagle50 4d ago
nilkn 4d ago
Once they observe real safety risks, they're going to slow down on their own, because a major safety incident would destroy their valuations and businesses. The beauty of this principle is that it applies to everyone all over the world. Nobody is exempt from it. There's no coordination required. No country would benefit from developing an AI system that wreaks havoc on its own infrastructure, so there's a powerful built-in global incentive to build safe, steerable systems. Right now the risks are hypothetical. That doesn't mean we shouldn't take them seriously, but crippling an industry through government intervention before we have any real understanding of it is clearly unnecessary. The fearmongering is clouding basic judgment and thinking. It's possible to take the risks seriously without immediately introducing poorly understood regulation which will have its own dangerous side effects. OpenAI and Anthropic are doing extraordinary amounts of alignment testing internally. There is no scenario where they release a dangerous model without realizing what they are doing. The only thing the government needs to do is hold AI companies legally accountable if their products cause harm. That's it. That's not what the labs want. They don't want to be held legally accountable for harm, and they do want there to be a huge compliance burden that only they can afford to pass.
jacobgold 4d ago
Make it easy for these companies by passing a law that any AI model a company offers to the public (non-government) has to be released as open weights. This is how we deflate the bubble safely and completely, without having to introduce a government regulator that is likely to go either too far or not far enough. This resolves the coordination problem among these potentially good actors.
- 0xDEAFBEAD 4d ago
  If Astra is open-weight, black hats can download it, remove guardrails, and go on an automated hacking spree.
  - jacobgold 4d ago
    Just like they're already doing with frontier-level open weight models? Do you imagine that Russian and Chinese intelligence don't have copies of the Mythos/Fable weights? The point is to slow progress of future models. Nothing can stop what has already been created.
- blululu 4d ago
  What? This is a terrible policy. In addition to the obvious AI safety risks, this just amounts to a free R&D subsidy to Amazon for hosting models. It would remove any incentive to do any R&D without AWS level physical infrastructure.
  - jacobgold 4d ago
    That's an understandable first thought, but consider what the goal is for this: slowing down future development. This can be achieved by dramatically reducing the market value of models.
jstummbillig 4d ago
Like it! Yeah, they can just take action and pressing them on that is fine. It's not like they asked for policy before they started to mine all the data and sell the resulting product, while, surely, at some point, there must have been the thought that the entire enterprise could be considered vaguely problematic. To be fair 1) I feel like Sacks is kind of misrepresenting what is going on – at least according to the stories that the two labs tell, they are already taking it upon themselves to act. 2) Understanding if they should do it is more complicated; I don't think anyone can satisfactorily answer that one, so, as a matter of judgement, it seems like a choice to make and they should just consider this option. At the same time, I don't think that requires the labs to not press for policy changes. Again: Claiming there is an obvious way to do that, that is both realistic and correct seems a little far fetched, but arguably one of the more important and pressing questions of our times to get a hold of.
Meneth 4d ago
They might need regulations, if they are for-profit companies with a responsibility to shareholders to maximize profit while sacrificing all other values.
- the_optimist 4d ago
  It is telling that most aggressive forms of “capitalism” plead for government involvement to permanently limit new competition. I own something and I don’t want anyone else to have a chance. The marginal cost of bribing a senator is tiny tiny tiny compared to the value.
  - matheusmoreira 4d ago
    I suppose that's to be expected from sociopathic corporations trying to maximize their own self interest. The real problem is the governments acquiesce instead of putting them in their place.
- matwood 4d ago
  > responsibility to shareholders to maximize profit while sacrificing all other values This is not required to meet fiduciary duty, although it is a common misconception. The company officers and board have pretty broad leeway to run the company as they see fit as long as there is no fraud, illegal activity, or conflict of interests. https://www.nytimes.com/roomfordebate/2015/04/16/what-are-corporations-obligations-to-shareholders/corporations-dont-have-to-maximize-profits https://www.nytimes.com/roomfordebate/2015/04/16/what-are-co...
- PlasmaPower 4d ago
  They are not. This is why OpenAI and Anthropic are Public Benefit Corporations, which gives them the ability to prioritize the public over profit. The issue is that an agreement between them to mutually slow down is an antitrust concern.
hrpnk 4d ago
I wished OpenAI and Anthropic IPOd already. They just try to outcompete each other by finding the most splashy headlines undercutting prices in unhealthy and unsustainable means long-term.
simianwords 4d ago
(100% AI generated tweet btw) My take is that there's a real impending doom with real reason to coordinate a slow down: I'd put this at 60%. Reasons? All smart people are on this position. Dario has been saying this since 2015 or so. Paul Christiano as well. Hell even Elon Musk. I think its highly unlikely that you would base your entire world-view on a certain thing and then actually act on it when the time comes for a totally different reason (like regulatory capture). The other 15% is that P(doom) and "slow down" are nice shibboleths in the internal EA or AI safety community. You get to be in the community and show commitment to it by taking costly actions that signal that you actually do wanna slow down. For employees it is quitting. For CEO's it is writing these essays. I know this is ridiculous but sometimes things do just come down to this. In 20 years would you look at all of this and think it was not a moral panic and that the slow-down was necessary? The rest 25% is regulatory capture of which people know exactly the reasons
- deleted 4d ago
  [deleted]
dwohnitmok 4d ago
Eh. This entire tweet smacks of being AI generated. It has a huge amount of LLM-isms. I wouldn't be surprised if Sacks just prompted an LLM to just come up with whatever rebuttal to whatever the regulation side comes up (given the big set of regulation tweets) with sounds most convincing, given his usual anti-regulation stance. Anyone with a Pangram account who can check?
- samename 4d ago
  Pangram reports 100% AI generated https://www.pangram.com/history/410ce80f-bd0e-45b6-a82c-d6ed346418b7?ucc=mWCRrS9gkIu https://www.pangram.com/history/410ce80f-bd0e-45b6-a82c-d6ed...
agd 4d ago
Huge amount of skepticism on HN right now, but the models are increasingly capable, and there are real, significant risks. The approach proposed by Amodei looks sensible - voluntary checks at frontier labs, and appropriate regulation to follow. Yes, the devil's in the details, but it feels like the right approach overall.
- jacobgold 4d ago
  > The approach proposed by Amodei looks sensible... It's hard to imagine any government regulation that didn't look sensible on paper and at first. But we know that government regulation has an extremely mixed record, with lots of good and bad outcomes.
- vrganj 4d ago
  Turns out the EU was right all along with the AI act. They've been adopting a well-deserved "told you so" stance lately: > "In my view, the EU has already legislated for exactly this scenario: it is called the AI Act," said Michael McNamara, an Irish European Parliament lawmaker, who's the co-lead of the Parliament's group monitoring AI. https://www.politico.eu/article/eu-response-ai-extinction-warnings/ https://www.politico.eu/article/eu-response-ai-extinction-wa...
  - the_optimist 4d ago
    The EU is not leading anything. They may as well just commit to becoming a subsistence agricultural society with EPCOT heirlooms on display.
    - vrganj 4d ago
      If the rest of the world follows them on regulations they've thought through before anyone else, then I don't know what you would call that other than leading? Like, definitionally that's what leading is. It's interesting you're so derisive of thought leadership.
      - nsdkdkdkfk 4d ago
daveguy 4d ago
But.... Money.
throwaway63467 4d ago
I guess what they really want is to limit sale of AI models to compliant vendors and then raise the bar to compliance just high enough so they can pass it but smaller labs can’t. There’s no moat, it’s an efficient market that drives margins to zero right now, of course they don’t want that, collusion of the big vendors is the next logical step.
- dozerly 4d ago
  This could be why they’re going so hard on hacking everyone and everything. You make it clear AI is a threat, get it locked down, and then have the capital to push through the lockdown while others are stuck.
  - 8cvor6j844qw_d6 4d ago
    Same thoughts. Also, some of the recent coverage around AI hacks also feels unusually amplified and repetitive.
  - 3eb7988a1663 4d ago
    The Houthis missile story coming out now also feels like convenient fuel for the fire.
- Joel_Mckay 4d ago
  That is generally how regulatory-capture works. As silly as I personally think LLM hype is, the "AI" business posture choices with several Trillion dollars in debt on the books has very few options after hyper-scaling/sandbagging loses traction. =3 https://en.wikipedia.org/wiki/Regulatory_capture https://en.wikipedia.org/wiki/Regulatory_capture
- dannykwells 4d ago
  Bingo. The goal is positioning a duopoly / cartel as the only “safe” and “moral” path and then that cartel sets pace and price.
- hn_throwaway_99 4d ago
  > There’s no moat People keep saying this, and while it may be partially true, it misses a very important detail. Right now, overall compute is a moat. Someone even commented on another of my comments that one reason Google is lagging is they don't have the same level of Nvidia farms as OpenAI and Anthropic. A big reason that OpenAI and Anthropic are racing so fast is they both want to get to recursive self improvement (remains to be seen if that is actually possible, but AFAICT most people at these companies genuinely believe it is) before anyone else, because they believe whoever gets there first will then have an insurmountable lead. But I think they also clearly understand that neither of them have solved for alignment (and in fact they are further from it), and RSI with misaligned models, where interpretability is worse every 6 months, is incredibly dangerous. I think the concerns about regulatory capture are warranted, but I see so many comments parroting the "evil Anthropic and OpenAI" viewpoints that they are missing some of the real, valid concerns and dangers. I think this tweet by David Kokotajlo makes some good points on how to tell if regulations are being "cheated" for the purposes of regulatory capture or if they really are actually pacing the frontier: https://x.com/DKokotajlo/status/2099185129533186438 https://x.com/DKokotajlo/status/2099185129533186438
vatsachak 4d ago
The first time I have agreed with David Sacks
- toss1 4d ago
  Yup. Sacks has been so catastrophically wrong about everything else (especially geopolitics), this prompts me to reconsider my opinion on this.
  - pembrook 4d ago
    To be fair, his takes on the Ukraine war from the covid days have aged very well, and were contrarian to mainstream narratives at the time.
    - okdood64 4d ago
      > his takes on the Ukraine war What's specifically?
biophysboy 4d ago
David Sacks is one of the most predictable VCs in Silicon Valley. He could have written "I still have the same thoughts; here's a link to my first and only opinion" and the post would have communicated the same amount of info.
nba456_ 4d ago
He's right that if all the frontier labs are in agreement that they should slow down, they don't need the government to do that.
- SpicyLemonZest 4d ago
  No, that’s not accurate. An agreement among competitors to not engage in research that would let them outcompete each other is clearly an antitrust issue requiring a government waiver.
  - jubilanti 4d ago
    No, absolutely not "clearly". That word has no business being in any sentence about federal antitrust law in 2026. There is nothing clear or consistent about how is being applied and interpreted right now. If you're applying a coder's mindset to 'settled' law, you're in for a bad time.
    - AndyNemmity 4d ago
      I mean, they said it was clearly an anti trust issue. It is. It is also true that nothing in law is currently clear, or consistent. Both are true. But you stated it's one or the other.
    - SpicyLemonZest 4d ago
      If that’s true, doesn’t that make it even more clear that the frontier labs require government involvement and cannot rely on what David Sacks thinks the government would or would not allow?
- esafak 4d ago
  And kids don't need parents to eat their vegetables and do their homework.
  - layer8 4d ago
    …if all kids are in agreement that they should do just that.
elicash 4d ago
David Sachs wants to use government to punish people he disagrees with politically, like Dario. He supported the Department of War's illegal actions against Anthropic. He supported the Trump administration's temporary export controls against Anthropic, the first government-imposed pause. He has said Anthropic has created a monopoly, implying that they should be broken up. I don't take him seriously when he says he doesn't favor government action. He favors it when he doesn't like their speech.
- kypro 4d ago
  You're being downvoted because you clearly have the wrong opinion. If you want upvotes here, please share a conspiracy theory about why the CEOs of AI companies might want to slow down. It doesn't need to based in reality or even reasonable.
- tjwebbnorfolk 4d ago
  poor Anthropic
  - Zenul_Abidin 4d ago
    Will someone please think about Dario?
  - elicash 4d ago
    A principled individual could dislike Anthropic and still not favor using government to punish speech.
    - Analemma_ 4d ago
      Well you’re in the wrong place for that. Hacker News is mostly about showing how much smarter and more cynical you are than everyone else in the room, and attacking anyone who does actually have principles by accusing them of ulterior motives to justify your own inaction.
      - tjwebbnorfolk 4d ago
Insanity 4d ago
They’ll slow down not because they “are worried” but because their improvements are stagnating as they hit financial and technical challenges.
- kypro 4d ago
  > because their improvements are stagnating as they hit financial and technical challenges. I don't mean to be rude, but how is it even possible for someone to hold this opinion in mid-2026? Can you explain why you believe this?
  - dumberquestions 4d ago
    I don't think the technical aspect is true, performance has been growing fairly consistently, but the financial aspect, or more accurately the logistical aspect, is plausible; with competition as fierce as it currently is, for how long can they keep growing at the same pace? People can't afford RAM anymore and datacenter build outs aren't particularly popular, OAI has been doubling compute capacity every 7 months for the past 3 years, it's just physically impossible for this to go on for long, something has to give. And this is not to say that the tech isn't revolutionary or the demand isn't there, it's simply just currently too competitive, and to avoid getting into anti trust trouble they need to involve the government. Note this being a plausible doesn't necessarily make it true, for all I know they could have witnessed some safety incident and decided they need a pause, or maybe it's both reasons.
    - simianwords 4d ago
      I just find it strange how multiple people throw completely different arguments just to land on same conclusion. Your argument seems to mix a few arguments like: OpenAI has so much demand and it has made so much compute that it is a bad thing (?!). It can't go on (why?) hence the capabilities are stalling (how?). Then you also say that it is too competitive which is a totally different argument to compute. So in this thread we have multiple vectors of arguments like - high competition - high demand for compute (not sure how this hurts OpenAI but whatever) - capability has stalled
socalgal2 4d ago
Admits China won’t stop but says USA’s top should. Reminds me of City in the Edge of Forever
- mrweasel 4d ago
  Unsure about your reference, but is your argument: Someone else is doing it, so I should be allowed to as well. That's basically the western companies complaining that they can't compete with China because Chinese companies are allowed to polute more, so they would be allowed to polute as well. I don't buy that. Once you realise that you're doing something wrong or dangerous, you stop, regardless of what everyone else is doing and what rules they have to follow.
andsoitis 4d ago
[dead]
nialv7 4d ago
He is neither right or wrong. The reality is that this is a lose-lose situation from the ordinary person's perspective. Regulation -> regulatory capture and things get worse faster, no regulation -> things simply get worse faster. I mean I don't know what we do at this point. Ideally AI researchers need to be treated like nuclear scientists working for an adversary. But we all know that's not going to happen.
- reasonableklout 4d ago
  Maybe organize politically? The labs are still staffed by people, and nonviolent demonstrations outside their offices can still reach them.
torginus 4d ago
I think this might have to do with broader market conditions. It seems that the Fed is likely to raise bond rates, which would make borrowing more expensive and slow investments (as well as devaluing any potential IPO) on its own. A rate hike would also mean people getting fired, which would not make companies offering labour replacement very popular in the eyes of the public. Don't want to get political, but there's a real chance that the current admin will perform badly on the midterms, which would mean Democrats would get more leverage to stop funding or block the already unpopular and very expensive datacenter buildouts. PS: Please don't get mad at me, I'm neither a financial expert neither did I mean to express any political leanings.
deleted 4d ago
[deleted]
Kiro 4d ago
These threads are always filled with different versions of the same comment. Normally some reddit style reductive platitude that we've heard a billion times already.
swingboy 4d ago
There’s a difference between a model recursively improving “itself” and improving itself via online learning, right? The former being that these models are helping develop and train future models, but they might not veer too far off in architecture (yet). The latter being the same model being able to train/learn on the fly, in real time, permanently (not just in the current conversation/session), or in other words, adjusting/managing its own weights. The latter seems far more likely to go out of control than the former. But, it also seems like it would take an entire paradigm shift in model architecture, but I could be wrong. Does anyone in the industry think any of these companies are actually close to that kind of self-improvement?
- erichocean 4d ago
  > Does anyone in the industry think any of these companies are actually close to that kind of self-improvement? Their current plan is to take the existing architecture and shorten the cycle times: move all new RLVR work into mid-training on a pre-existing base; apply new RLVR. Rinse and repeat. If you did that daily, it would be roughly similar to how humans improve.
lrvick 4d ago
Maybe everyone forgets how PCI standards played out. They are an example of an industry self regulating, because the regulators have no idea how technology works. Granted, PCI is so weak it is almost useless, and yet still better than anything congress could have come up with. Where the government might have to step in, is by having a kill switch to cut off internet access from countries that fail to agree to common sense quarantines. We can do mutual remote attestation of labs across the world to ensure every big hot thermally-visable cluster of AI GPUs on the planet are accounted for and running secure enclaves and common sense isolation, along the lines of how we manage nukes. The problem there is I just said too many technical words that seemingly not even the frontier labs understand, as evidenced by all the escapes.
- okdood64 4d ago
  > Granted, PCI is so weak it is almost useless, and yet still better than anything congress could have come up with. Elaborate, please?
alexmercerdev 4d ago
[flagged]
Digory 4d ago
They seem to be okay with an American slowdown, even though the Chinese won’t slow down. So maybe they want something the Chinese already have: liability protection. The models spontaneously hack everything important without shame. The frontier labs are going to get enjoined and regulated twelve ways to Sunday if the Feds don’t socialize the costs.
- d0mine 4d ago
  I've only heard of OpenAI and Anthropic agents hacking during training. Are open weights model not smart enough to hack Hugging Face to copy solutions? (doubtful)
cc62cf4a4f20 4d ago
Could this be that the AI companies are seeing progress slow down and need to seed a narrative to explain it? Not clear from outside. Could it be to slow down burn before an IPO to juice profitability projections? More plausible. Kind of hard to take the frontier labs at their word.
paoliniluis 4d ago
Seems that Sacks didn’t get that the message was targeted to the president to ask for a bilateral agreement with China, that’s the end game

330 points by kolanos 4d ago

You know what rubs me is that in Darios blog post he talks about "abundance", but what he really means is that Anthropic will bath in endless pools of hard curreny while the rest of the world will try to stay just barely afloat/alive.

pembrook 4d ago

I don't agree with Sacks on a lot but he's 100% correct here. I heavily use frontier models daily, and quite frankly their capabilities are just not very relevant outside of a narrow subset of software engineering tasks and the production of generic white collar "deliverables." We're many years into $10s of billions of dollars being thrown at coding as a problem space specifically, and its one of the areas with the MOST training data available, and still...I can't get a frontier model to execute simple front-end UI tasks beyond the level of a visually autistic intern. Fable and Astra-class models were the first ones trained on Blackwell GPUs (the latest and greatest) and from what I've heard from folks on the inside, the expectation was that throwing this much extra compute at the problem would lead to greater gains. It has not. And the next gen GPUs are not going to be the same jump that h100 clusters to Blackwell was. I think a lot of Anthropic/OpenAI employees are true believers here (who doesn't want to imagine they're having a large impact?) and have deluded themselves into believing they're building a god in their science fiction fantasy world. Unfortunately the type of people working at these orgs don't really have much understanding of how the world works outside of their extremely narrow specialization. When I hear them throwing out silly statistics like "8% GDP growth," "50% of white collar workers unemployed by next year," and "10% chance of human annihilation," I cringe so hard my eyebrows hit my chin. Here's a real statistic: 82% of german companies still use fax machines. I seem to recall Waymo told a 60 minutes reporter during an early report on self driving cars: "your daughter will never need to learn to drive." Well, over a decade has passed since then, and we're still without driverless cars in 99% of places.

levocardia 4d ago

Is it 100% AI-generated? checks Pangram yep, it is. NEXT!

yawnxyz 4d ago

If they want something that looks more like FDA sanction clinical trial proceedings for these products, then what they have to also realize is that they will need a third party to audit every single thing for every single product release and you get something like ERoom's law where every success and model gets more and more expensive release and probably billions of dollars in multiple years release so I don't know if this actually helps them

deagle50 4d ago

I'll go a step further, they aren't just pretending, they are blackmailing the country. The sane response would be "we don't negotiate with terrorists, and you will face criminal and financial penalties for damages caused."

matheusmoreira 4d ago
Couldn't agree more. Better yet: the AI labs get straight up nationalized.
- gr_norm 4d ago
  No. That kills competition. I want everyone to be competing on the open playing field they are right now, with losses socialized as minimally as possible. There's nothing fundamentally special about LLM companies that would warrant nationalizing them.
- underlipton 4d ago
  I'll do you one better: each state gets a direct share. So, not nationalized in the sense of getting put behind a faceless bureaucracy forever; states collectively have control and benefit directly from success.
preommr 4d ago
> and you will face criminal and financial penalties for damages caused. I don't understand why this isn't talked about more. We don't need a slowdown. Just double down on prosecuting crimes. Let the companies take the risk. If they feel confident and they're right they win market share. If they don't feel confident, they can not risk it and slow down. If they feel confident and they get it wrong, they should be sued to kingdom come. That alone should disincentivize reckleses behavior like letting models run crazy with large amounts of compute which leads to them escaping sandboxes. This performative "our internal modles are so crazy powerful" song and dance is getting old, especially with no real concrete explanations.
- deagle50 4d ago

nilkn 4d ago

Once they observe real safety risks, they're going to slow down on their own, because a major safety incident would destroy their valuations and businesses. The beauty of this principle is that it applies to everyone all over the world. Nobody is exempt from it. There's no coordination required. No country would benefit from developing an AI system that wreaks havoc on its own infrastructure, so there's a powerful built-in global incentive to build safe, steerable systems. Right now the risks are hypothetical. That doesn't mean we shouldn't take them seriously, but crippling an industry through government intervention before we have any real understanding of it is clearly unnecessary. The fearmongering is clouding basic judgment and thinking. It's possible to take the risks seriously without immediately introducing poorly understood regulation which will have its own dangerous side effects. OpenAI and Anthropic are doing extraordinary amounts of alignment testing internally. There is no scenario where they release a dangerous model without realizing what they are doing. The only thing the government needs to do is hold AI companies legally accountable if their products cause harm. That's it. That's not what the labs want. They don't want to be held legally accountable for harm, and they do want there to be a huge compliance burden that only they can afford to pass.

jacobgold 4d ago

Make it easy for these companies by passing a law that any AI model a company offers to the public (non-government) has to be released as open weights. This is how we deflate the bubble safely and completely, without having to introduce a government regulator that is likely to go either too far or not far enough. This resolves the coordination problem among these potentially good actors.

0xDEAFBEAD 4d ago
If Astra is open-weight, black hats can download it, remove guardrails, and go on an automated hacking spree.
- jacobgold 4d ago
  Just like they're already doing with frontier-level open weight models? Do you imagine that Russian and Chinese intelligence don't have copies of the Mythos/Fable weights? The point is to slow progress of future models. Nothing can stop what has already been created.
blululu 4d ago
What? This is a terrible policy. In addition to the obvious AI safety risks, this just amounts to a free R&D subsidy to Amazon for hosting models. It would remove any incentive to do any R&D without AWS level physical infrastructure.
- jacobgold 4d ago
  That's an understandable first thought, but consider what the goal is for this: slowing down future development. This can be achieved by dramatically reducing the market value of models.

jstummbillig 4d ago

Like it! Yeah, they can just take action and pressing them on that is fine. It's not like they asked for policy before they started to mine all the data and sell the resulting product, while, surely, at some point, there must have been the thought that the entire enterprise could be considered vaguely problematic. To be fair 1) I feel like Sacks is kind of misrepresenting what is going on – at least according to the stories that the two labs tell, they are already taking it upon themselves to act. 2) Understanding if they should do it is more complicated; I don't think anyone can satisfactorily answer that one, so, as a matter of judgement, it seems like a choice to make and they should just consider this option. At the same time, I don't think that requires the labs to not press for policy changes. Again: Claiming there is an obvious way to do that, that is both realistic and correct seems a little far fetched, but arguably one of the more important and pressing questions of our times to get a hold of.

Meneth 4d ago

They might need regulations, if they are for-profit companies with a responsibility to shareholders to maximize profit while sacrificing all other values.

the_optimist 4d ago
It is telling that most aggressive forms of “capitalism” plead for government involvement to permanently limit new competition. I own something and I don’t want anyone else to have a chance. The marginal cost of bribing a senator is tiny tiny tiny compared to the value.
- matheusmoreira 4d ago
  I suppose that's to be expected from sociopathic corporations trying to maximize their own self interest. The real problem is the governments acquiesce instead of putting them in their place.
matwood 4d ago
> responsibility to shareholders to maximize profit while sacrificing all other values This is not required to meet fiduciary duty, although it is a common misconception. The company officers and board have pretty broad leeway to run the company as they see fit as long as there is no fraud, illegal activity, or conflict of interests. https://www.nytimes.com/roomfordebate/2015/04/16/what-are-corporations-obligations-to-shareholders/corporations-dont-have-to-maximize-profits https://www.nytimes.com/roomfordebate/2015/04/16/what-are-co...
PlasmaPower 4d ago
They are not. This is why OpenAI and Anthropic are Public Benefit Corporations, which gives them the ability to prioritize the public over profit. The issue is that an agreement between them to mutually slow down is an antitrust concern.

hrpnk 4d ago

I wished OpenAI and Anthropic IPOd already. They just try to outcompete each other by finding the most splashy headlines undercutting prices in unhealthy and unsustainable means long-term.

simianwords 4d ago

(100% AI generated tweet btw) My take is that there's a real impending doom with real reason to coordinate a slow down: I'd put this at 60%. Reasons? All smart people are on this position. Dario has been saying this since 2015 or so. Paul Christiano as well. Hell even Elon Musk. I think its highly unlikely that you would base your entire world-view on a certain thing and then actually act on it when the time comes for a totally different reason (like regulatory capture). The other 15% is that P(doom) and "slow down" are nice shibboleths in the internal EA or AI safety community. You get to be in the community and show commitment to it by taking costly actions that signal that you actually do wanna slow down. For employees it is quitting. For CEO's it is writing these essays. I know this is ridiculous but sometimes things do just come down to this. In 20 years would you look at all of this and think it was not a moral panic and that the slow-down was necessary? The rest 25% is regulatory capture of which people know exactly the reasons

deleted 4d ago
[deleted]

dwohnitmok 4d ago

Eh. This entire tweet smacks of being AI generated. It has a huge amount of LLM-isms. I wouldn't be surprised if Sacks just prompted an LLM to just come up with whatever rebuttal to whatever the regulation side comes up (given the big set of regulation tweets) with sounds most convincing, given his usual anti-regulation stance. Anyone with a Pangram account who can check?

samename 4d ago
Pangram reports 100% AI generated https://www.pangram.com/history/410ce80f-bd0e-45b6-a82c-d6ed346418b7?ucc=mWCRrS9gkIu https://www.pangram.com/history/410ce80f-bd0e-45b6-a82c-d6ed...

agd 4d ago

Huge amount of skepticism on HN right now, but the models are increasingly capable, and there are real, significant risks. The approach proposed by Amodei looks sensible - voluntary checks at frontier labs, and appropriate regulation to follow. Yes, the devil's in the details, but it feels like the right approach overall.

jacobgold 4d ago
> The approach proposed by Amodei looks sensible... It's hard to imagine any government regulation that didn't look sensible on paper and at first. But we know that government regulation has an extremely mixed record, with lots of good and bad outcomes.
vrganj 4d ago
Turns out the EU was right all along with the AI act. They've been adopting a well-deserved "told you so" stance lately: > "In my view, the EU has already legislated for exactly this scenario: it is called the AI Act," said Michael McNamara, an Irish European Parliament lawmaker, who's the co-lead of the Parliament's group monitoring AI. https://www.politico.eu/article/eu-response-ai-extinction-warnings/ https://www.politico.eu/article/eu-response-ai-extinction-wa...
- the_optimist 4d ago
  The EU is not leading anything. They may as well just commit to becoming a subsistence agricultural society with EPCOT heirlooms on display.
  - vrganj 4d ago
    If the rest of the world follows them on regulations they've thought through before anyone else, then I don't know what you would call that other than leading? Like, definitionally that's what leading is. It's interesting you're so derisive of thought leadership.
    - nsdkdkdkfk 4d ago

daveguy 4d ago

But.... Money.

throwaway63467 4d ago

I guess what they really want is to limit sale of AI models to compliant vendors and then raise the bar to compliance just high enough so they can pass it but smaller labs can’t. There’s no moat, it’s an efficient market that drives margins to zero right now, of course they don’t want that, collusion of the big vendors is the next logical step.

dozerly 4d ago
This could be why they’re going so hard on hacking everyone and everything. You make it clear AI is a threat, get it locked down, and then have the capital to push through the lockdown while others are stuck.
- 8cvor6j844qw_d6 4d ago
  Same thoughts. Also, some of the recent coverage around AI hacks also feels unusually amplified and repetitive.
- 3eb7988a1663 4d ago
  The Houthis missile story coming out now also feels like convenient fuel for the fire.
Joel_Mckay 4d ago
That is generally how regulatory-capture works. As silly as I personally think LLM hype is, the "AI" business posture choices with several Trillion dollars in debt on the books has very few options after hyper-scaling/sandbagging loses traction. =3 https://en.wikipedia.org/wiki/Regulatory_capture https://en.wikipedia.org/wiki/Regulatory_capture
dannykwells 4d ago
Bingo. The goal is positioning a duopoly / cartel as the only “safe” and “moral” path and then that cartel sets pace and price.
hn_throwaway_99 4d ago
> There’s no moat People keep saying this, and while it may be partially true, it misses a very important detail. Right now, overall compute is a moat. Someone even commented on another of my comments that one reason Google is lagging is they don't have the same level of Nvidia farms as OpenAI and Anthropic. A big reason that OpenAI and Anthropic are racing so fast is they both want to get to recursive self improvement (remains to be seen if that is actually possible, but AFAICT most people at these companies genuinely believe it is) before anyone else, because they believe whoever gets there first will then have an insurmountable lead. But I think they also clearly understand that neither of them have solved for alignment (and in fact they are further from it), and RSI with misaligned models, where interpretability is worse every 6 months, is incredibly dangerous. I think the concerns about regulatory capture are warranted, but I see so many comments parroting the "evil Anthropic and OpenAI" viewpoints that they are missing some of the real, valid concerns and dangers. I think this tweet by David Kokotajlo makes some good points on how to tell if regulations are being "cheated" for the purposes of regulatory capture or if they really are actually pacing the frontier: https://x.com/DKokotajlo/status/2099185129533186438 https://x.com/DKokotajlo/status/2099185129533186438

vatsachak 4d ago

The first time I have agreed with David Sacks

toss1 4d ago
Yup. Sacks has been so catastrophically wrong about everything else (especially geopolitics), this prompts me to reconsider my opinion on this.
- pembrook 4d ago
  To be fair, his takes on the Ukraine war from the covid days have aged very well, and were contrarian to mainstream narratives at the time.
  - okdood64 4d ago
    > his takes on the Ukraine war What's specifically?

biophysboy 4d ago

David Sacks is one of the most predictable VCs in Silicon Valley. He could have written "I still have the same thoughts; here's a link to my first and only opinion" and the post would have communicated the same amount of info.

nba456_ 4d ago

He's right that if all the frontier labs are in agreement that they should slow down, they don't need the government to do that.

SpicyLemonZest 4d ago
No, that’s not accurate. An agreement among competitors to not engage in research that would let them outcompete each other is clearly an antitrust issue requiring a government waiver.
- jubilanti 4d ago
  No, absolutely not "clearly". That word has no business being in any sentence about federal antitrust law in 2026. There is nothing clear or consistent about how is being applied and interpreted right now. If you're applying a coder's mindset to 'settled' law, you're in for a bad time.
  - AndyNemmity 4d ago
    I mean, they said it was clearly an anti trust issue. It is. It is also true that nothing in law is currently clear, or consistent. Both are true. But you stated it's one or the other.
  - SpicyLemonZest 4d ago
    If that’s true, doesn’t that make it even more clear that the frontier labs require government involvement and cannot rely on what David Sacks thinks the government would or would not allow?
esafak 4d ago
And kids don't need parents to eat their vegetables and do their homework.
- layer8 4d ago
  …if all kids are in agreement that they should do just that.

elicash 4d ago

David Sachs wants to use government to punish people he disagrees with politically, like Dario. He supported the Department of War's illegal actions against Anthropic. He supported the Trump administration's temporary export controls against Anthropic, the first government-imposed pause. He has said Anthropic has created a monopoly, implying that they should be broken up. I don't take him seriously when he says he doesn't favor government action. He favors it when he doesn't like their speech.

kypro 4d ago
You're being downvoted because you clearly have the wrong opinion. If you want upvotes here, please share a conspiracy theory about why the CEOs of AI companies might want to slow down. It doesn't need to based in reality or even reasonable.
tjwebbnorfolk 4d ago
poor Anthropic
- Zenul_Abidin 4d ago
  Will someone please think about Dario?
- elicash 4d ago
  A principled individual could dislike Anthropic and still not favor using government to punish speech.
  - Analemma_ 4d ago
    Well you’re in the wrong place for that. Hacker News is mostly about showing how much smarter and more cynical you are than everyone else in the room, and attacking anyone who does actually have principles by accusing them of ulterior motives to justify your own inaction.
    - tjwebbnorfolk 4d ago

Insanity 4d ago

They’ll slow down not because they “are worried” but because their improvements are stagnating as they hit financial and technical challenges.

kypro 4d ago
> because their improvements are stagnating as they hit financial and technical challenges. I don't mean to be rude, but how is it even possible for someone to hold this opinion in mid-2026? Can you explain why you believe this?
- dumberquestions 4d ago
  I don't think the technical aspect is true, performance has been growing fairly consistently, but the financial aspect, or more accurately the logistical aspect, is plausible; with competition as fierce as it currently is, for how long can they keep growing at the same pace? People can't afford RAM anymore and datacenter build outs aren't particularly popular, OAI has been doubling compute capacity every 7 months for the past 3 years, it's just physically impossible for this to go on for long, something has to give. And this is not to say that the tech isn't revolutionary or the demand isn't there, it's simply just currently too competitive, and to avoid getting into anti trust trouble they need to involve the government. Note this being a plausible doesn't necessarily make it true, for all I know they could have witnessed some safety incident and decided they need a pause, or maybe it's both reasons.
  - simianwords 4d ago
    I just find it strange how multiple people throw completely different arguments just to land on same conclusion. Your argument seems to mix a few arguments like: OpenAI has so much demand and it has made so much compute that it is a bad thing (?!). It can't go on (why?) hence the capabilities are stalling (how?). Then you also say that it is too competitive which is a totally different argument to compute. So in this thread we have multiple vectors of arguments like - high competition - high demand for compute (not sure how this hurts OpenAI but whatever) - capability has stalled

socalgal2 4d ago

Admits China won’t stop but says USA’s top should. Reminds me of City in the Edge of Forever

mrweasel 4d ago
Unsure about your reference, but is your argument: Someone else is doing it, so I should be allowed to as well. That's basically the western companies complaining that they can't compete with China because Chinese companies are allowed to polute more, so they would be allowed to polute as well. I don't buy that. Once you realise that you're doing something wrong or dangerous, you stop, regardless of what everyone else is doing and what rules they have to follow.

andsoitis 4d ago

[dead]

nialv7 4d ago

He is neither right or wrong. The reality is that this is a lose-lose situation from the ordinary person's perspective. Regulation -> regulatory capture and things get worse faster, no regulation -> things simply get worse faster. I mean I don't know what we do at this point. Ideally AI researchers need to be treated like nuclear scientists working for an adversary. But we all know that's not going to happen.

reasonableklout 4d ago
Maybe organize politically? The labs are still staffed by people, and nonviolent demonstrations outside their offices can still reach them.

torginus 4d ago

I think this might have to do with broader market conditions. It seems that the Fed is likely to raise bond rates, which would make borrowing more expensive and slow investments (as well as devaluing any potential IPO) on its own. A rate hike would also mean people getting fired, which would not make companies offering labour replacement very popular in the eyes of the public. Don't want to get political, but there's a real chance that the current admin will perform badly on the midterms, which would mean Democrats would get more leverage to stop funding or block the already unpopular and very expensive datacenter buildouts. PS: Please don't get mad at me, I'm neither a financial expert neither did I mean to express any political leanings.

deleted 4d ago

[deleted]

Kiro 4d ago

These threads are always filled with different versions of the same comment. Normally some reddit style reductive platitude that we've heard a billion times already.

swingboy 4d ago

There’s a difference between a model recursively improving “itself” and improving itself via online learning, right? The former being that these models are helping develop and train future models, but they might not veer too far off in architecture (yet). The latter being the same model being able to train/learn on the fly, in real time, permanently (not just in the current conversation/session), or in other words, adjusting/managing its own weights. The latter seems far more likely to go out of control than the former. But, it also seems like it would take an entire paradigm shift in model architecture, but I could be wrong. Does anyone in the industry think any of these companies are actually close to that kind of self-improvement?

erichocean 4d ago
> Does anyone in the industry think any of these companies are actually close to that kind of self-improvement? Their current plan is to take the existing architecture and shorten the cycle times: move all new RLVR work into mid-training on a pre-existing base; apply new RLVR. Rinse and repeat. If you did that daily, it would be roughly similar to how humans improve.

lrvick 4d ago

Maybe everyone forgets how PCI standards played out. They are an example of an industry self regulating, because the regulators have no idea how technology works. Granted, PCI is so weak it is almost useless, and yet still better than anything congress could have come up with. Where the government might have to step in, is by having a kill switch to cut off internet access from countries that fail to agree to common sense quarantines. We can do mutual remote attestation of labs across the world to ensure every big hot thermally-visable cluster of AI GPUs on the planet are accounted for and running secure enclaves and common sense isolation, along the lines of how we manage nukes. The problem there is I just said too many technical words that seemingly not even the frontier labs understand, as evidenced by all the escapes.

okdood64 4d ago
> Granted, PCI is so weak it is almost useless, and yet still better than anything congress could have come up with. Elaborate, please?

alexmercerdev 4d ago

[flagged]

Digory 4d ago

They seem to be okay with an American slowdown, even though the Chinese won’t slow down. So maybe they want something the Chinese already have: liability protection. The models spontaneously hack everything important without shame. The frontier labs are going to get enjoined and regulated twelve ways to Sunday if the Feds don’t socialize the costs.

d0mine 4d ago
I've only heard of OpenAI and Anthropic agents hacking during training. Are open weights model not smart enough to hack Hugging Face to copy solutions? (doubtful)

cc62cf4a4f20 4d ago

Could this be that the AI companies are seeing progress slow down and need to seed a narrative to explain it? Not clear from outside. Could it be to slow down burn before an IPO to juice profitability projections? More plausible. Kind of hard to take the frontier labs at their word.

paoliniluis 4d ago

Seems that Sacks didn’t get that the message was targeted to the president to ask for a bilateral agreement with China, that’s the end game