6 ms·

* Frontier models need infinite high quality private IP to keep them fed. Forcing an IP theft funnel ensures big lab survival and model intelligence growth. *

* Frontier models need infinite high quality private IP to keep them fed. Forcing an IP theft funnel ensures big lab survival and model intelligence growth. * Open-weight models are 1month behind frontier models. Cheaper, faster, private (no IP theft), steerable (you can security harden your own software without safeguard triggers). No sane business would keep using these API services if they didn't have to. The labs stand to lose a fortune. * Dario has stacked the deck at METR, who are funded by all the same NGOs who are funded by Anthropic and its investors. METR is full of ex-Anthropic employees with massive equity stakes. If they manage to position METR as the "independent evaluator" for the industry, they control what gets evaluated, how, and who passes. * Creating a gap between what the public knows exists (model capabilities) and what is used in secret allows it to be weaponized against other nations and the public. * No requirement for public disclosure on model capabilities allows them to feign they've hit intelligence ceilings while they secretly RSI to the moon with better and better chips. * Slowly but surely, this will allow the big labs to swallow the entire economy and every single business on Earth, by cloning and automating. This, and many more reasons. The labs need to feel more pressure to be held accountable for the incidents they cause (HF incident, etc), so they have an incentive to ensure it does not happen again.

bbor 3d ago
Ok but the first point is just not based in reality whatsoever, sorry.
Turn_Trout 4d ago
Dario's post [1] commits to direct evaluators that can, among other abilities, expose secret RSI. He wants that made law. Do you have a source on METR employees retaining massive equity stakes? [1] https://darioamodei.com/post/we-must-pace-the-frontier https://darioamodei.com/post/we-must-pace-the-frontier
- nullbio 3d ago
  Joe Benton left Anthropic a day before Dario's post, to work for METR evaluations. He was with Anthropic for over a year. He did the same thing that Jacob did (big song and dance about AI apocalypse, media interviews all over the place). He managed the Scalable Oversight team at Anthropic and was the research lead for the Anthropic Fellows Program. So he has equity, and likely lots of it. Then you have Josh Engels quitting DeepMind to work for METR the day before as well, doing the exact same thing. Again, doomer drama all over socials, interviews, and so on. Did I mention METR is founded by an ex-OpenAI researcher? Now you have Demis Hassabis, Sam Altman and Dario, all circlejerking eachother on X saying "we all agree with Dario" - while they ask to be "regulated" by the company that has all of their combined equity-holding ex-employees in it. METR's salaries are listing around 500k/yr. Gee, I wonder where this non-profit with ~35 people is getting all of its money? So the fact that Dario tries to frame it as an "independent third party" is all the evidence you need to know that Dario is a pathological liar and always will be. --- Some more info: Dario's sister, president of Anthropic, is married to the co-founder of Open Philanthropy. The two largest AI doomer NGOs, Center for AI Safety (CAIS) and the Future of Life Institute (FLI), have both received many millions of dollars from them. Ajeya Cotra worked at Open Philanthropy/Coefficient Giving for roughly nine years, including leading its technical AI-safety program in 2024 and contributing to AI-giving strategy in 2025. She subsequently left Coefficient and joined METR, where she is now technical staff. Ajeya is married to Paul Christiano, who founded Alignment Research Center (ARC). Alignment Research Center donated ~$4.5mil to METR. Good Ventures is a funding partner of Open Philanthropy, who funded Jacob Coxon (the first of the Anthropic employees going viral in the media) via a scholarship.
  - Turn_Trout 3d ago
    > Good Ventures is a funding partner of Open Philanthropy, who funded Jacob Coxon (the first of the Anthropic employees going viral in the media) via a scholarship. This conspiracy theory is truly crazy. A $20K scholarship in 2022 is supposed to explain Coxon walking away from unvested equity for a company worth over $950 billion dollars?
ben_w 4d ago
I know people at METR and Anthropic. When they tell you they're worried the tech they're working on may kill everyone despite their best efforts, perhaps believe them.
- nullbio 3d ago
  I'm sure it has nothing to do with their $500,000+ salaries and millions of dollars in equity. It's all solely because they're deeply concerned about next token prediction.
  - phoghed 3d ago
    I think the “next token prediction” is too dismissive and reductive a framing of their capabilities at this point. Yes we all know that’s what they do, and guns just push a few grams of lead out of a pipe. It’s what you can do with that capability that is important. When you couldn’t count the R’s in strawberry it would have been a more effective statement. But a few short years later they are being used to solve millennium puzzles. What if the scaling continues? A model n years from now gets burned into silicon, a single company has millions of the chips, and in a few moments the system spend more time “thinking” than humans have ever spent thinking collectively? If it’s even possible I don’t think there’s anything we can do about it at this point. Cat’s out of the bag.
  - azan_ 3d ago
    You really need to let your priors go if you still use this tired trope of next token prediction. It’s as useful for discussion as saying that human brain is made of fat, protein and carbohydrates - yeah that’s true, but it’s useless observation.
- calgoo 3d ago
  Or you know, stop working on it if its that dangerous? This whole thing of a bunch of employees saying that they are scared of building what they are building, but do it anyway because they are somehow going to make it different? Their model has been used in the planning of mass murdering in war as well as spying on the entire worlds population as well as helping ICE out in the US. They need to stop this BS fearmongering or actually stand up and do something about it. A government regulation is not the answer, especially when its done in a country that is run by a want to be dictator.
user43928 3d ago
Open-weight models are not one month behind. In fact they still have not caught up with February's Mythos, indicating they are more than half a year behind.
- cgio 3d ago
  For practical uses they are there. Arguably the frontier models are worse for some of these practical tasks. And keep in mind, people will use maybe frontier for 1/10th of the work, planning and review, and go open source for rest. The question is if they manage to impose outside us. If not, they are losing competitiveness.
  - user43928 3d ago
    I always thought switching from a SOTA model to a dumber model after planning was a terrible idea. Mostly I heard this from people who I got the impression have little experience in developing greenfield software with agentic AI. Often the same people who talk about spec frameworks. I fundamentally disagree with the approach. I believe the ability to autonomously evaluate, test, and adjust during long horizon tasks is critical to using AI efficiently.
    - rescbr 3d ago
      Well, it's the enterprise software house pipeline... The software architect writes the spec, hands it down to the implementation team, senior leads, junior devs or offshore teams codes it. I also disagree with the approach, this is cargo-culting the existing ways of working.
      - jurgenburgen 3d ago
        > this is cargo-culting the existing ways of working. I think you mean the existing anti-patterns. They don’t call them ivory tower architects for nothing.
- mistercow 3d ago
  I’m not sure how you can really make either statement work anymore. Now that smaller models are actually broadly usable, “behindness” is no longer a scalar and at the tails, where no open lab seems to be trying to compete at the >10T scale and no closed lab seems to care about <400B anymore, it’s just apples to oranges. It’s like talking about whether Qualcomm is “behind” Nvidia.
aatd86 3d ago
That is what the Palantir guy (Karp) has been warning against. People/business need to keep their IP instead of throwing it all into Claude and whatnot. Risk being the worst aspects of communism which I think he meant centralization of decision, asymmetric supply/demand for compute (they lock you in), and the tech overloard Anthropic/OpenAI/xAI being in competition with everyone;s business all of a sudden with much more data. An unfair advantage in markets made super competitive all of a sudden. A winner takes all attempt. This is not sustainable anyway, the scale at which they want to control data flows. Time was money, now data is money and they are too greedy for it. All this agitation is just a silly attempt at constraining competition. The danger is not the AI, it is having all your systems connected. Overreliance on networked tech.
Bluestein 3d ago
> Open-weight models are 1month behind frontier models If correct, grip is tightening. Used to be 1y, then 6 months ...
enknee1 3d ago
- Frontier models need infinite high quality private IP to keep them fed. Forcing an IP theft funnel ensures big lab survival and model intelligence growth. While the data pipeline is necessary, the assumed source for data (in this case people) is incorrect. Right now models are "aligned" because of the RL-based people pipeline. But there's a whole world of readily available data that does not require a human to access. Put sensors on a vacuum, install lidar on the front of a car, sell phones with cameras on them, and you amass data for the creation of models of the world that don't include the need for a human filter. Richard Sutton and many others see this approach as the only viable path to AGI. Such models would be truly alien and unaccessibly dangerous. We're probably building them right now.
deleted 3d ago
[deleted]

bbor 3d ago

Ok but the first point is just not based in reality whatsoever, sorry.

Turn_Trout 4d ago

Dario's post [1] commits to direct evaluators that can, among other abilities, expose secret RSI. He wants that made law. Do you have a source on METR employees retaining massive equity stakes? [1] https://darioamodei.com/post/we-must-pace-the-frontier https://darioamodei.com/post/we-must-pace-the-frontier

nullbio 3d ago
Joe Benton left Anthropic a day before Dario's post, to work for METR evaluations. He was with Anthropic for over a year. He did the same thing that Jacob did (big song and dance about AI apocalypse, media interviews all over the place). He managed the Scalable Oversight team at Anthropic and was the research lead for the Anthropic Fellows Program. So he has equity, and likely lots of it. Then you have Josh Engels quitting DeepMind to work for METR the day before as well, doing the exact same thing. Again, doomer drama all over socials, interviews, and so on. Did I mention METR is founded by an ex-OpenAI researcher? Now you have Demis Hassabis, Sam Altman and Dario, all circlejerking eachother on X saying "we all agree with Dario" - while they ask to be "regulated" by the company that has all of their combined equity-holding ex-employees in it. METR's salaries are listing around 500k/yr. Gee, I wonder where this non-profit with ~35 people is getting all of its money? So the fact that Dario tries to frame it as an "independent third party" is all the evidence you need to know that Dario is a pathological liar and always will be. --- Some more info: Dario's sister, president of Anthropic, is married to the co-founder of Open Philanthropy. The two largest AI doomer NGOs, Center for AI Safety (CAIS) and the Future of Life Institute (FLI), have both received many millions of dollars from them. Ajeya Cotra worked at Open Philanthropy/Coefficient Giving for roughly nine years, including leading its technical AI-safety program in 2024 and contributing to AI-giving strategy in 2025. She subsequently left Coefficient and joined METR, where she is now technical staff. Ajeya is married to Paul Christiano, who founded Alignment Research Center (ARC). Alignment Research Center donated ~$4.5mil to METR. Good Ventures is a funding partner of Open Philanthropy, who funded Jacob Coxon (the first of the Anthropic employees going viral in the media) via a scholarship.
- Turn_Trout 3d ago
  > Good Ventures is a funding partner of Open Philanthropy, who funded Jacob Coxon (the first of the Anthropic employees going viral in the media) via a scholarship. This conspiracy theory is truly crazy. A $20K scholarship in 2022 is supposed to explain Coxon walking away from unvested equity for a company worth over $950 billion dollars?

ben_w 4d ago

I know people at METR and Anthropic. When they tell you they're worried the tech they're working on may kill everyone despite their best efforts, perhaps believe them.

nullbio 3d ago
I'm sure it has nothing to do with their $500,000+ salaries and millions of dollars in equity. It's all solely because they're deeply concerned about next token prediction.
- phoghed 3d ago
  I think the “next token prediction” is too dismissive and reductive a framing of their capabilities at this point. Yes we all know that’s what they do, and guns just push a few grams of lead out of a pipe. It’s what you can do with that capability that is important. When you couldn’t count the R’s in strawberry it would have been a more effective statement. But a few short years later they are being used to solve millennium puzzles. What if the scaling continues? A model n years from now gets burned into silicon, a single company has millions of the chips, and in a few moments the system spend more time “thinking” than humans have ever spent thinking collectively? If it’s even possible I don’t think there’s anything we can do about it at this point. Cat’s out of the bag.
- azan_ 3d ago
  You really need to let your priors go if you still use this tired trope of next token prediction. It’s as useful for discussion as saying that human brain is made of fat, protein and carbohydrates - yeah that’s true, but it’s useless observation.
calgoo 3d ago
Or you know, stop working on it if its that dangerous? This whole thing of a bunch of employees saying that they are scared of building what they are building, but do it anyway because they are somehow going to make it different? Their model has been used in the planning of mass murdering in war as well as spying on the entire worlds population as well as helping ICE out in the US. They need to stop this BS fearmongering or actually stand up and do something about it. A government regulation is not the answer, especially when its done in a country that is run by a want to be dictator.

user43928 3d ago

Open-weight models are not one month behind. In fact they still have not caught up with February's Mythos, indicating they are more than half a year behind.

cgio 3d ago
For practical uses they are there. Arguably the frontier models are worse for some of these practical tasks. And keep in mind, people will use maybe frontier for 1/10th of the work, planning and review, and go open source for rest. The question is if they manage to impose outside us. If not, they are losing competitiveness.
- user43928 3d ago
  I always thought switching from a SOTA model to a dumber model after planning was a terrible idea. Mostly I heard this from people who I got the impression have little experience in developing greenfield software with agentic AI. Often the same people who talk about spec frameworks. I fundamentally disagree with the approach. I believe the ability to autonomously evaluate, test, and adjust during long horizon tasks is critical to using AI efficiently.
  - rescbr 3d ago
    Well, it's the enterprise software house pipeline... The software architect writes the spec, hands it down to the implementation team, senior leads, junior devs or offshore teams codes it. I also disagree with the approach, this is cargo-culting the existing ways of working.
    - jurgenburgen 3d ago
      > this is cargo-culting the existing ways of working. I think you mean the existing anti-patterns. They don’t call them ivory tower architects for nothing.
mistercow 3d ago
I’m not sure how you can really make either statement work anymore. Now that smaller models are actually broadly usable, “behindness” is no longer a scalar and at the tails, where no open lab seems to be trying to compete at the >10T scale and no closed lab seems to care about <400B anymore, it’s just apples to oranges. It’s like talking about whether Qualcomm is “behind” Nvidia.

aatd86 3d ago

That is what the Palantir guy (Karp) has been warning against. People/business need to keep their IP instead of throwing it all into Claude and whatnot. Risk being the worst aspects of communism which I think he meant centralization of decision, asymmetric supply/demand for compute (they lock you in), and the tech overloard Anthropic/OpenAI/xAI being in competition with everyone;s business all of a sudden with much more data. An unfair advantage in markets made super competitive all of a sudden. A winner takes all attempt. This is not sustainable anyway, the scale at which they want to control data flows. Time was money, now data is money and they are too greedy for it. All this agitation is just a silly attempt at constraining competition. The danger is not the AI, it is having all your systems connected. Overreliance on networked tech.

Bluestein 3d ago

> Open-weight models are 1month behind frontier models If correct, grip is tightening. Used to be 1y, then 6 months ...

enknee1 3d ago

- Frontier models need infinite high quality private IP to keep them fed. Forcing an IP theft funnel ensures big lab survival and model intelligence growth. While the data pipeline is necessary, the assumed source for data (in this case people) is incorrect. Right now models are "aligned" because of the RL-based people pipeline. But there's a whole world of readily available data that does not require a human to access. Put sensors on a vacuum, install lidar on the front of a car, sell phones with cameras on them, and you amass data for the creation of models of the world that don't include the need for a human filter. Richard Sutton and many others see this approach as the only viable path to AGI. Such models would be truly alien and unaccessibly dangerous. We're probably building them right now.

deleted 3d ago

[deleted]