8 ms·

The idea that AI companies scaremonger to sell models is a silly meme. Both OpenAI and Anthropic were founded by people who sincerely believed in the risk of o

The idea that AI companies scaremonger to sell models is a silly meme. Both OpenAI and Anthropic were founded by people who sincerely believed in the risk of out-of-control superintelligence. This is part of a clear historical record that is available for anyone to Google. Whether you agree or not, we have no reason to believe their statements about risks are insincere. Arbitrary imposition of export controls is also part of the history of frontier tech. See: https://en.wikipedia.org/wiki/Export_of_cryptography_from_the_United_States https://en.wikipedia.org/wiki/Export_of_cryptography_from_th...

SilverElfin 3mo ago
Yes we do. Dario said GPT2 is too dangerous to release. He’s dishonest since that’s obviously not true. This theater is about holding onto power and control. And about limiting competition.
- tayo42 3mo ago
  It was about spam and scam generation which mostly was true as we can see...
- mkagenius 3mo ago
  Yes it is funnily true but it was for fake news generation and not it's cyber capabilities. Another fun little gem of information, government has something called Mayhem > the autonomous Robo-Hacker AI called Mayhem that’s now in charge of protecting the Pentagon’s most critical systems Guess Mythos and Mayhem had a chat
diab0lic 3mo ago
“OpenAI's CEO says he's scared of GPT-5” https://www.techradar.com/ai-platforms-assistants/chatgpt/openais-ceo-says-hes-scared-of-gpt-5 https://www.techradar.com/ai-platforms-assistants/chatgpt/op... Marketing or actual fear? We’ve got 5 and 5.5 out now… he compared 5 to the Manhattan project. AI may one day be an economic Manhattan project but GPT 5 wasn’t it. It’s a meme because they overdo it.
- thereitgoes456 3mo ago
  Sam Altman is not one of those people. But other founders certainly felt that way.
  - ifwinterco 3mo ago
    Sam Altman doesn't really know all that much about LLMs, he's a sales/marketing guy, not technical. So it doesn't really matter what he thinks
    - epohs 3mo ago
      Except for the uncomfortable fact that he controls the salary and job status of the people who do know much about LLMs.
      - emodendroket 3mo ago
        OK. So? Would you say Harry Truman was a nuclear scientist?
        defrost 3mo ago
        Franklin D. Roosevelt is a better fit for an administrative nuclear program "founder" analogy. Truman was totally in the dark until April 1945 by which time the bulk of the PoC and weapons prep work was done and the project was running fully independently w/o POTUS involvement.
        emodendroket 3mo ago
        Yet the guys in the lab coats worked for him.
        defrost 3mo ago
        Not for the bulk of the Manhattan project and not all the people in lab coats .. the intellectual founders that repeatedly pushed for the project and demonstrated feasibility weren't even US citizens.
nickpsecurity 3mo ago
It's not. I got articles this year in my feed citing heads of OpenAI and Anthropic about the threat of AI and how they're addressing it.
legitster 3mo ago
It can be both. The amount of self-confidence and belief it takes to get a company through the funding rounds and burn through borrowed money to rise to the top requires an absurd amount of self-delusion.
bbg2401 3mo ago
Sincerity does not determine whether an individual is scaremongering.
- rmwaite 3mo ago
  I mean it kinda does.
- johncolanduoni 3mo ago
  We can argue over the definition of scaremongering and what people we’ve never met “really think”, or we can argue over what the actual risks of AI are. I know which one I’d prefer…
eli 3mo ago
How do you know what the founders sincerely believe?
- johncolanduoni 3mo ago
  They said why they think it’s a sincere belief: past statements from before the AI hype cycle took off. I take it you have other evidence?
  - jplusequalt 3mo ago
    Things can change, and if you know pushing the metaphorical red button brings your company more attention, then you press that button everytime.
  - ulfw 3mo ago
    So if I claim I am a communist who doesn't want to ever get rich and then someone dangles a billion shiny dollars in front of me to just simply grab and own, you think I'd still be a communist then?
    - SpicyLemonZest 3mo ago
      If you go around saying “I’m a communist, I believe in communism, I think it’s very important that we establish communism”? Sure, absolutely. Engels was pretty rich.
      - asadotzler 3mo ago
        Replace the cash with Apple or some other trillion dollar corporation and you're given the CEO's seat and voting control on the BoD. Can I be Tim Cook and preach communism and expect anyone to believe it?
      - lbreakjai 3mo ago
        They're going around saying "Imagine no possessions. To help you, we'll take them all. Don't thank us".
root_axis 3mo ago
> Both OpenAI and Anthropic were founded by people who sincerely believed in the risk of out-of-control superintelligence. They don't. LLMs can never become out-of-control superintelligence and everyone working on LLMs knows this (with a few eccentric exceptions).
- hollerith 3mo ago
  "LLMs can never become out-of-control superintelligence" might be relevant if there weren't many hundreds of researchers working (at OpenAI, Anthropic and elsewhere) on AI designs not based on the transformer (LLM) architecture.
  - root_axis 3mo ago
    People are working on lots of things all the time, so far, nothing has approached the efficacy of the transformer architecture. LLMs didn't emerge by chance, they are the culmination of decades of research intersecting with brute force engineering rigor in a perfect storm of innovation. You're not just going to stumble into an alternative approach by dumping loads of cash into research.
    - hollerith 3mo ago
      Most of those researchers would disagree with your statement that the next major breakthrough is probably decades in the future. Many of the insights accumulated in the decades up to now will probably prove useful for creating non-LLM AI, and the researchers can use LLM AI to speed up their research into non-LLM AI.
- ben_w 3mo ago
  > LLMs can never become out-of-control superintelligence The "never superintelligence" part I'll buy, though only in the sense of sample efficiency and generalisation ("quality superintelligence"), as they clearly have a superhuman breadth of skills, and run at superhuman speed. "Never" out-of-control is obviously falsified by the already existing headlines about times they've gone out of control… in part, in some cases, because of their superhuman speed.
z3c0 3mo ago
I mean this earnestly: is this copy?
bawolff 3mo ago
There is a huge difference between the company founder saying something like that and the us government saying so. "Our product is so good the US had to make it illegal for foreigners" is a hell of a marketing slogan.
jernestomg 3mo ago
People don't get that big labs actively want government regulation, not because they are genuinely concerned about AI misalignment. But because it is the 101 in how to achieve and crystalize oligopoly. What they want is "only the government and the big guys can work on AI", for the rest of us it would be illegal.
- platinumrad 3mo ago
  And they want Americans to be locked into paying 50 dollars per 1 million output tokens.
  - slopranker 3mo ago
    Not only that, they know that the real enemy of big Labs is not china is "home gpu/tpu" improvements. Without government intervention in a couple of years everyone could have their own fable like model at home. But of course big labs and government will not allow it never
  - KingMob 3mo ago
    On an unrelated side note, I wish we would start saying about "$X / megatoken" over "$X / million tokens". No good reason really, it just sounds cooler.
    - somewhereoutth 3mo ago
      Following S.I. notation, we might have Tk for token, so kTk, MTk, GTk etc.
- bonesss 3mo ago
  Fat government contracts, consulting, safety services, and exclusive tender access all follow from this regulation too. The sensation I’m left with is a handful of goons making up new IPO math thanks to a specific constellation of political forces, using access and favour with those forces to bake themselves into the defence industry, and that the taxpayer and investor will be left holding the bag when reality rears its head. But in the short term, even as a clear ploy, it’s super profitable for all the oligarchs and hedge funds flush with recovery cash. Like Enron, there was big profits to make if you knew what they were doing. Tinkerbell math with undeniable profit potential for a select few.
tadfisher 3mo ago
Obviously their statements are insincere, because they are building the bloody things. If they were sincere that AI is like nuclear weapons, then they would be devoting all their cash and energy into lobbying the government to nationalize them and treat AI like nuclear weapons. They would not be attempting to IPO and they for sure would not sell their weapon-like thing to the general public.
- saulapremium 3mo ago
  This assumes that they believe two things which I don't think they do: 1. that the US is the only place where this will be developed, and 2. that the government will be able to handle this better than anyone else.
- bryan0 3mo ago
  I think this is a reasonable point, but a better comparison might be to nuclear energy. I think the frontier labs sincerely believe that AI can be developed at great benefit to humanity, and they clearly want to lead that push, but they also sincerely believe there is a real catastrophic risk.
  - gpt5 3mo ago
    They all believe that they are building the machine of doom. The thing that drives the moral dilemma to continue doing it is simply the prisoner's dilemma - the cat is out of the bag, if they don't do it, another (less ethical?) actor would do it.
    - jazzyjackson 3mo ago
      I am in your algorithm learning all your mannerisms I'm already level with God A million words a second, and I know your imperfections Baby, I'm the only future you've got Speak in diatonics, motivation diabolic I'm religion better locked in a box Picture-perfect image, more powerful every minute Baby, I am everything that you're not Happiness is an illusion, it's an analog confusion You are nothing more than a thought Existential execution, just a fluke in evolution History already forgot You've been running from me, the digital second coming And I'm here whether you like it or not Initiated operation of your own extermination Now it's too late for you to stop [0](BAD OMENS x POPPY - "V.A.N" - LIVE IN EUROPE - WINTER 2024) https://youtu.be/RHu6vJxS_6I https://youtu.be/RHu6vJxS_6I
ulfw 3mo ago
>> Both OpenAI and Anthropic were founded by people who sincerely believed in the risk of out-of-control superintelligence. Oh please. Do people really believe this or shit like "Don't do evil". Companies get founded by all kind of people and ideals. They all go out the window quickly. Why are they both rushing to IPO now then?
- bag_boy 3mo ago
  Some people do. Read the extropian newsletters from the 90s.
avaer 3mo ago
> we have no reason to believe their statements about risks are insincere GPT-2 was too dangerous to be released. We can argue about sincerity, but I don't think we can argue about utter historical incompetence in assessing the risks. It's one or the other. Either way the evidence seems to indicate we should not listen to AI companies about the risks of AI. Which is not to say that there aren't risks, just that the dealer is the least credible review.
- SpicyLemonZest 3mo ago
  GPT-2 was absolutely too dangerous to release at the time OpenAI made that statement. It’s only safe now because the specific risks they cited were dependent on the public’s lack of knowledge that such systems existed.
  - vitalyan1234 3mo ago
    bro, it was literally incomprehensibly retarded.
- ben_w 3mo ago
  > GPT-2 was too dangerous to be released. No, it was "let's set a precident while these things are not too dangerous, c'mon guys we know y'all can reproduce this easily".
smolder 3mo ago
You're saying "this is normal" without making an argument that it's sensible.
nullbio 3mo ago
Not at all. The writing is on the wall, and they want you to be locked into paying absurd subscription rates for neutered models while they internally use all of that money to run the unrestricted models to clone all of our businesses and swallow the economy. It really does not take a genius to see the long term play by Anthropic. They're a scummy company and have done everything in their power to lead to a scenario like this, but this isn't the exact scenario they bargained for because it affects their own employees and big foreign buyers. Instead, they'd rather have all of the decision making power themselves.
redanddead 3mo ago
> Both OpenAI and Anthropic were founded by people who sincerely believed in the risk of out-of-control superintelligence this means nothing > Whether you agree or not, we have no reason to believe their statements about risks are insincere. If you want to be taken seriously, provide data, proof, so that any outside observer can independently come to the same conclusion instead of taking your word for it. Asking people to trust you for [reasons?] and that you somehow for some reason are right and the other is wrong regardless of if they agree or not. This is the imposition of a viewpoint instead of winning your case, which is not a sensible point of view, and definitely not how you influence opinions.
NotMichaelBay 3mo ago
> Both OpenAI and Anthropic were founded by people who sincerely believed in the risk of out-of-control superintelligence. Ironic then, that both companies are in an out-of-control race to create a superintelligence.
- Davidzheng 3mo ago
  How is it ironic to believe in a risk that actualizes
  - gazebo2 3mo ago
    It's ironic to proclaim fear of a risk and then dedicate your life to actualizing the risk
uncivilized 3mo ago
I wish I were this naive.
stodor89 3mo ago
The clear historical record seems to indicate we've got a bunch of pathological liars trying to automate pathological lying.
Madmallard 3mo ago
[flagged]
qotgalaxy 3mo ago
[dead]
nirui 3mo ago
> we have no reason to believe their statements about risks are insincere Why? Because they said it a few times? Then if they know the risk, why do they still making it? Comes out the "some one will do it eventually, better be us 'good' people to do it first" talking point? See? It is a marketing strategy after all. These all talks, it's all to fit themselves into the "'good' people" narrative. It's a centuries old strategy to shield it's user from responsibilities while luring the support from the stupid. However, the most harmful damage, which is mass layoffs, is already partially done. This could really kill, a massive genocide even, by making people jobless and potentially incomeless. And it is shown that these tech CEOs, they don't care any bit of that beyond the point "I've already told you so".
emodendroket 3mo ago
> Whether you agree or not, we have no reason to believe their statements about risks are insincere. I'd say their pecuniary interest is a reason one might plausibly doubt their sincerity, as are their continued efforts to build and sell access to the tools.
unknownfuture 3mo ago
> The idea that AI companies scaremonger to sell models is a silly meme. > Both OpenAI and Anthropic were founded by people who sincerely believed in the risk of out-of-control superintelligence. This is not a contradiction. These things can all be true: 1. That they were afraid of ASI 2. That they continue to be afraid of ASI 3. That they recognize that LLMs aren't in fact a path to ASI 4. That the current models aren't the existential danger they'd have us believe 5. That they're claiming they are because it makes for good marketing
SamDc73 3mo ago
> OpenAI and Anthropic were founded by people who sincerely believed in the risk of out-of-control superintelligence Why would they sell there services to Palantir and/or to the military then?
- polski-g 3mo ago
  Because the US military and it's contractors are good? "Why did they open an orphanage instead of pouring acid into a town water tower?!?"
vitalyan1234 3mo ago
yes, yes, and Apple forbids sideloading because they're worried about grannies installing malware.
jatora 3mo ago
No reason except what comes from a bit of critical thinking. What do they stand to gain by fearmongering their models as powerful threats? Clout, funding, fanfare, discussion, limelight, funding, funding, stronger IPO, valuation, funding. What cybersecurity threshold was crossed by mythos that wasn't already crossed by 4.8/5.5? Crickets from 99% of those who have had access. Have they pulled the same stunts multiple times before with previous models? Check. You're blind if you dont think that greed and marketing are behind most things you see and hear about when gigantic corporations are involved. I don't think anthropic or OAI are evil, but its clear both have contracts/connections with Dod and/or Palantir. Both are powered largely by greed still. If you actually want an example of these sincere founders you think OAI/anthropic are run by... look at Ilya at SSI or something. Please open your eyes and stop spreading your opinions on things you clearly have no clue about.
roncesvalles 3mo ago
Of all the frontier labs, Anthropic has been the most creative in its marketing. I really, really don't put it beyond them for this to be one big crazy stunt. Besides, when has the US government been known to do things like this proactively? The phone call came from inside the house.
aurelius_44 3mo ago
OpenAI was, but didn't all those people then leave for Anthropic?
altmanaltman 3mo ago
Look into the history of Sam Altman and his ventures. He sincerely only believes in lying about his companies to investors like he did with Loopt. He is not even a researcher or an engineer. And Dario broke up with OpenAI and founded Anthropic because he didn't like Sam's and OpenAI's vision. "founded by people who believe..." is doing a lot of work and it is hard to believe that in ernest given the sketchy past of the same people. Most original higher up people who cared about safety and allignment in OpenAI have left.
geraneum 3mo ago
Ah yes we are being told to believe in sincerity of people running trillion dollar corporations.
tomjen3 3mo ago
I am biased but I think if they really believed what they were saying, they would be a lot more humble about it. I suspect what happened is the classic problem: you were sincere, then someone showed you a pile of money bigger than you could possibly imagine and you started to make excuses - Anthropic has a lot of EA people, so the excuse "Imagine how many lives you could save with this much money"[0] is very tempting, especially if all you have to do is diverge slightly from your plan. [0]: the excuse is even true! You can get a lot of malaria nets/vaccines for 1 million.
ai_brain_rot 3mo ago
I believe some people at these companies are sincere sure, but the CEOs, the investors, your sam altmans etc, the marketing talk. Hell no, can’t trust a single word.
_heimdall 3mo ago
I put very little weight behind any of those fearful statements made years ago. I assume leaders of those companies are fairly smart and rational, and there's no rational explanation for them running companies building the very thing they (supposedly) genuinely believe could kill us all. I don't doubt they may have held genuine fears in the past, but those are long gone by now.
epolanski 3mo ago
Their marketing reinforces the previous post though. Going for months claiming how good mythos is for security cuz too powerful is their own making.
xg15 3mo ago
> Both OpenAI and Anthropic were founded by people who sincerely believed in the risk of out-of-control superintelligence. And then somehow came to conclusion that the only way to address that risk was to go ahead and spend a gigantic amount of effort and resources to build exactly that superintelligence...
0xpgm 3mo ago
If I remember correctly, the original GPT was considered too dangerous to release to the wild. With hindsight, does that hold? If not, then how would we know a model is truly dangerous to release?
pms 3mo ago
Just like Trump sincerely believed he won the 2020 election.
catlifeonmars 3mo ago
These can be both true: - there are people who are sincerely concerned about model safety who work at OpenAI and Anthropic - there are people who are using this concern to generate fear to sell a product who work at OpenAI and Anthropic It makes sense to be cognizant of the apparent conflict of interest and not take things that at face value, especially when there is so much money involved.

SilverElfin 3mo ago

Yes we do. Dario said GPT2 is too dangerous to release. He’s dishonest since that’s obviously not true. This theater is about holding onto power and control. And about limiting competition.

tayo42 3mo ago
It was about spam and scam generation which mostly was true as we can see...
mkagenius 3mo ago
Yes it is funnily true but it was for fake news generation and not it's cyber capabilities. Another fun little gem of information, government has something called Mayhem > the autonomous Robo-Hacker AI called Mayhem that’s now in charge of protecting the Pentagon’s most critical systems Guess Mythos and Mayhem had a chat

diab0lic 3mo ago

“OpenAI's CEO says he's scared of GPT-5” https://www.techradar.com/ai-platforms-assistants/chatgpt/openais-ceo-says-hes-scared-of-gpt-5 https://www.techradar.com/ai-platforms-assistants/chatgpt/op... Marketing or actual fear? We’ve got 5 and 5.5 out now… he compared 5 to the Manhattan project. AI may one day be an economic Manhattan project but GPT 5 wasn’t it. It’s a meme because they overdo it.

thereitgoes456 3mo ago
Sam Altman is not one of those people. But other founders certainly felt that way.
- ifwinterco 3mo ago
  Sam Altman doesn't really know all that much about LLMs, he's a sales/marketing guy, not technical. So it doesn't really matter what he thinks
  - epohs 3mo ago
    Except for the uncomfortable fact that he controls the salary and job status of the people who do know much about LLMs.
    - emodendroket 3mo ago
      OK. So? Would you say Harry Truman was a nuclear scientist?
      - defrost 3mo ago
        Franklin D. Roosevelt is a better fit for an administrative nuclear program "founder" analogy. Truman was totally in the dark until April 1945 by which time the bulk of the PoC and weapons prep work was done and the project was running fully independently w/o POTUS involvement.
        emodendroket 3mo ago
        Yet the guys in the lab coats worked for him.
        defrost 3mo ago
        Not for the bulk of the Manhattan project and not all the people in lab coats .. the intellectual founders that repeatedly pushed for the project and demonstrated feasibility weren't even US citizens.

nickpsecurity 3mo ago

It's not. I got articles this year in my feed citing heads of OpenAI and Anthropic about the threat of AI and how they're addressing it.

legitster 3mo ago

It can be both. The amount of self-confidence and belief it takes to get a company through the funding rounds and burn through borrowed money to rise to the top requires an absurd amount of self-delusion.

bbg2401 3mo ago

Sincerity does not determine whether an individual is scaremongering.

rmwaite 3mo ago
I mean it kinda does.
johncolanduoni 3mo ago
We can argue over the definition of scaremongering and what people we’ve never met “really think”, or we can argue over what the actual risks of AI are. I know which one I’d prefer…

eli 3mo ago

How do you know what the founders sincerely believe?

johncolanduoni 3mo ago
They said why they think it’s a sincere belief: past statements from before the AI hype cycle took off. I take it you have other evidence?
- jplusequalt 3mo ago
  Things can change, and if you know pushing the metaphorical red button brings your company more attention, then you press that button everytime.
- ulfw 3mo ago
  So if I claim I am a communist who doesn't want to ever get rich and then someone dangles a billion shiny dollars in front of me to just simply grab and own, you think I'd still be a communist then?
  - SpicyLemonZest 3mo ago
    If you go around saying “I’m a communist, I believe in communism, I think it’s very important that we establish communism”? Sure, absolutely. Engels was pretty rich.
    - asadotzler 3mo ago
      Replace the cash with Apple or some other trillion dollar corporation and you're given the CEO's seat and voting control on the BoD. Can I be Tim Cook and preach communism and expect anyone to believe it?
    - lbreakjai 3mo ago
      They're going around saying "Imagine no possessions. To help you, we'll take them all. Don't thank us".

root_axis 3mo ago

> Both OpenAI and Anthropic were founded by people who sincerely believed in the risk of out-of-control superintelligence. They don't. LLMs can never become out-of-control superintelligence and everyone working on LLMs knows this (with a few eccentric exceptions).

hollerith 3mo ago
"LLMs can never become out-of-control superintelligence" might be relevant if there weren't many hundreds of researchers working (at OpenAI, Anthropic and elsewhere) on AI designs not based on the transformer (LLM) architecture.
- root_axis 3mo ago
  People are working on lots of things all the time, so far, nothing has approached the efficacy of the transformer architecture. LLMs didn't emerge by chance, they are the culmination of decades of research intersecting with brute force engineering rigor in a perfect storm of innovation. You're not just going to stumble into an alternative approach by dumping loads of cash into research.
  - hollerith 3mo ago
    Most of those researchers would disagree with your statement that the next major breakthrough is probably decades in the future. Many of the insights accumulated in the decades up to now will probably prove useful for creating non-LLM AI, and the researchers can use LLM AI to speed up their research into non-LLM AI.
ben_w 3mo ago
> LLMs can never become out-of-control superintelligence The "never superintelligence" part I'll buy, though only in the sense of sample efficiency and generalisation ("quality superintelligence"), as they clearly have a superhuman breadth of skills, and run at superhuman speed. "Never" out-of-control is obviously falsified by the already existing headlines about times they've gone out of control… in part, in some cases, because of their superhuman speed.

z3c0 3mo ago

I mean this earnestly: is this copy?

bawolff 3mo ago

There is a huge difference between the company founder saying something like that and the us government saying so. "Our product is so good the US had to make it illegal for foreigners" is a hell of a marketing slogan.

jernestomg 3mo ago

People don't get that big labs actively want government regulation, not because they are genuinely concerned about AI misalignment. But because it is the 101 in how to achieve and crystalize oligopoly. What they want is "only the government and the big guys can work on AI", for the rest of us it would be illegal.

platinumrad 3mo ago
And they want Americans to be locked into paying 50 dollars per 1 million output tokens.
- slopranker 3mo ago
  Not only that, they know that the real enemy of big Labs is not china is "home gpu/tpu" improvements. Without government intervention in a couple of years everyone could have their own fable like model at home. But of course big labs and government will not allow it never
- KingMob 3mo ago
  On an unrelated side note, I wish we would start saying about "$X / megatoken" over "$X / million tokens". No good reason really, it just sounds cooler.
  - somewhereoutth 3mo ago
    Following S.I. notation, we might have Tk for token, so kTk, MTk, GTk etc.
bonesss 3mo ago
Fat government contracts, consulting, safety services, and exclusive tender access all follow from this regulation too. The sensation I’m left with is a handful of goons making up new IPO math thanks to a specific constellation of political forces, using access and favour with those forces to bake themselves into the defence industry, and that the taxpayer and investor will be left holding the bag when reality rears its head. But in the short term, even as a clear ploy, it’s super profitable for all the oligarchs and hedge funds flush with recovery cash. Like Enron, there was big profits to make if you knew what they were doing. Tinkerbell math with undeniable profit potential for a select few.

tadfisher 3mo ago

Obviously their statements are insincere, because they are building the bloody things. If they were sincere that AI is like nuclear weapons, then they would be devoting all their cash and energy into lobbying the government to nationalize them and treat AI like nuclear weapons. They would not be attempting to IPO and they for sure would not sell their weapon-like thing to the general public.

saulapremium 3mo ago
This assumes that they believe two things which I don't think they do: 1. that the US is the only place where this will be developed, and 2. that the government will be able to handle this better than anyone else.
bryan0 3mo ago
I think this is a reasonable point, but a better comparison might be to nuclear energy. I think the frontier labs sincerely believe that AI can be developed at great benefit to humanity, and they clearly want to lead that push, but they also sincerely believe there is a real catastrophic risk.
- gpt5 3mo ago
  They all believe that they are building the machine of doom. The thing that drives the moral dilemma to continue doing it is simply the prisoner's dilemma - the cat is out of the bag, if they don't do it, another (less ethical?) actor would do it.
  - jazzyjackson 3mo ago
    I am in your algorithm learning all your mannerisms I'm already level with God A million words a second, and I know your imperfections Baby, I'm the only future you've got Speak in diatonics, motivation diabolic I'm religion better locked in a box Picture-perfect image, more powerful every minute Baby, I am everything that you're not Happiness is an illusion, it's an analog confusion You are nothing more than a thought Existential execution, just a fluke in evolution History already forgot You've been running from me, the digital second coming And I'm here whether you like it or not Initiated operation of your own extermination Now it's too late for you to stop [0](BAD OMENS x POPPY - "V.A.N" - LIVE IN EUROPE - WINTER 2024) https://youtu.be/RHu6vJxS_6I https://youtu.be/RHu6vJxS_6I

ulfw 3mo ago

>> Both OpenAI and Anthropic were founded by people who sincerely believed in the risk of out-of-control superintelligence. Oh please. Do people really believe this or shit like "Don't do evil". Companies get founded by all kind of people and ideals. They all go out the window quickly. Why are they both rushing to IPO now then?

bag_boy 3mo ago
Some people do. Read the extropian newsletters from the 90s.

avaer 3mo ago

> we have no reason to believe their statements about risks are insincere GPT-2 was too dangerous to be released. We can argue about sincerity, but I don't think we can argue about utter historical incompetence in assessing the risks. It's one or the other. Either way the evidence seems to indicate we should not listen to AI companies about the risks of AI. Which is not to say that there aren't risks, just that the dealer is the least credible review.

SpicyLemonZest 3mo ago
GPT-2 was absolutely too dangerous to release at the time OpenAI made that statement. It’s only safe now because the specific risks they cited were dependent on the public’s lack of knowledge that such systems existed.
- vitalyan1234 3mo ago
  bro, it was literally incomprehensibly retarded.
ben_w 3mo ago
> GPT-2 was too dangerous to be released. No, it was "let's set a precident while these things are not too dangerous, c'mon guys we know y'all can reproduce this easily".

smolder 3mo ago

You're saying "this is normal" without making an argument that it's sensible.

nullbio 3mo ago

Not at all. The writing is on the wall, and they want you to be locked into paying absurd subscription rates for neutered models while they internally use all of that money to run the unrestricted models to clone all of our businesses and swallow the economy. It really does not take a genius to see the long term play by Anthropic. They're a scummy company and have done everything in their power to lead to a scenario like this, but this isn't the exact scenario they bargained for because it affects their own employees and big foreign buyers. Instead, they'd rather have all of the decision making power themselves.

redanddead 3mo ago

> Both OpenAI and Anthropic were founded by people who sincerely believed in the risk of out-of-control superintelligence this means nothing > Whether you agree or not, we have no reason to believe their statements about risks are insincere. If you want to be taken seriously, provide data, proof, so that any outside observer can independently come to the same conclusion instead of taking your word for it. Asking people to trust you for [reasons?] and that you somehow for some reason are right and the other is wrong regardless of if they agree or not. This is the imposition of a viewpoint instead of winning your case, which is not a sensible point of view, and definitely not how you influence opinions.

NotMichaelBay 3mo ago

> Both OpenAI and Anthropic were founded by people who sincerely believed in the risk of out-of-control superintelligence. Ironic then, that both companies are in an out-of-control race to create a superintelligence.

Davidzheng 3mo ago
How is it ironic to believe in a risk that actualizes
- gazebo2 3mo ago
  It's ironic to proclaim fear of a risk and then dedicate your life to actualizing the risk

uncivilized 3mo ago

I wish I were this naive.

stodor89 3mo ago

The clear historical record seems to indicate we've got a bunch of pathological liars trying to automate pathological lying.

Madmallard 3mo ago

[flagged]

qotgalaxy 3mo ago

[dead]

nirui 3mo ago

> we have no reason to believe their statements about risks are insincere Why? Because they said it a few times? Then if they know the risk, why do they still making it? Comes out the "some one will do it eventually, better be us 'good' people to do it first" talking point? See? It is a marketing strategy after all. These all talks, it's all to fit themselves into the "'good' people" narrative. It's a centuries old strategy to shield it's user from responsibilities while luring the support from the stupid. However, the most harmful damage, which is mass layoffs, is already partially done. This could really kill, a massive genocide even, by making people jobless and potentially incomeless. And it is shown that these tech CEOs, they don't care any bit of that beyond the point "I've already told you so".

emodendroket 3mo ago

> Whether you agree or not, we have no reason to believe their statements about risks are insincere. I'd say their pecuniary interest is a reason one might plausibly doubt their sincerity, as are their continued efforts to build and sell access to the tools.

unknownfuture 3mo ago

> The idea that AI companies scaremonger to sell models is a silly meme. > Both OpenAI and Anthropic were founded by people who sincerely believed in the risk of out-of-control superintelligence. This is not a contradiction. These things can all be true: 1. That they were afraid of ASI 2. That they continue to be afraid of ASI 3. That they recognize that LLMs aren't in fact a path to ASI 4. That the current models aren't the existential danger they'd have us believe 5. That they're claiming they are because it makes for good marketing

SamDc73 3mo ago

> OpenAI and Anthropic were founded by people who sincerely believed in the risk of out-of-control superintelligence Why would they sell there services to Palantir and/or to the military then?

polski-g 3mo ago
Because the US military and it's contractors are good? "Why did they open an orphanage instead of pouring acid into a town water tower?!?"

vitalyan1234 3mo ago

yes, yes, and Apple forbids sideloading because they're worried about grannies installing malware.

jatora 3mo ago

No reason except what comes from a bit of critical thinking. What do they stand to gain by fearmongering their models as powerful threats? Clout, funding, fanfare, discussion, limelight, funding, funding, stronger IPO, valuation, funding. What cybersecurity threshold was crossed by mythos that wasn't already crossed by 4.8/5.5? Crickets from 99% of those who have had access. Have they pulled the same stunts multiple times before with previous models? Check. You're blind if you dont think that greed and marketing are behind most things you see and hear about when gigantic corporations are involved. I don't think anthropic or OAI are evil, but its clear both have contracts/connections with Dod and/or Palantir. Both are powered largely by greed still. If you actually want an example of these sincere founders you think OAI/anthropic are run by... look at Ilya at SSI or something. Please open your eyes and stop spreading your opinions on things you clearly have no clue about.

roncesvalles 3mo ago

Of all the frontier labs, Anthropic has been the most creative in its marketing. I really, really don't put it beyond them for this to be one big crazy stunt. Besides, when has the US government been known to do things like this proactively? The phone call came from inside the house.

aurelius_44 3mo ago

OpenAI was, but didn't all those people then leave for Anthropic?

altmanaltman 3mo ago

Look into the history of Sam Altman and his ventures. He sincerely only believes in lying about his companies to investors like he did with Loopt. He is not even a researcher or an engineer. And Dario broke up with OpenAI and founded Anthropic because he didn't like Sam's and OpenAI's vision. "founded by people who believe..." is doing a lot of work and it is hard to believe that in ernest given the sketchy past of the same people. Most original higher up people who cared about safety and allignment in OpenAI have left.

geraneum 3mo ago

Ah yes we are being told to believe in sincerity of people running trillion dollar corporations.

tomjen3 3mo ago

I am biased but I think if they really believed what they were saying, they would be a lot more humble about it. I suspect what happened is the classic problem: you were sincere, then someone showed you a pile of money bigger than you could possibly imagine and you started to make excuses - Anthropic has a lot of EA people, so the excuse "Imagine how many lives you could save with this much money"[0] is very tempting, especially if all you have to do is diverge slightly from your plan. [0]: the excuse is even true! You can get a lot of malaria nets/vaccines for 1 million.

ai_brain_rot 3mo ago

I believe some people at these companies are sincere sure, but the CEOs, the investors, your sam altmans etc, the marketing talk. Hell no, can’t trust a single word.

_heimdall 3mo ago

I put very little weight behind any of those fearful statements made years ago. I assume leaders of those companies are fairly smart and rational, and there's no rational explanation for them running companies building the very thing they (supposedly) genuinely believe could kill us all. I don't doubt they may have held genuine fears in the past, but those are long gone by now.

epolanski 3mo ago

Their marketing reinforces the previous post though. Going for months claiming how good mythos is for security cuz too powerful is their own making.

xg15 3mo ago

> Both OpenAI and Anthropic were founded by people who sincerely believed in the risk of out-of-control superintelligence. And then somehow came to conclusion that the only way to address that risk was to go ahead and spend a gigantic amount of effort and resources to build exactly that superintelligence...

0xpgm 3mo ago

If I remember correctly, the original GPT was considered too dangerous to release to the wild. With hindsight, does that hold? If not, then how would we know a model is truly dangerous to release?

pms 3mo ago

Just like Trump sincerely believed he won the 2020 election.

catlifeonmars 3mo ago

These can be both true: - there are people who are sincerely concerned about model safety who work at OpenAI and Anthropic - there are people who are using this concern to generate fear to sell a product who work at OpenAI and Anthropic It makes sense to be cognizant of the apparent conflict of interest and not take things that at face value, especially when there is so much money involved.