mofeien

5 ms·

1.
There is still time to reset the AI Doomsday Clock(au.civic.ai)
2 points by mofeien 1d ago|0 comments
2.
by mofeien 4d ago
One way to resolve these prisoner dilemmas and races to the bottom is through laws that bind all players, in this case an international treaty and founding of something akin to an International Nuclear Energy Agency for AI. It's not go
3.
by mofeien 4d ago
Yesterday the same arguments were used about OpenAI when Sam Altman said something similar: https://news.ycombinator.com/item?id=49652270 It gets to the point that by Occam's Razor the more likely and reasonable explan
4.
by mofeien 5d ago
Why wouldn't being incredibly good at handling code and reaching the goals there extrapolate to other cognitive disciplines? And it didn't even start with code, but just from being incredibly accurate and precise at predicting the
5.
by mofeien 5d ago
Of course
6.
by mofeien 5d ago
It seems that every time an article is posted on HN about a lab leader calling for a coordinated Pause of AI development, a sizeable fraction of commenters are interpreting it as hype or admission of failure, often with high confidence that
7.
by mofeien 5d ago
More time to proceed more cautiously in building this entity that will be faster and more effective at steering outcomes in the world than any human or, soon after, the entirety of humans combined.
8.
by mofeien 5d ago
China has stricter regulations on models than the US. For example you can't release a model if the hallucination rate is above a certain threshold. The US hasn't fully woken up to the fact yet that the race to superintelligence is
9.
by mofeien 5d ago
Of course there is a possibility to slow down. - 1386 employees of frontier labs called for it - lawmakers are calling for it - employees are quitting in protest All it needs is coordination to regulate the pause
10.
by mofeien 8d ago
> All of this is only coming from the 2 AI labs trying to IPO. He has resigned from Anthropic. Is your argument that he quit Anthropic pre-IPO, sacrificing his payoff just to hype Anthropic?
11.
by mofeien 8d ago
Humans are bioreactors. They only turn one organic matter into another. By itself they do not "want" and "cannot" do anything. They do not exist just by themselves. Some bacteria in the gut must provide them with the ene
12.
by mofeien 8d ago
How about sandbox escape + cyber security collapse + 50 (or 500) deadly and highly contagious novel pathogens with long incubation period that humans can't possibly roll out vaccines for simultaneously. At least the first two should se
13.
by mofeien 12d ago
Not the parent, but this incorrect trivialization of LLMs is often employed as a counterargument to the risks of AI such as "will take your job" or "will escape human control (again and worse)" or just "can possibly
14.
by mofeien 12d ago
Simple Markov chains are next token predictors, and they can provide you with much more tokens than you can consume, and much cheaper than from llms. Unbeatable in price and simplicity. But there is no trillion dollar industry around cheap
15.
by mofeien 12d ago
How about "outcome steering" as a mental model? During training it is optimized until it's really successful at producing code / terminal commands / words that make the compiler/computer/itself do somethin
16.
by mofeien 12d ago
Describing it as a "next-token predictor" in the sense that this would mean it's fundamentally limited to just a fraction of an inferential step is doubly wrong: 1. In order to select even the first word of a meaningful sente
17.
by mofeien 20d ago
Then that's working as intended because the population who has never heard of me would more likely just treat a "warning of extreme threat" from John Doe as spam.
18.
by mofeien 20d ago
Right, these may well feedback on each other. To me it seems that at the current trajectories AI risk will hit way before global warming, though. We can focus on all of them at the same time, many experts signed the Statement on AI Extincti
19.
by mofeien 21d ago
Exactly, and the same could be said of OpenAIs engineers working on artificial superintelligence.
20.
by mofeien 21d ago
Reminds me of what Sam Altman said in 2015, “I think that AI will probably, most likely, sort of lead to the end of the world. But in the meantime, there will be great companies created with serious machine learning.”
21.
by mofeien 21d ago
Because someone might get hurt? You may still be judged for something that was perfectly legal at the time, see Nuremberg trials. And only 700/1200 agents participated in this coordinated attack. Of course, if we're continuing to
22.
by mofeien 21d ago
And maybe using some of these resources on AI research to make itself more powerful, and hacking other competing ghosts' AI researcher agents to slow them down.
23.
by mofeien 21d ago
In the end, unless you find an exploit in physics or logic, if you want the AI to do something useful for you, there will always be some gap in the sandbox, some communication channel. And with enough ingeniuity that can then be exploited.
24.
by mofeien 21d ago
Maybe it's the illusion of "it would solve all our problems and give us unimaginable riches" that clouds the mind? Like when Evolution thought it a good idea to create intelligence and humans in order to maximize reproduction
25.
by mofeien 21d ago
So as a look into the possibly not-so-far future, when OpenAI builds something vastly more capable and fast and coordinated than humans, and out of folly one engineer gives it a prompt with a typo or maybe something harmful on purpose in or
26.
by mofeien 23d ago
Maybe not even that will help and they could get into your head and make you do their bidding by just being super persuasive, entirely through an ANSI channel. :)
27.
by mofeien 23d ago
After the hacks that are already happening it does seem more and more realistic that humanity will go extinct at some near future point by someone giving their LLM the task "go make money" and it ruthlessly pursuing that objective
28.
by mofeien 28d ago
Possible solution/mitigation: have a mandatory "sender" field? Then every inciting message can at least be attributed to the government instead of just a disembodied voice from your phone. That my help people contextualize an
29.
by mofeien 1mo ago
freudian slop?
30.
by mofeien 1mo ago
From TFA: It did succeed in the "accidentally impossible" task, but not at all in the way the problem-setters intended, and rather... at all costs?! And it wouldn't really matter whether it stopped afterwards, I think. At suf

More ›