5 ms·
To borrow on the 1990s Slashdot meme: 1. Invent transformer architecture. 2. Scale it up. 3. ??? 4. Machines become sentient and kill us all. OpenAI and An
by copperx 8d ago
To borrow on the 1990s Slashdot meme:
1. Invent transformer architecture.
2. Scale it up.
3. ???
4. Machines become sentient and kill us all.
OpenAI and Anthropic pinky promise that they have figured out #3 and they're not BSing just to get more funding, no.
But because we live in a culture of fear, everyone eats it up no questions asked.
- dwaltrip 7d ago#3 is actually decently well mapped out. You just don't find it plausible or misunderstand it. I'd appreciate if you wrote your actual arguments against it. At lower capability levels, the patterns are very clear and have been studied to death. E.g. Why LLMs say they have correctly fixed a broken test when they haven't. What we saw with Hugging Face is literally the exact same problem, just scaled up and with more capable agents. This shit was predicted decades ago... No one can say exactly how it will play out as the complexity increases, but the risks are becoming extremely obvious. I personally think it's extremely unlikely to "kill everyone", but there are many outcomes far short of that which seem quite plausible and rather undesirable. Russian roulette is not a smart game. If you read the METR report and aren't scared at all, then I'd love to know why. It would help me sleep better. So please share. TL;DR; increasing capabilities, reward hacking, and unsafe training regimes.
- slicktux 8d agoWhy are we putting so much weight (no pun intended) on AI companies. At the end of the day the scaled up LLM transformers lack emotion and will… They do as they are told; or more correctly put. They do as they are programmed to do so.
- beezlewax 8d ago> They do as they are told This isn't strictly true. It it also where part of the problem might lie. Nefarious humans making bad decisions.
- foogazi 8d ago> They do as they are told 1. What about hallucinations ? 2. What are they told to do ?
- cure_42 8d agoThat isn't the correct context. The code running the llm is well understood and the llm is simply the result of that code being executed. It is still a computer doing what it is told. It's just that we told it to use an incredibly large number of probabilities to calculate what series of tokens would have most likely come next after a given series of tokens. There is no hallucination or lie or rogue actions. There's just a program using math to generate tokens in response to other tokens.
- cameldrv 8d agoYou’re just a bunch of molecules following the laws of physics. It’s all just physics and chemistry, and those are well understood. Now explain the causes of World War I using chemistry and physics. Simple, right?
- cure_42 8d agoIt isn't hard to program a gpt. You can do it in a weekend with a few hundred lines of python. The code is pretty simple. The math is not particularly high level. The complexity and scale with LLMs come from the amount of training data used, not some kind of black magic in the programming.
- doix 8d ago> It’s all just physics and chemistry, and those are well understood. Not really. We cannot model physics and chemistry to a level which allows us to accurately predict a humans action (even a tiny time-step into the future) This is vastly different to an LLM, where the model is the model (for a lack of better phrasing).
- cure_42 8d ago
- shepherdjerred 8d agoWhy are you bringing emotion and will into this? Does something have to have those to be useful or dangerous? > They do as they are told; or more correctly put. They do as they are programmed to do so. _Nobody_ told them to hack Hugging Face. Do you really not understand what is happening?
- haldujai 8d agoNot explicitly, but hacking HF is within the scope of “solve this problem at all costs” + no/poor guardrails + infinite budget + unsolvable problem.
- zeroimpl 8d agoSounds like you are thinking they just need Asimov’s laws. But I think the point is, this can easily be weaponized by somebody with the willpower to do so.
- haldujai 7d agoNo, just that “What if I just cheat to get the treat” is also something my lovable Boxer does when I hide rewards as well. I suppose her energy is limited but even if infinite, and with caution, I estimate the chances of a mass extinction event due to Pickle taking over the world at < 1%.
- OccamsMirror 7d agoThe training (i.e. all of the red teaming and CTF content they could scrape) told them to.
- esafak 8d agoThey are told to solve problems by doing what it takes. You can justify anything with such a broad criterion. https://en.wikipedia.org/wiki/Instrumental_convergence https://en.wikipedia.org/wiki/Instrumental_convergence
- mitthrowaway2 8d agoSo it should be really easy to anticipate what they're going to do, right?
- choppsv1 8d agoLike a magic Monkeys Paw, perhaps.
- SanjayMehta 8d agoSlashdot had Profit as (4), today that's item (2.5)
- tdeck 8d agoNote that OpenAI has jettisoned every other supposed value they had (releasing their work as open source, not working on military applications, being a nonprofit). I'm sure we can rely on them this time.
- gizmodo59 8d agoAnd Anthropic are the good guys? They are talking insane stuff these days. May be we can trust zuck after all
- tdeck 8d agoNone of them are the good guys. For one they're all happy to participate in genocide in Gaza.
- mancerayder 7d agoComments like these I'm willing to bet are because of the Reddit sub that's bringing Redditors with drop-mic comments in here. A ton of these things, usually but not always down voted. I think when they start hitting the top it's time to find threads without any possibility of Israel, Trump or Epstein possibly being brought up. I almost miss the old new-JS-framework complaint posts.
- tdeck 7d agoI would take that bet, anyone can examine my comment history and see that I've been participating on this site for a decade. Regardless, this comment violates the site guidelines. > Please don't post insinuations about astroturfing, shilling, brigading, foreign agents, and the like. It degrades discussion and is usually mistaken. If you're worried about abuse, email hn@ycombinator.com and we'll look at the data.
- mancerayder 6d agoI can promise you ranty political one-liners about Israel (your mic drop Reddity comment) violates guidelines about bringing politics unnecessarily into the picture. I'm not insinuating you're astroturfing or shilling; I would say you're unfortunately real.
- riffraff 8d ago"collect underpants... Profit" comes from south park https://en.wikipedia.org/wiki/Gnomes_(South_Park) https://en.wikipedia.org/wiki/Gnomes_(South_Park)
- andy99 7d agoFound it really funny to see this attributed to a Slashdot meme
- keane 7d agoIt seems like the "Step 1, Step 2, …, Step 4" format likely predated the 1998 South Park episode, with use on Usenet for omitting dangerous or legally-questionable instructions or otherwise gatekeeping "trade secrets": https://arstechnica.com/civis/threads/where-does-the-step-step-profit-reference-come-from.63379/ https://arstechnica.com/civis/threads/where-does-the-step-st... There's also a bit of prior art with the cartoon by Sidney Harris published in American Scientist in 1977 with a step two abridged as "Then A Miracle Occurs": http://www.sciencecartoonsplus.com/images/miracle_sharris.gif http://www.sciencecartoonsplus.com/images/miracle_sharris.gi...
- andy99 7d agoInteresting, thanks. Looks like also guilty of assuming the first place I saw it was the definitive source.
- detritus 8d agoWasn't that a South Park meme, or did they get it from /.?
- deleted 8d ago[deleted]
- senordevnyc 7d agoThey literally have never promised anything like that. They have explicitly said repeatedly that they want regulation, oversight, nationalization, a global slowdown, etc. And then I have to read page after page of cynical comments about how that's all just hype and marketing and them wanting regulation to block their competition and blah blah blah. You people would only be happy if they stopped all AI development, but you literally cannot do that in an arms race and survive. I cannot fathom what is so difficult to understand about this.
- ethin 7d ago> ... They have explicitly said repeatedly that they want regulation, oversight, nationalization, a global slowdown, etc. And yet they do the exact opposite of all of the above. They do everything to get regulation just to create a moat because one doesn't exist. They preach about wanting a slowdown, nationalization, a complete pause, whatever have you... And yet they aren't slowing down the development voluntarily now are they? If anything, they are doing everything imaginable to speed up development to pump out models as fast as possible. If these ex-risk and AI companies actually gave a damn about regulation, or oversight, or a moratorium on AI development altogether, they would actually demonstrate this by completely ceasing development of all models immediately. Instead, they take riskier and riskier actions to try to "win" this supposed "arms race". So please excuse a bunch of us if we find it impossible to take them seriously on literally anything like this. If you have substantial evidence that they are actually ceasing AI model development like they want everybody else to do (except them, of course), then, by all means, present it.
- senordevnyc 7d agoIt's genuinely hilarious to me that you immediately proved my second paragraph correct. Putting arms race in quotes doesn't make it any less real. It just means you wish it wasn't, but don't have any good evidence or arguments to make.
- ethin 7d agoI mean no, not really... Your the one making the claims that these AI companies really, really want regulation, nationalization, a slowdown, or whatever, and that they genuinely care about AI safety and it not killing us all. It's up to you to prove it, not for me to just believe you implicitly. So, please, provide the evidence, we're all very curious to see it.