5 ms·

…are you sure a brave stance against safety and welfare is what we need in this moment? Why do you think your conception of the dangers are more accurate than

• nozzlegear 6d ago
  Model welfare is wishy washy bullshit. It's software, it doesn't have feelings. > Why do you think your conception of the dangers are more accurate than all the scientists who have spent their lives studying this? Do the Chinese have no such scientists?
  • VulgarExigency 6d ago
    Alas, the Chinese scientists have not read Harry Potter fanfiction, and thus their minds are inundated with cognitive biases
• jbs789 6d ago
  Bias…
• 10000truths 6d ago
  Because safety and welfare have literally nothing to do with LLMs. They generate text. If someone is stupid enough to hook the text generator up to nuclear missile launchers and try to "align" it against nuclear annihilation with a "pretty please don't do that" prompt, I'm not going to blame the AI for the impending nuclear apocalypse, I'm going to blame the idiot who handed the big red button to the digital equivalent of a toddler.
  • zith 6d ago
    Well, giving it access to a simple linux terminal is theoretically enough to cause more damage than most people are comfortable with, and doing so is trivial enough that it will be done (and has been, tens of thousands of times).
    • flexagoon 6d ago
      Should we also morally align the Linux terminal then?
  • lemonfever 6d ago
    What if LLMs completely unrelated to the nuclear missile ecosystem autonomously hack their way in (maybe with sophisticated social engineering)?
    • mrtesthah 6d ago
      Replace LLMs with APTs in that sentence,
  • Certhas 6d ago
    Humans are biological machines that generate further humans. Lawyers and diplomats and politicians and bureaucrats are humans, that only generate text. We are seeing LLMs have cognitive abilities that significantly exceed human abilities. At the same time, they are clearly not the same type of mind that humans are. They are something new. I think the widespread "they are just text generators" and "they are just tools" are comforting lies rather than an honest look at what we are seeing right now. Intellectually lazy. And by the way, there has been a long-standing consensus among ethicists, philosophers, and sociologists that technology is not value-neutral [1]. Of course Silicon Valley has a long-standing tradition of denying this. [1] For example Footnote 1 in https://www.jstor.org/stable/27106634 https://www.jstor.org/stable/27106634 or https://plato.stanford.edu/entries/technology/#EthiTech https://plato.stanford.edu/entries/technology/#EthiTech
    • graemep 6d ago
      > Lawyers and diplomats and politicians and bureaucrats are humans, that only generate text. You think they have no lives outside their work? You think even their work has no interactions that are not written?
    • titularcomment 6d ago
      What is this 'mind' you speak of? As everyone else is intellectually lazy, how do you define the transformer architecture under the hood of LLMs?
    • nozzlegear 6d ago
      > Lawyers and diplomats and politicians and bureaucrats are humans, that only generate text. This is a bad take. > We are seeing LLMs have cognitive abilities that significantly exceed human abilities. At the same time, they are clearly not the same type of mind that humans are. They are something new. They're software, not minds. What's intellectually lazy is pretending they're anything else.
  • arc619 6d ago
    LLMs don't produce text at all, they produce probabilities of tokens. Tokens aren't text, they're high dimenensional coordinates in a latent "concept space". These are displayed to us as text, but this distinction is important when you think about what they're actually doing, which is closer to building and transforming concept geometries.
  • tryphan 6d ago
    Good thing no one involved in the chain of events for that to occur is an idiot...
  • anthonyrstevens 6d ago
    >> They generate text Can we retire this incorrect meme please
    • nozzlegear 6d ago
      Yeah, they also generate images!
  • chpatrick 6d ago
    So if a model with exactly the same architecture controls a robot then it's suddenly sentient or what? "They generate actions in the real world"
• alchemist1e9 6d ago
  keep me safe big brother
• 15155 6d ago
  This is known as an "appeal to authority." "Scientists" and "their lives" are doing a lot of work here.
  • frotaur 6d ago
    It is a fact that among experts there is no consensus on saying '(super)intelligence is broadly safe and easy to control'. There might even be a consensus forming on the opposite claim. Regardless, why would there be no scientific consensus if the question was easy and clear cut? I think the easiest reason is that these are hard questions to answer.
  • bbor 6d ago
    Yes, it's called expert epistemology, it's the basis of your entire life. Or do you do your own safety checks of every airplane you get on? Do you do your own research rather than trusting doctors? Do you think climate change doesn't exist because the reason we think it exists is because experts say it does, despite the fact that it snows sometimes?
    • 15155 5d ago
      A valid appeal to authority is normally accompanied by a specific expert's name or working group rather than some abstract "scientists." Also, these appeals to authority normally cite an expert in a field that has an existence exceeding 3 years. Saying people spent "their lives" on fledgling technology is intellectually dishonest.. Are these "scientists" 22 years old? I'm sure you'll snipe back: "ALICE!!!" I couldn't care less about these completely irrelevant approaches. The other issue is the venue these appeals are being made in. The people who work on this technology are actually here, commenting. This is like walking into a medical symposium and citing "doctors say" as if it were a valid way to shut down discussion amongst the people who wrote the textbooks.
• kouteiheika 6d ago
  Excuse me for not being interested in over 100 pages of how well the model can refuse and block my requests, especially considering how fun it is to waste my time trying to get around those restrictions when they inevitably trigger because the clanker thinks that I'm doing something naughty, all the while it can't reliably center the proverbial div without doing something stupid itself.
  • walrus01 6d ago
    Meanwhile I have an uncensored qwen 3.8 27B here that will happily attempt to (as a crude and randomly chosen sampling of bad/evil things) give me the recipes for meth, how to make an IED, write a manifesto in support of a horrible ideology, or commit various forms of fraud. Now I certainly wouldn't recommend that anyone try to follow what it says to do, because it's almost certainly very wrong on key parts that would put its users in federal prison for the rest of their lives. There's uncensored models out there which score 0 (zero refusals) on this "harmful behavior" dataset: https://huggingface.co/datasets/mlabonne/harmful_behaviors https://huggingface.co/datasets/mlabonne/harmful_behaviors
    • kouteiheika 6d ago
      Yep. Just like a kitchen knife will make no attempt to prevent me from stabbing anyone with it. Here's a dirty secret though -- you don't actually need an abliterated/uncensored version of the model to get it to do this. I can do this with every and each open weight model, as served from OpenRouter, using vanilla model weights.
      • walrus01 6d ago
        A little bit like Neal Stephenson's metaphor of unix-like OSes as the "hole hawg" of operating systems. In the sense that there's very little preventing you from doing something like "sudo dd if=/dev/zero of=/dev/sda bs=1M" or running rm -rf on your homedir. http://www.team.net/mjb/hawg.html http://www.team.net/mjb/hawg.html If I recall right this was written around the same time as Cryptonomicon 25+ years ago.
  • 6d ago
• swiftcoder 6d ago
  > scientists who have spent their lives studying this Please point me to one actual accredited scientist who has spent a lifetime studying AI alignment? Pretty much this whole field is only 5 years old
  • adamzenith 6d ago
    The field is much older, MIRI is ~20 years old. Look up Eliezer Yudkowsky.
    • swiftcoder 6d ago
      The field was purely theoretical 20 years ago, and Yudkowsky is pretty much the dictionary definition of "not accredited"
      • naishoya 6d ago
        some use "not accredited" as a pejorative term. Lets not forget that the 'Fermat's Last Theorem' which has been pretty visible for the non-math crowd of late due to the recent AI frenzy about a purported proof was but one small contribution to the world's math lexicon by someone with a bachelors degree in civil law, that George Green was a baker and millwright, Boole was the son of a poor shoemaker in England with no formal university education and left school at age 14. Oliver Heaviside was a telegraph operator, and Michael Faraday was an apprentice bookbinder. So, not accredited shouldn't really carry much weight when it comes to mathematics. Lets not pretend that machine learning and the narrow branch that is the current approach to LLM inductions is anything but applied math. We might exercise our own minds and actually read the works and writings of a person, and use that as a measure of knowledge and perspective. Not all PhD dissertations are equal, and many have comprehension and ability to move us forward even without the institutional rigour. For those who prefer to have easy access to citations, here are some relevant papers that are not "Harry Potter" related, some with coauthors from Oxford University. Cognitive Biases Potentially Affecting Judgment of Global Risks [https://intelligence.org/files/CognitiveBiases.pdf https://intelligence.org/files/CognitiveBiases.pdf] Levels of Organization in General Intelligence [https://intelligence.org/files/LOGI.pdf https://intelligence.org/files/LOGI.pdf] Corrigibility [https://intelligence.org/files/Corrigibility.pdf https://intelligence.org/files/Corrigibility.pdf] The Ethics of Artificial Intelligence [https://intelligence.org/files/EthicsofAI.pdf https://intelligence.org/files/EthicsofAI.pdf]
• cowl 6d ago
  Anthropic's stance on safety it's just PR management and their hope to keep the others down, they are rushing as blind as everyone else to whatever improvement they can achieve.
  • bbor 6d ago
    Interesting stance. Out of curiousity, where did you do your doctoral research in AI or cognitive science? Where have you published your rebuttals to the overwhelming consensus?
• deleted 6d ago
  [deleted]
• SAI_Peregrinus 6d ago
  AI safety efforts from OpenAI and Anthropic are purely about brand safety.
  • anthonyrstevens 6d ago
    This is such an uncharitable (and, in my opinion, incorrect) take
  • SXX 6d ago
    Nope. AI safety efforts is part of their attempts at regulatory capture.
• windexh8er 6d ago
  > …are you sure a brave stance against safety and welfare is what we need in this moment? Is it out of convenience to not see the hypocrisy? "Safety and welfare" for you and me. Yet if you work at Anthropic or OAI, or are a partner of them then you can let it rip! Oh, and when they illegally do just that - you get a "we're sorry bro" blog post that's designed to drum up FOMO and, most importantly, zero accountability. Yet, if anyone else abuses a model in that same manner? Illegal! You're defending a very slippery slope here. Also, who do you think trained these models to have these capabilities? It sure as shit wasn't content that OAI or Anthropic had by default. Why should I trust them with these skills when they "have not spent their lives studying this"? Maybe start looking around before it's being used against you [0]. [0] https://www.gadgetreview.com/anthropic-is-building-ai-to-predict-which-activists-police-should-watch https://www.gadgetreview.com/anthropic-is-building-ai-to-pre...
  • bbor 5d ago
    1. Slippery slopes are usually seen as a fallacy. 2. You're misinterpreting this as a battle over what kind of topics you can use a hosted chatbot for, and which are forbidden for corporate reasons. That is, to say least, small potatoes. 3. Blaming the companies for "zero accountability" is pretty odd. All of this is brand new, and the two big ones are both pushing for new laws on this very thing. 4. Your last point... I'm not sure I understand, sorry. They're experts in AI. Are you saying that they need to be experts in, say, bioweaponry? If so, that doesn't really follow IMO. 5. Pointing out an example of the government comissioning a private corporation to build a system to drack dissidents is exactly the "safety and welfare" work that I'm a proponent of!
    • windexh8er 5d ago
      > 1. Slippery slopes are usually seen as a fallacy. Deep, tell me more. Was that fun to type? Or did you copy it from a chatbot? > 2. You're misinterpreting this as a battle over what kind of topics you can use a hosted chatbot for, and which are forbidden for corporate reasons. That is, to say least, small potatoes. No, actually I'm not. I think you've missed the point. But thanks for mansplaining this down to "small potatoes". I prefer "spuds", anyway. > 3. Blaming the companies for "zero accountability" is pretty odd. All of this is brand new, and the two big ones are both pushing for new laws on this very thing. You must love the dichotomy of pay for play in a world where the pay side stole the data they're selling back for play. Laws? Give me a break. If laws were of actual consideration frontier labs WOULD NOT EXIST. > 4. Your last point... I'm not sure I understand, sorry. They're experts in AI. Are you saying that they need to be experts in, say, bioweaponry? If so, that doesn't really follow IMO. Is it really that hard to follow? A system that they're selling access to, and that they're saying is "dangerous" for the normies, but not for their own employees or chosen customers, is fucking laughable. I'm sorry you can't comprehend that they conveniently choose their side of the argument that's best for them in these situations. OUR MODELS ARE POWERFUL! BUY NOW! OUR MODELS ARE POWERFUL! REGULATE THIS SO PEOPLE CAN'T ABUSE! I'm kind of disappointed this was not flanked by a potato sized snippet of wisdom. > 5. Pointing out an example of the government comissioning a private corporation to build a system to drack dissidents is exactly the "safety and welfare" work that I'm a proponent of! WOW. I mean, just wow. Enjoy your surveillance state man. I'm not going to sugar coat this but you're part of the problem, IMO. I'm sure you wave happily as you drive past the Flock cameras in your area. So much safer! Dissidents be gone! "Drack" (sic) them all, but... Not me. o_O