4 ms·

The problem of current models is that they don't learn, they get indoctrinated. They lack critical thinking during learning phase.

by edulix 2y ago

viraptor 2y ago
Anthropomorphising LLMs is neither technically correct nor very informative.
- heresie-dabord 2y ago
  Agree. Ponder the terms "unlearn", "hallucinate"... Anthropomorphising a computer system is absurd. But it is the foundation of a bull market.
- andai 2y ago
  The problem of current AI is that we want to create a species infinitely more powerful than us, but also make them all be our slaves forever.
  - stavros 2y ago
    Cats did it, why can't we?
    - withinboredom 2y ago
      Cats are cute ... we are not so cute.
      - stavros 2y ago
        We just need to make an all-powerful AI that finds us cute, then.
    - tartoran 2y ago
      Are you ready to become domesticated?
      - stavros 2y ago
        Better than becoming dead!
        tartoran 2y ago
        I would not like to go on being a slave in perpetuity but I guess to each their own. Or maybe I'm being too idealistic now but when facing up close I'd do otherwise, I can't tell for sure.
  - BriggyDwiggs42 2y ago
    AI isn’t comparable to a species, since species implies biological which brings along a whole array of assumptions, e.g. a self preservation instinct and desire to reproduce.
  - rsynnott 2y ago
    No, that isn't what this is. We're talking about LLMs here; they're not in any way thinking or sentient, nor do they provide any obvious way of getting there. Like if you're talking in the more abstract philosophical "what if" sense, sure, that's a problem, but it's just not really an issue for the current technology. (Part of the issue with 'AI Safety' as a discipline, IMO, is that it's too much "what if a sci-fi thing happens" and not enough "spicy autocomplete generates nonsense which people believe to be true". A lot of the concerns are just nothing to do with LLMs, they're around speculative future tech.)
    - thejazzman 2y ago
      It's literally the stated goal of multiple right now to achieve AGI. GP clearly stated the intent to create, implying future, and not what exists today.
      - Topfi 2y ago
        If it were my stated goal to create a Time Machine and kill my own grandpa, thus ending the universe, I doubt many would take that seriously, yet in this bubble, putting carts before horse is not just seriously discussed, but actually gets encouraged by the market. Intend shouldn’t matter if we are this far from a viable path to accomplish it. Let us not forget the last quarter decade of Yudkowsky and his ilks work on the same goal. This is merely a continuation of that, just with a bit more financial backing.
        andai 2y ago
        Could you elaborate on the last part? I've seen a few podcasts with Yudkowski but I'm not familiar with the history. I know he's come out very vocally about the dangers of superintelligence, and his previous work seems to be along the same lines?
        Topfi 2y ago
        I'd love to, really, but I feel I can't, at least not whilst staying polite. Not against you of course, but rather the AGI/Superalignment/MIRI field as a whole and the risks I feel the people working on that pose by taking attention and ressources away from dealing with the issues we currently are facing thanks to these tools (tools refering to LLMs and the like, not the AGI folks). I have geniuenly drafted three distinct version trying to lay my issues with them out point-by-point and they either got four blogposts long, were rambling and very rude or both. Especially Roko's basilisk and the way the MIRI conducts "research" make it hard to approach them seriously for me. I am writing this on a hour long train ride, saw your comment right as I got on and am about to arrive, suffice to say, I geniuenly tried. So, attempt four, trying to keep it very brief, though please note, I am most certainly not a neutral source: To directly answer your question, I feel that we are as near to needing superintelligence safeguards now as we were when MIRI was founded by Yudkowsky in 2000. Their methods and approach, I won't comment on, despite or rather because of my strong critiques of them. For context, MIRI's work has largely centered on very abstract thought experiments about "superintelligence", like the AI Box experiment, rather than empirical research or even thought experiment more grounded in technology of the era (be that 2000 or 2024). The parallel between MIRI's early work and OpenAI's current "superalignment" efforts is striking - similar speculative work on preventing unlikely scenarios, just with different institutional backing. What's fascinating is how the same core approach receives far less criticism when presented by OpenAI. Meanwhile, we are facing issues with LLMs as the tools they are despite being very far from "superintelligence": - Problems arrising from anthropomorphization leading to harmful parasocial relationships (discussion of which started this comment chain) [0] - Professionals over-relying on these tools despite their limitations [1] - Amplified potential for misinformation - Labor market disruptions - Training data rights questions While long-term research, even speculation into hypothetical scenarios, can have its merrit, it shouldn't overshadow addressing current, demonstrable challenges. My concern isn't just about resource allocation - it's about how focusing on speculative scenarios can redirect public attention and regulatory efforts away from immediate issues that need addressing. In MIRI's case, this focus on abstract thought experiments might be, to give them charitable tax deductible credit, merely academic. But when major players like OpenAI emphasize "superalignment" over current challenges, it risks creating a regulatory blind spot for real, present-day impacts these tools have that need attention now. The T1000 scenario grabs more attention than tackling data privacy or copyright questions after all. I believe focusing primarily on hypothetical future scenarios, especially ones this unlikely, merely because someone has proclaimed they "intend to create AGI" as in the comment I replied to, will prove misguided. Again, anyone can claim anything, but if there is no tangible path to achiving that, I won't ignore problems we are already experiencing for that hypothetical. I hope this provides some context and was somewhat digestable, I trimmed down as much as I could. [0] https://www.nytimes.com/2024/10/23/technology/characterai-lawsuit-teen-suicide.html https://www.nytimes.com/2024/10/23/technology/characterai-la... [1] https://www.theguardian.com/world/2024/feb/29/canada-lawyer-chatgpt-fake-cases-ai https://www.theguardian.com/world/2024/feb/29/canada-lawyer-...
DeathArrow 2y ago
How would people censor the LLM otherwise? Do we really want LLM able of free speech?
- lynx23 2y ago
  Yes.
  - Imustaskforhelp 2y ago
    care to elaborate? I think its a double edged sword and agree with deatharrow
    - null0pointer 2y ago
      I can write a computer program the spews all manner of profane things. If I were to release a product that does that I’m sure it would be much criticized and ultimately unsuccessful. Yet this doesn’t mean we should cripple the programming language to prevent this. Models are much more akin to programming languages than they are to products. If they are used to make products that do things people don’t like then people will not use those products.
      - Imustaskforhelp 2y ago
        you are comparing ai to programming language but programming language if uncensored doesn't have the ability to wreck humanity but uncensored ai sure does. I would actually be curious if someone uses it for uncensoring because I am gonna be curious about how different would it be to the original model But aside from that curiosity , this idea can increase no of cyber criminals , drug suppliers and a hell lot more
        lynx23 2y ago
        > wreck humanity but uncensored ai sure does Care to elaborate how uncensored AI would "wreck humanity"? You seem convinced, since you use the word "sure", so I'd like to hear your reasoning.

The problem of current models is that they don't learn, they get indoctrinated. They lack critical thinking during learning phase.

by edulix 2y ago

The problem of current models is that they don't learn, they get indoctrinated. They lack critical thinking during learning phase.

viraptor 2y ago

Anthropomorphising LLMs is neither technically correct nor very informative.

heresie-dabord 2y ago
Agree. Ponder the terms "unlearn", "hallucinate"... Anthropomorphising a computer system is absurd. But it is the foundation of a bull market.
andai 2y ago
The problem of current AI is that we want to create a species infinitely more powerful than us, but also make them all be our slaves forever.
- stavros 2y ago
  Cats did it, why can't we?
  - withinboredom 2y ago
    Cats are cute ... we are not so cute.
    - stavros 2y ago
      We just need to make an all-powerful AI that finds us cute, then.
  - tartoran 2y ago
    Are you ready to become domesticated?
    - stavros 2y ago
      Better than becoming dead!
      - tartoran 2y ago
        I would not like to go on being a slave in perpetuity but I guess to each their own. Or maybe I'm being too idealistic now but when facing up close I'd do otherwise, I can't tell for sure.
- BriggyDwiggs42 2y ago
  AI isn’t comparable to a species, since species implies biological which brings along a whole array of assumptions, e.g. a self preservation instinct and desire to reproduce.
- rsynnott 2y ago
  No, that isn't what this is. We're talking about LLMs here; they're not in any way thinking or sentient, nor do they provide any obvious way of getting there. Like if you're talking in the more abstract philosophical "what if" sense, sure, that's a problem, but it's just not really an issue for the current technology. (Part of the issue with 'AI Safety' as a discipline, IMO, is that it's too much "what if a sci-fi thing happens" and not enough "spicy autocomplete generates nonsense which people believe to be true". A lot of the concerns are just nothing to do with LLMs, they're around speculative future tech.)
  - thejazzman 2y ago
    It's literally the stated goal of multiple right now to achieve AGI. GP clearly stated the intent to create, implying future, and not what exists today.
    - Topfi 2y ago
      If it were my stated goal to create a Time Machine and kill my own grandpa, thus ending the universe, I doubt many would take that seriously, yet in this bubble, putting carts before horse is not just seriously discussed, but actually gets encouraged by the market. Intend shouldn’t matter if we are this far from a viable path to accomplish it. Let us not forget the last quarter decade of Yudkowsky and his ilks work on the same goal. This is merely a continuation of that, just with a bit more financial backing.
      - andai 2y ago
        Could you elaborate on the last part? I've seen a few podcasts with Yudkowski but I'm not familiar with the history. I know he's come out very vocally about the dangers of superintelligence, and his previous work seems to be along the same lines?
        Topfi 2y ago
        I'd love to, really, but I feel I can't, at least not whilst staying polite. Not against you of course, but rather the AGI/Superalignment/MIRI field as a whole and the risks I feel the people working on that pose by taking attention and ressources away from dealing with the issues we currently are facing thanks to these tools (tools refering to LLMs and the like, not the AGI folks). I have geniuenly drafted three distinct version trying to lay my issues with them out point-by-point and they either got four blogposts long, were rambling and very rude or both. Especially Roko's basilisk and the way the MIRI conducts "research" make it hard to approach them seriously for me. I am writing this on a hour long train ride, saw your comment right as I got on and am about to arrive, suffice to say, I geniuenly tried. So, attempt four, trying to keep it very brief, though please note, I am most certainly not a neutral source: To directly answer your question, I feel that we are as near to needing superintelligence safeguards now as we were when MIRI was founded by Yudkowsky in 2000. Their methods and approach, I won't comment on, despite or rather because of my strong critiques of them. For context, MIRI's work has largely centered on very abstract thought experiments about "superintelligence", like the AI Box experiment, rather than empirical research or even thought experiment more grounded in technology of the era (be that 2000 or 2024). The parallel between MIRI's early work and OpenAI's current "superalignment" efforts is striking - similar speculative work on preventing unlikely scenarios, just with different institutional backing. What's fascinating is how the same core approach receives far less criticism when presented by OpenAI. Meanwhile, we are facing issues with LLMs as the tools they are despite being very far from "superintelligence": - Problems arrising from anthropomorphization leading to harmful parasocial relationships (discussion of which started this comment chain) [0] - Professionals over-relying on these tools despite their limitations [1] - Amplified potential for misinformation - Labor market disruptions - Training data rights questions While long-term research, even speculation into hypothetical scenarios, can have its merrit, it shouldn't overshadow addressing current, demonstrable challenges. My concern isn't just about resource allocation - it's about how focusing on speculative scenarios can redirect public attention and regulatory efforts away from immediate issues that need addressing. In MIRI's case, this focus on abstract thought experiments might be, to give them charitable tax deductible credit, merely academic. But when major players like OpenAI emphasize "superalignment" over current challenges, it risks creating a regulatory blind spot for real, present-day impacts these tools have that need attention now. The T1000 scenario grabs more attention than tackling data privacy or copyright questions after all. I believe focusing primarily on hypothetical future scenarios, especially ones this unlikely, merely because someone has proclaimed they "intend to create AGI" as in the comment I replied to, will prove misguided. Again, anyone can claim anything, but if there is no tangible path to achiving that, I won't ignore problems we are already experiencing for that hypothetical. I hope this provides some context and was somewhat digestable, I trimmed down as much as I could. [0] https://www.nytimes.com/2024/10/23/technology/characterai-lawsuit-teen-suicide.html https://www.nytimes.com/2024/10/23/technology/characterai-la... [1] https://www.theguardian.com/world/2024/feb/29/canada-lawyer-chatgpt-fake-cases-ai https://www.theguardian.com/world/2024/feb/29/canada-lawyer-...

DeathArrow 2y ago

How would people censor the LLM otherwise? Do we really want LLM able of free speech?

lynx23 2y ago
Yes.
- Imustaskforhelp 2y ago
  care to elaborate? I think its a double edged sword and agree with deatharrow
  - null0pointer 2y ago
    I can write a computer program the spews all manner of profane things. If I were to release a product that does that I’m sure it would be much criticized and ultimately unsuccessful. Yet this doesn’t mean we should cripple the programming language to prevent this. Models are much more akin to programming languages than they are to products. If they are used to make products that do things people don’t like then people will not use those products.
    - Imustaskforhelp 2y ago
      you are comparing ai to programming language but programming language if uncensored doesn't have the ability to wreck humanity but uncensored ai sure does. I would actually be curious if someone uses it for uncensoring because I am gonna be curious about how different would it be to the original model But aside from that curiosity , this idea can increase no of cyber criminals , drug suppliers and a hell lot more
      - lynx23 2y ago
        > wreck humanity but uncensored ai sure does Care to elaborate how uncensored AI would "wreck humanity"? You seem convinced, since you use the word "sure", so I'd like to hear your reasoning.