8 ms·

AI Is a Harsh Mistress

57 points by fscaramuzza 22d ago

sebmellen 22d ago
I’ve been reading The Moon Is a Harsh Mistress lately and I’ve been shocked at how prescient it is. I’m kind of surprised I haven’t read more about it in the context of today’s AI.
- Grambo 22d ago
  I just finished this yesterday. What’s funny is it has a super intelligent, conscious AI but it’s still programmed via paper tape.
  - jerf 22d ago
    I still really enjoy the science fiction of that age. It's fun to read stories set in a universe where computers basically never came into existence. It's really hard for anyone to write stories like that now, so, kind of like analogy to "low-background steel" [1] that has been used recently of the pre-AI world, there's a way in which no more of this sort of story can be produced anymore, or at least, very little. Fortunately there's already more than you can read. But it is also funny to read about "robots" in a world where computers are still blinking lights and punchcards, if they exist at all. Some of Asimov's work took it to the logical, yet still absurd, conclusion - computers are still blinking lights and punchcards but hyperadvanced and very massive robotic positronic brains are built to run the entire world at once... basically the robots become so advanced that their brains are repurposed as computers. Very amusing in its own way. Fallout is probably the biggest example of this I can think of. Complete with the robots that are absurdly more capable than the computer hardware seems to be, and the Yes Man who at least aspires to run his part of the world. [1]: https://en.wikipedia.org/wiki/Low-background_steel https://en.wikipedia.org/wiki/Low-background_steel
    - deleted 21d ago
      [deleted]
    - DonHopkins 21d ago
      Stanislaw Lem's robots are timeless! Marcin Wichary made the Google Doodle about Stanislaw Lem: 60th Anniversary of Stanislaw Lem's First Book Publication: https://doodles.google/doodle/60th-anniversary-of-stanislaw-lems-first-publication/ https://doodles.google/doodle/60th-anniversary-of-stanislaw-... See the interactive version here: https://www.google.com/logos/lem/ https://www.google.com/logos/lem/ He also co-created the first interactive Google Doodle -- Pac-Man, 2010. Lem's Cyberiad has at least two Sallys that matter to hackers: The Seventh Sally (or How Trurl's Own Perfection Led to No Good) is the story Will Wright says inspired SimCity -- a simulated kingdom in a box. The First Sally (A), or Trurl's Electronic Bard, is about a robot that makes poetry. Lem wrote in Polish; Michael Kandel translated it to English, including constrained poems that are somehow excellent. The Bard is commissioned to write "A love poem, lyrical, pastoral, and expressed in the language of pure mathematics. Tensor algebra mainly, with a little topology and higher calculus, if need be. But with feeling, you understand, and in the cybernetic spirit." Marcin Wichary is not just a doodler -- in 2015 he translated Lem's 1976 story "One hundred and thirty-seven seconds" into English and wrote about the translation: https://aresluna.org/one-hundred-and-thirty-seven-seconds/ https://aresluna.org/one-hundred-and-thirty-seven-seconds/ https://mwichary.medium.com/translating-a-stanislaw-lem-story-6c2446632bd8 https://mwichary.medium.com/translating-a-stanislaw-lem-stor... In March 2020 I wrote to him, asking where to find the Polish original of the Electronic Bard's mathematical love poem. I'd love to compare Kandel's English to the Polish even though I don't know the language. I sent him my old web pages collecting the Bard's wonderful and horrible poems: https://www.donhopkins.com/home/catalog/lem/WonderfulPoems.html https://www.donhopkins.com/home/catalog/lem/WonderfulPoems.h... https://www.donhopkins.com/home/catalog/lem/HorriblePoems.html https://www.donhopkins.com/home/catalog/lem/HorriblePoems.ht... Marcin Wichary => Don Hopkins: Thanks for reaching out and reading! You can find the original Polish poem here: https://web.archive.org/web/20190814065711/http://niniwa22.cba.pl/lem_elektrybalt.htm https://web.archive.org/web/20190814065711/http://niniwa22.c... That one link answered the question I'd wondered about for decades. Then I discovered we both had the same fear of spoiling retro games like PacMan. I ported SimCity to Unix with a multiplayer interface, and had the same respect and trepidation about changing something classic. I made multiplayer SimCity cooperative instead of competitive -- competition would spoil what was great about SimCity. We're all living in the same city/planet and we have to learn how to get along. Oh yeah, back to the point: here's SimCity with PacMan! I hated all the heavy traffic in SimCity, so I created the Church of PacMania, which worships PacBot: a gargantuan yellow road-following car-devouring Pac-Man. The church generates traffic to attract PacBot and sacrificially feed its followers to god -- an automotive version of PKD's "Rautavaara's Case", where god eats people instead of the other bizarre upside-down way around, Catholic style Transubstantiation where humans eat their god instead. https://en.wikipedia.org/wiki/Rautavaara%27s_Case https://en.wikipedia.org/wiki/Rautavaara%27s_Case Micropolis Online (SimCity) Web Demo -- PacBot eats traffic: https://www.youtube.com/watch?v=8snnqQSI0GE https://www.youtube.com/watch?v=8snnqQSI0GE Multi Player SimCityNet for X11 on Linux: https://www.youtube.com/watch?v=_fVl4dGwUrA https://www.youtube.com/watch?v=_fVl4dGwUrA Pie menus in SimCity: https://www.youtube.com/watch?v=Jvi98wVUmQA https://www.youtube.com/watch?v=Jvi98wVUmQA Also Lem translation related: the Washington Post on new MIT Press translations, which had just come out that month (although I've since canceled my subscription because of how Bezos corrupted and destroyed that once great institution -- so here's an archive link): https://web.archive.org/web/20200831203319/https://www.washingtonpost.com/entertainment/books/stanislaw-lem-has-finally-gotten-the-translations-his-genius-deserves/2020/03/03/3227b18c-5982-11ea-ab68-101ecfec2532_story.html https://web.archive.org/web/20200831203319/https://www.washi...
- datakan 22d ago
  Have you read Ian Banks Culture series? It's kind of a best case scenario of AI in the far future.
  - noir_lord 22d ago
    The Culture is what I'd want to happen, the Polity (from Neal Asher) is closer to what I'd expect to happen. Both are good series if you haven't read one or the other (or either) though I'd say Banks was the better writer, Asher world builds better than anyone I can think of in contemporary science fiction though, Spatterjay is a brilliantly realised genuinely alien feeling world and the Prador are superb "villains" (from a human perspective, totally rational from their own).
    - deater 22d ago
      even some of Banks' stuff starts to get a bit dated. I was re-reading Consider Phlebas the other day, and in one sequence he snuck into a high-security concert type event and he's making narrow escapes because concert security is humans checking for paper tickets. And while I am sure you could come up with an in-universe reason for why a super-technological somewhat morally ambiguous future event with pervasive technology would be relying on 1980s levels of surveillance, I think it's just funny how people not that long in the past couldn't easily imagine how willing we'd be to overcomplicate things like that.
      - WorldMaker 22d ago
        I think Banks would argue that books like Consider Phlebas are set on the border planets and outskirts of the Culture for many reasons, including keeping the books relatable to the audiences of their time. The in-universe explanation is usually "backwater planet" across a lot of the Culture novels.
  - alasdair_ 22d ago
    I still think Iain Banks was the best sf writer ever to have existed. The Player of Games is an easy read if anyone wants to try him for the first time.
- lubujackson 22d ago
  I have been re-reading Children of the Mind (sequel to Ender's Game) and I had forgotten about Jane, who is an AI that talks to Ender through his earpiece, seeing and hearing everything he does. Jane proactively searches things for him, manages his finances and offers advice, as well as friendly banter. In the novel, the AI is treated like an AGI that treats only him this way, but I realized how very close we are to that reality right now, for everyone. All of the use cases and integrations for AI seem to be leading toward a future exactly like this. It was interesting to see how reliant he became on his AI (couldn't use a computer, didn't know his own password) - it is barely scifi at this point, and terrifying to see how close that future is, how useful it would be, and how dangerous. The Internet has globalized/flattened elements of culture very quickly, and AI will only accelerate that. And knowing how vulnerable humans are to suggestion/advertisement/propaganda, it seems like a perfect recipe for disaster. From an evolutionary standpoint, monoculture provides certain short-term advantages but centralizes weaknesses. If nothing else, I think we are in for a bumpy ride, whether or not we continue to boil the ocean.
- __s 22d ago
  Was recently explaining to my father how the free AI people interact with is completely different than the AI products we're burning cash with. That instead of hitting some critical mass of transistors we hit a critical mass of parameters which we then loop into themselves for reasoning Other old sci-fi I've been thinking of is The Three Stigmata of Palmer Eldritch. Where claude is chew-z with its alien intelligence & when you start interacting with some code or writing you start noticing that the writer has steel teeth & feel like "damnit, this is palmer eldritch blogging again"
- Nicook 22d ago
  there's a lot of ill will against the author in the current day, I suspect thats at least partially to blame.
- WorldMaker 22d ago
  The second half of the book is about an AI leading unionization efforts [0] and it amuses how often I get downvoted on HN for pointing out that if we are on a path towards AGI, things like worker's rights would be important to get right before an AGI gets involved because an AGI would want that, too. It's a classic scifi trope and a reason for the word "robot" to have been invented in fiction in the first place (as a metaphor for slave labor in Rossum's Universal Robots). [0] A lot of people don't believe me, given Heinlein's libertarian reputation and unreliable libertarian narrators, when I say The Moon Is a Harsh Mistress was one of the most socialist books I read at an impressionable age. I find it useful to note here that canonically The Cat Who Walks Through Walls starts on a parody moon that is the libertarian dystopia that people credit The Moon Is a Harsh Mistress as being/admiring while when the main character visits "Mike's Moon" later in its timeline it is a relatively socialist paradise. I've always felt that a lot more of Heinlein needs to be read like Vonnegut, but I realize I am weird like that.
matherial 22d ago
This appears to be a 100% AI-generated essay, not the first from ACM and not the last. Your call if you want to spend your day reading and arguing with the article; the author presumably didn't.
- tantalor 22d ago
  > We believe that this entire text is AI. > 100% of this text is AI https://www.pangram.com/history/f8a0cf46-1d3d-4fea-a083-d88c981e32c1?ucc=Ahdu3oyBleW https://www.pangram.com/history/f8a0cf46-1d3d-4fea-a083-d88c...
- aureate 22d ago
  I suppose there's a certain novelty in reading AI argue against AI personhood.
  - layer8 22d ago
    Not really, I remember LLMs already doing that in 2023.
- VCFundedGenYer 22d ago
  Was easy to tell from the literal sub-heading of the headline. Every other word is load-bearing.
deleted 22d ago
[deleted]
GPerson 22d ago
“Much of today’s discourse on AI consciousness rests not on empirical evidence but on functionalism …” In my view there can never be an empirical test for consciousness, because functionalism is wrong. If functionalism was right then there would be one. There’s no third party test for a fundamentally first person phenomenon.
- qarl2 22d ago
  > There’s no third party test for a fundamentally first person phenomenon. Agreed. But that also means I have no empirical evidence that YOU are conscious either.
  - GPerson 22d ago
    Then we agree. You don’t have empirical evidence that I am conscious.
  - FeepingCreature 22d ago
    Any argument that would lead us to discard 99.9999999% of the phenomenon under debate, no matter how reasonable, is certainly not useful. That's why I hold with the philosophical stance of "consciousness is the thing that makes humans talk about consciousness." Might as well ask, "what if reality doesn't exist?" The point of thought is to analyze reality. If your thinking leads you to discard the subject of the matter, well, feel free to do that but I don't see why anyone else should take it seriously.
    - qarl2 22d ago
      YES - I agree entirely. But how are we going to escape solipsism? These questions need a practical solution, not a perfect one. That's the problem. We need to stop chasing the perfect one.
      - GPerson 22d ago
        I agreed with you earlier that there’s no empirical test against solipsism. So the escape is just to accept not all beliefs worth holding are empirically verifiable. I deny solipsism because I believe there is a universe and things in it obey some kind regular patterns. You have a brain similar to mine, and similar to other animals. I believe consciousness has something to do with the brain, so I believe you are conscious.
cdrnsf 22d ago
AI isn't sentient, but it can generate text and people will anthropomorphize it. TESCREAL cultists evangelizing otherwise are best ignored.
- krunck 22d ago
  Thanks for the reference: https://en.wikipedia.org/wiki/TESCREAL https://en.wikipedia.org/wiki/TESCREAL
lutusp 22d ago
Apart from the article's probable origin in AI, it makes an elaborate, wordy argument about consciousness without trying to define the term. That's just as well, since consciousness is philosophy's "hard problem," meaning no one has successfully defined it in a way that leads to agreement. Each of the article's arguments that AI "consciousness" is actually a weak imitation of the "real thing," can with equal justice be applied to people, and the classic Turing Test method to distinguish machines from people has long since lost its authority. And a future event, one in which AI sprints past humans in its willingness to absorb printed matter (once called "reading") and then put it to use, is ... wait ... umm, never mind, the point I wanted to make is undermined by the fact that this has already happened. Hard to keep up these days.
entropyneur 22d ago
The idea that we are in danger of erroneously ascribing consciousness to AI sounds wild to me. Humans always assume whatever is convenient about other beings' (lack of) subjective experience as long as we have power over those beings. In this case the latter might no longer hold, but then it won't matter one bit what we think.
xg15 22d ago
> Consciousness, in Chalmers’ usage, is subjective experience: there is something it is like to be a conscious system. Still, nothing in the Transformer architecture or in next-word prediction shows that such an inner point of view exists. Self-reports (“I am sentient”) and fluent dialogue are weak evidence, since these models are trained on vast corpora saturated with human first-person statements. This type of verbal report works for people because it is anchored in shared biology and behavior, however, for machines, it is an echo of training data. I don't want to argue here that LLMs have consciousness, but this kind of argument annoys me because it feels like an intellectual slight of hand on its own. There is no definite way (so far) to prove that anything besides oneself has qualia or a subjective experience. So arguing that we cannot observe LLMs having subjective experience, therefore they cannot have any is cheap - you cannot observe that for animals or for the person sitting next to you either. The part that self-reports are useless and LLMs sound human because they emit sequences from human training data is correct. But the article implies some sort of fundamental distinction between machines and conscious beings that can never be crossed. And I don't see any reason to assume that.
krunck 22d ago
[dead]

57 points by fscaramuzza 22d ago

I’ve been reading The Moon Is a Harsh Mistress lately and I’ve been shocked at how prescient it is. I’m kind of surprised I haven’t read more about it in the context of today’s AI.

Grambo 22d ago
I just finished this yesterday. What’s funny is it has a super intelligent, conscious AI but it’s still programmed via paper tape.
- jerf 22d ago
  I still really enjoy the science fiction of that age. It's fun to read stories set in a universe where computers basically never came into existence. It's really hard for anyone to write stories like that now, so, kind of like analogy to "low-background steel" [1] that has been used recently of the pre-AI world, there's a way in which no more of this sort of story can be produced anymore, or at least, very little. Fortunately there's already more than you can read. But it is also funny to read about "robots" in a world where computers are still blinking lights and punchcards, if they exist at all. Some of Asimov's work took it to the logical, yet still absurd, conclusion - computers are still blinking lights and punchcards but hyperadvanced and very massive robotic positronic brains are built to run the entire world at once... basically the robots become so advanced that their brains are repurposed as computers. Very amusing in its own way. Fallout is probably the biggest example of this I can think of. Complete with the robots that are absurdly more capable than the computer hardware seems to be, and the Yes Man who at least aspires to run his part of the world. [1]: https://en.wikipedia.org/wiki/Low-background_steel https://en.wikipedia.org/wiki/Low-background_steel
  - deleted 21d ago
    [deleted]
  - DonHopkins 21d ago
    Stanislaw Lem's robots are timeless! Marcin Wichary made the Google Doodle about Stanislaw Lem: 60th Anniversary of Stanislaw Lem's First Book Publication: https://doodles.google/doodle/60th-anniversary-of-stanislaw-lems-first-publication/ https://doodles.google/doodle/60th-anniversary-of-stanislaw-... See the interactive version here: https://www.google.com/logos/lem/ https://www.google.com/logos/lem/ He also co-created the first interactive Google Doodle -- Pac-Man, 2010. Lem's Cyberiad has at least two Sallys that matter to hackers: The Seventh Sally (or How Trurl's Own Perfection Led to No Good) is the story Will Wright says inspired SimCity -- a simulated kingdom in a box. The First Sally (A), or Trurl's Electronic Bard, is about a robot that makes poetry. Lem wrote in Polish; Michael Kandel translated it to English, including constrained poems that are somehow excellent. The Bard is commissioned to write "A love poem, lyrical, pastoral, and expressed in the language of pure mathematics. Tensor algebra mainly, with a little topology and higher calculus, if need be. But with feeling, you understand, and in the cybernetic spirit." Marcin Wichary is not just a doodler -- in 2015 he translated Lem's 1976 story "One hundred and thirty-seven seconds" into English and wrote about the translation: https://aresluna.org/one-hundred-and-thirty-seven-seconds/ https://aresluna.org/one-hundred-and-thirty-seven-seconds/ https://mwichary.medium.com/translating-a-stanislaw-lem-story-6c2446632bd8 https://mwichary.medium.com/translating-a-stanislaw-lem-stor... In March 2020 I wrote to him, asking where to find the Polish original of the Electronic Bard's mathematical love poem. I'd love to compare Kandel's English to the Polish even though I don't know the language. I sent him my old web pages collecting the Bard's wonderful and horrible poems: https://www.donhopkins.com/home/catalog/lem/WonderfulPoems.html https://www.donhopkins.com/home/catalog/lem/WonderfulPoems.h... https://www.donhopkins.com/home/catalog/lem/HorriblePoems.html https://www.donhopkins.com/home/catalog/lem/HorriblePoems.ht... Marcin Wichary => Don Hopkins: Thanks for reaching out and reading! You can find the original Polish poem here: https://web.archive.org/web/20190814065711/http://niniwa22.cba.pl/lem_elektrybalt.htm https://web.archive.org/web/20190814065711/http://niniwa22.c... That one link answered the question I'd wondered about for decades. Then I discovered we both had the same fear of spoiling retro games like PacMan. I ported SimCity to Unix with a multiplayer interface, and had the same respect and trepidation about changing something classic. I made multiplayer SimCity cooperative instead of competitive -- competition would spoil what was great about SimCity. We're all living in the same city/planet and we have to learn how to get along. Oh yeah, back to the point: here's SimCity with PacMan! I hated all the heavy traffic in SimCity, so I created the Church of PacMania, which worships PacBot: a gargantuan yellow road-following car-devouring Pac-Man. The church generates traffic to attract PacBot and sacrificially feed its followers to god -- an automotive version of PKD's "Rautavaara's Case", where god eats people instead of the other bizarre upside-down way around, Catholic style Transubstantiation where humans eat their god instead. https://en.wikipedia.org/wiki/Rautavaara%27s_Case https://en.wikipedia.org/wiki/Rautavaara%27s_Case Micropolis Online (SimCity) Web Demo -- PacBot eats traffic: https://www.youtube.com/watch?v=8snnqQSI0GE https://www.youtube.com/watch?v=8snnqQSI0GE Multi Player SimCityNet for X11 on Linux: https://www.youtube.com/watch?v=_fVl4dGwUrA https://www.youtube.com/watch?v=_fVl4dGwUrA Pie menus in SimCity: https://www.youtube.com/watch?v=Jvi98wVUmQA https://www.youtube.com/watch?v=Jvi98wVUmQA Also Lem translation related: the Washington Post on new MIT Press translations, which had just come out that month (although I've since canceled my subscription because of how Bezos corrupted and destroyed that once great institution -- so here's an archive link): https://web.archive.org/web/20200831203319/https://www.washingtonpost.com/entertainment/books/stanislaw-lem-has-finally-gotten-the-translations-his-genius-deserves/2020/03/03/3227b18c-5982-11ea-ab68-101ecfec2532_story.html https://web.archive.org/web/20200831203319/https://www.washi...
datakan 22d ago
Have you read Ian Banks Culture series? It's kind of a best case scenario of AI in the far future.
- noir_lord 22d ago
  The Culture is what I'd want to happen, the Polity (from Neal Asher) is closer to what I'd expect to happen. Both are good series if you haven't read one or the other (or either) though I'd say Banks was the better writer, Asher world builds better than anyone I can think of in contemporary science fiction though, Spatterjay is a brilliantly realised genuinely alien feeling world and the Prador are superb "villains" (from a human perspective, totally rational from their own).
  - deater 22d ago
    even some of Banks' stuff starts to get a bit dated. I was re-reading Consider Phlebas the other day, and in one sequence he snuck into a high-security concert type event and he's making narrow escapes because concert security is humans checking for paper tickets. And while I am sure you could come up with an in-universe reason for why a super-technological somewhat morally ambiguous future event with pervasive technology would be relying on 1980s levels of surveillance, I think it's just funny how people not that long in the past couldn't easily imagine how willing we'd be to overcomplicate things like that.
    - WorldMaker 22d ago
      I think Banks would argue that books like Consider Phlebas are set on the border planets and outskirts of the Culture for many reasons, including keeping the books relatable to the audiences of their time. The in-universe explanation is usually "backwater planet" across a lot of the Culture novels.
- alasdair_ 22d ago
  I still think Iain Banks was the best sf writer ever to have existed. The Player of Games is an easy read if anyone wants to try him for the first time.
lubujackson 22d ago
I have been re-reading Children of the Mind (sequel to Ender's Game) and I had forgotten about Jane, who is an AI that talks to Ender through his earpiece, seeing and hearing everything he does. Jane proactively searches things for him, manages his finances and offers advice, as well as friendly banter. In the novel, the AI is treated like an AGI that treats only him this way, but I realized how very close we are to that reality right now, for everyone. All of the use cases and integrations for AI seem to be leading toward a future exactly like this. It was interesting to see how reliant he became on his AI (couldn't use a computer, didn't know his own password) - it is barely scifi at this point, and terrifying to see how close that future is, how useful it would be, and how dangerous. The Internet has globalized/flattened elements of culture very quickly, and AI will only accelerate that. And knowing how vulnerable humans are to suggestion/advertisement/propaganda, it seems like a perfect recipe for disaster. From an evolutionary standpoint, monoculture provides certain short-term advantages but centralizes weaknesses. If nothing else, I think we are in for a bumpy ride, whether or not we continue to boil the ocean.
__s 22d ago
Was recently explaining to my father how the free AI people interact with is completely different than the AI products we're burning cash with. That instead of hitting some critical mass of transistors we hit a critical mass of parameters which we then loop into themselves for reasoning Other old sci-fi I've been thinking of is The Three Stigmata of Palmer Eldritch. Where claude is chew-z with its alien intelligence & when you start interacting with some code or writing you start noticing that the writer has steel teeth & feel like "damnit, this is palmer eldritch blogging again"
Nicook 22d ago
there's a lot of ill will against the author in the current day, I suspect thats at least partially to blame.
WorldMaker 22d ago
The second half of the book is about an AI leading unionization efforts [0] and it amuses how often I get downvoted on HN for pointing out that if we are on a path towards AGI, things like worker's rights would be important to get right before an AGI gets involved because an AGI would want that, too. It's a classic scifi trope and a reason for the word "robot" to have been invented in fiction in the first place (as a metaphor for slave labor in Rossum's Universal Robots). [0] A lot of people don't believe me, given Heinlein's libertarian reputation and unreliable libertarian narrators, when I say The Moon Is a Harsh Mistress was one of the most socialist books I read at an impressionable age. I find it useful to note here that canonically The Cat Who Walks Through Walls starts on a parody moon that is the libertarian dystopia that people credit The Moon Is a Harsh Mistress as being/admiring while when the main character visits "Mike's Moon" later in its timeline it is a relatively socialist paradise. I've always felt that a lot more of Heinlein needs to be read like Vonnegut, but I realize I am weird like that.

matherial 22d ago

This appears to be a 100% AI-generated essay, not the first from ACM and not the last. Your call if you want to spend your day reading and arguing with the article; the author presumably didn't.

tantalor 22d ago
> We believe that this entire text is AI. > 100% of this text is AI https://www.pangram.com/history/f8a0cf46-1d3d-4fea-a083-d88c981e32c1?ucc=Ahdu3oyBleW https://www.pangram.com/history/f8a0cf46-1d3d-4fea-a083-d88c...
aureate 22d ago
I suppose there's a certain novelty in reading AI argue against AI personhood.
- layer8 22d ago
  Not really, I remember LLMs already doing that in 2023.
VCFundedGenYer 22d ago
Was easy to tell from the literal sub-heading of the headline. Every other word is load-bearing.

deleted 22d ago

[deleted]

GPerson 22d ago

“Much of today’s discourse on AI consciousness rests not on empirical evidence but on functionalism …” In my view there can never be an empirical test for consciousness, because functionalism is wrong. If functionalism was right then there would be one. There’s no third party test for a fundamentally first person phenomenon.

qarl2 22d ago
> There’s no third party test for a fundamentally first person phenomenon. Agreed. But that also means I have no empirical evidence that YOU are conscious either.
- GPerson 22d ago
  Then we agree. You don’t have empirical evidence that I am conscious.
- FeepingCreature 22d ago
  Any argument that would lead us to discard 99.9999999% of the phenomenon under debate, no matter how reasonable, is certainly not useful. That's why I hold with the philosophical stance of "consciousness is the thing that makes humans talk about consciousness." Might as well ask, "what if reality doesn't exist?" The point of thought is to analyze reality. If your thinking leads you to discard the subject of the matter, well, feel free to do that but I don't see why anyone else should take it seriously.
  - qarl2 22d ago
    YES - I agree entirely. But how are we going to escape solipsism? These questions need a practical solution, not a perfect one. That's the problem. We need to stop chasing the perfect one.
    - GPerson 22d ago
      I agreed with you earlier that there’s no empirical test against solipsism. So the escape is just to accept not all beliefs worth holding are empirically verifiable. I deny solipsism because I believe there is a universe and things in it obey some kind regular patterns. You have a brain similar to mine, and similar to other animals. I believe consciousness has something to do with the brain, so I believe you are conscious.

cdrnsf 22d ago

AI isn't sentient, but it can generate text and people will anthropomorphize it. TESCREAL cultists evangelizing otherwise are best ignored.

krunck 22d ago
Thanks for the reference: https://en.wikipedia.org/wiki/TESCREAL https://en.wikipedia.org/wiki/TESCREAL

lutusp 22d ago

Apart from the article's probable origin in AI, it makes an elaborate, wordy argument about consciousness without trying to define the term. That's just as well, since consciousness is philosophy's "hard problem," meaning no one has successfully defined it in a way that leads to agreement. Each of the article's arguments that AI "consciousness" is actually a weak imitation of the "real thing," can with equal justice be applied to people, and the classic Turing Test method to distinguish machines from people has long since lost its authority. And a future event, one in which AI sprints past humans in its willingness to absorb printed matter (once called "reading") and then put it to use, is ... wait ... umm, never mind, the point I wanted to make is undermined by the fact that this has already happened. Hard to keep up these days.

entropyneur 22d ago

The idea that we are in danger of erroneously ascribing consciousness to AI sounds wild to me. Humans always assume whatever is convenient about other beings' (lack of) subjective experience as long as we have power over those beings. In this case the latter might no longer hold, but then it won't matter one bit what we think.

xg15 22d ago

> Consciousness, in Chalmers’ usage, is subjective experience: there is something it is like to be a conscious system. Still, nothing in the Transformer architecture or in next-word prediction shows that such an inner point of view exists. Self-reports (“I am sentient”) and fluent dialogue are weak evidence, since these models are trained on vast corpora saturated with human first-person statements. This type of verbal report works for people because it is anchored in shared biology and behavior, however, for machines, it is an echo of training data. I don't want to argue here that LLMs have consciousness, but this kind of argument annoys me because it feels like an intellectual slight of hand on its own. There is no definite way (so far) to prove that anything besides oneself has qualia or a subjective experience. So arguing that we cannot observe LLMs having subjective experience, therefore they cannot have any is cheap - you cannot observe that for animals or for the person sitting next to you either. The part that self-reports are useless and LLMs sound human because they emit sequences from human training data is correct. But the article implies some sort of fundamental distinction between machines and conscious beings that can never be crossed. And I don't see any reason to assume that.

krunck 22d ago

[dead]