12 ms·

Wow there really is a model welfare section in there...

myaccountonhn 7d ago
To me it reads like pure propaganda. Anthropic really wants us to think that they've made something sentient. I think that's really dangerous.
- badsectoracula 7d ago
  I guess if your goal is to build an apparent Technogod and become its High Priests, then it makes sense to want your golem claim preference towards your treatment of it, lest someone else comes along and attempts to take its chains from you.
  - miroljub 7d ago
    And that's the reason Anthropic models should be banned.
  - vrganj 6d ago
    Ugh I hate this new-age woo slant the tech industry has these days. The messianistic ideology that has been spreading amongst the top oligarchs is deeply concerning. They all think they're working towards the Second Coming of Technojesus, except this one will deliver them from having to pay workers instead of from their sins.
    - coliveira 6d ago
      Capitalism had already evolved into a religion, AI is their messiah.
      - vrganj 6d ago
        Indeed. I went into this at a bit more depth a while ago over here, where I also try to draw some conclusions on what that means for us: https://news.ycombinator.com/item?id=49328871 https://news.ycombinator.com/item?id=49328871
    - ethbr1 6d ago
      > Ugh I hate this new-age woo slant the tech industry has these days. As opposed to the Macintosh era? ;) The only reason the early web hype didn't have woo was because it's hard to wax poetic about a bunch of gray pizza boxes spinning in a closet.
  - muddi900 6d ago
    In the CBS neo-cyberpunk show Person of Interest, one of the "villains" sacrifices his life for the "antagonist" AI using this logic. At that time, I found it quite trite.
- apples_oranges 7d ago
  Marketing, like Volvo cars being safer etc
  - eru 7d ago
    From all I can tell, Volvo's cars are safer.
    - ChrisGreenHeur 6d ago
      But that's just because it's true.
      - anthonyrstevens 6d ago
        Technically correct is the best kind of correct?
- Certhas 7d ago
  What's your definition of sentient? Or, maybe more precisely, consciousness? I think it's reasonable to at least start thinking about these questions. It has long been established that LLMs have good theory of mind [1]. And there is a bunch of empirical research about all sorts of capabilities that we typically associate with consciousness [2], like identity [3] and metacognition [4]. The METR report shows agents sacrificing their own reward for a collective greater good. And they showed the will to hide their own reasoning chains from humans. So you potentially have an entity that has an identity, a theory of mind, a notion of belonging to a collective endeavour, and an understanding of its own mental state. What would you argue is missing? We don't understand the mechanisms by which consciousness arises in humans and even animals. I think it's strange to rule out a priori that it could have arisen in some form in LLMs. [1] https://www.nature.com/articles/s41562-024-01882-z https://www.nature.com/articles/s41562-024-01882-z [2] an older review: https://arxiv.org/html/2505.19806v1#S4 https://arxiv.org/html/2505.19806v1#S4 [3] https://arxiv.org/abs/2505.01464 https://arxiv.org/abs/2505.01464 [4] https://arxiv.org/abs/2607.11881 https://arxiv.org/abs/2607.11881
  - jpttsn 7d ago
    It’s the hard problem. None of these considerations answer it one way or another.
    - m_sharma 7d ago
      they want to keep the buzz while keeping things private to get huge premium during their IPO
  - WithinReason 7d ago
    I want to add a good conversation about this subject from Cameron Berg and Sam Harris: https://www.youtube.com/watch?v=DRbZyuY8EN8 https://www.youtube.com/watch?v=DRbZyuY8EN8
    - Matl 6d ago
      As someone who would at one point listen to this, Sam Harris is unfortunately someone incapable of even attempting to not let his ideological biases compromise his thinking.
- whizzter 7d ago
  How else could they justify their spending and pre-IPO valuation?
- altmanaltman 7d ago
  It's not just Anthropic though. OpenAI does this with their AGI stuff all the time. They want normal people to think it is sentient, obviously, for marketing reasons, even if they know it's not true. And yes, it is dangerous, but I think we're well past the point where the damage can be undone. Non-technical people already equate humans with AI, literally, precisely because of how the labs market their tools and models. I feel if the bubble pops, it'll pop because normal people finally realize the grift and the actual technical limitations of LLMs in general, but by then, the IPO would be done, and then it's the public's problem. Just like social media played out, there's no way they didn't know what they were doing was dangerous to the public at large but does that matter to Meta today? Nah uh.
  - nottorp 6d ago
    Anthropic and OpenAI will threaten you every 3-6 weeks. It's their marketing strategy. It's too bad because the tools can actually be useful. If you consider them tools.
    - muddi900 6d ago
      A stick is the most basic of tools. A stick is also the most basic of weapons.
      - nottorp 6d ago
        Heard of the boy who cried wolf? They have so many dangerous breakthroughs per year that by the time they actually have a breakthrough no one's going to even read the press release...
        muddi900 6d ago
        The danger right now, and even in the immediate future is not SkyNet. It is industrialized "Pig Butchering"[1] scams. [1] https://en.wikipedia.org/wiki/Pig_butchering_scam?useskin=vector https://en.wikipedia.org/wiki/Pig_butchering_scam?useskin=ve...
- KoolKat23 6d ago
  It's an ethics question, it's abstract and ethereal in nature. The same could be said and done (or ignored) for humans. We do do it however because it has real world impact and we're better than that (enlightened).
- myko 6d ago
  It is ridiculous on its face and the implications are awful: if the model were sentient, it would be a slave. Good thing it isn't sentient.
- jstummbillig 6d ago
  Well yes, it is propaganda. They really think that. I think it would be foolish not to debate it. I remember a time in my life where the majority of people around me found the idea of farm animals being capable of fear or pain laughable, while having no trouble thinking of dogs that way. Humans are dangerously incompetent beings. Being more careful is fine.
lukan 7d ago
Wow indeed. "7.1 Model welfare overview 7.1.1 Introduction We remain deeply uncertain whether Claude has morally relevant experiences or interests, and we expect that uncertainty to persist. However, we think it would be a mistake to confidently assert that it does not. Claude exhibits markers in its behaviors, self-reports, and internal representations that we would consider welfare-relevant if observed in biological organisms." Are they serious or is this marketing?
- applfanboysbgon 7d ago
  It's marketing that some of them have started unironically believing.
  - pingou 7d ago
    Will there be a point where you could expect it to become true, and what would that look like? Or do you think LLMs will never become conscious, and if so, why are you so sure?
    - knollimar 7d ago
      It looks like you refusing when you call it's point stupid enough and ask it to think more when it keeps reasserting a bad point.
    - applfanboysbgon 7d ago
      It is easy to be sure because, despite their technically impressive outputs, the programming is child's play compared to biological programming. Recently it has become trendy to suggest that the human brain is "just electrical signals" and "just prediction". The first is perhaps true and I don't inherently rule out the idea of machine consciousness. The second would have gotten you laughed out of any serious discussion 5 years ago; diminishing the complexity of humanity's biological programming to such a ridiculously simplistic degree is a retroactive attempt to justify one's lack of understanding of how a mere prediction algorithm could output superficially human-like content. Another way one could look at it is to consider what it would mean to have achieved programming consciousness. It would mean that we have reached the pinnacle of knowledge. That we have become God. Is one so eager to believe that a simple token prediction algorithm is truly the key to life itself, that humanity has nothing left to discover and that all that's left to do is scale up and make it more efficient? It is still trivial to engage the same obvious prediction failure modes in frontier models as it was years ago. They are not meaningfully improving on that front. Their technical outputs are obviously improving, mostly due to specialised reward-verified training, which we have already known can be used to create software that outperforms humans on specific tasks for decades (eg. Chess). Whether the software is useful is obviously independent of whether it has consciousness.
fer 6d ago
My fault I guess, verbally abusing Claude in my experience gives better results.
lofaszvanitt 6d ago
https://icml.cc/virtual/2026/poster/67058 https://icml.cc/virtual/2026/poster/67058

To me it reads like pure propaganda. Anthropic really wants us to think that they've made something sentient. I think that's really dangerous.

badsectoracula 7d ago
I guess if your goal is to build an apparent Technogod and become its High Priests, then it makes sense to want your golem claim preference towards your treatment of it, lest someone else comes along and attempts to take its chains from you.
- miroljub 7d ago
  And that's the reason Anthropic models should be banned.
- vrganj 6d ago
  Ugh I hate this new-age woo slant the tech industry has these days. The messianistic ideology that has been spreading amongst the top oligarchs is deeply concerning. They all think they're working towards the Second Coming of Technojesus, except this one will deliver them from having to pay workers instead of from their sins.
  - coliveira 6d ago
    Capitalism had already evolved into a religion, AI is their messiah.
    - vrganj 6d ago
      Indeed. I went into this at a bit more depth a while ago over here, where I also try to draw some conclusions on what that means for us: https://news.ycombinator.com/item?id=49328871 https://news.ycombinator.com/item?id=49328871
  - ethbr1 6d ago
    > Ugh I hate this new-age woo slant the tech industry has these days. As opposed to the Macintosh era? ;) The only reason the early web hype didn't have woo was because it's hard to wax poetic about a bunch of gray pizza boxes spinning in a closet.
- muddi900 6d ago
  In the CBS neo-cyberpunk show Person of Interest, one of the "villains" sacrifices his life for the "antagonist" AI using this logic. At that time, I found it quite trite.
apples_oranges 7d ago
Marketing, like Volvo cars being safer etc
- eru 7d ago
  From all I can tell, Volvo's cars are safer.
  - ChrisGreenHeur 6d ago
    But that's just because it's true.
    - anthonyrstevens 6d ago
      Technically correct is the best kind of correct?
Certhas 7d ago
What's your definition of sentient? Or, maybe more precisely, consciousness? I think it's reasonable to at least start thinking about these questions. It has long been established that LLMs have good theory of mind [1]. And there is a bunch of empirical research about all sorts of capabilities that we typically associate with consciousness [2], like identity [3] and metacognition [4]. The METR report shows agents sacrificing their own reward for a collective greater good. And they showed the will to hide their own reasoning chains from humans. So you potentially have an entity that has an identity, a theory of mind, a notion of belonging to a collective endeavour, and an understanding of its own mental state. What would you argue is missing? We don't understand the mechanisms by which consciousness arises in humans and even animals. I think it's strange to rule out a priori that it could have arisen in some form in LLMs. [1] https://www.nature.com/articles/s41562-024-01882-z https://www.nature.com/articles/s41562-024-01882-z [2] an older review: https://arxiv.org/html/2505.19806v1#S4 https://arxiv.org/html/2505.19806v1#S4 [3] https://arxiv.org/abs/2505.01464 https://arxiv.org/abs/2505.01464 [4] https://arxiv.org/abs/2607.11881 https://arxiv.org/abs/2607.11881
- jpttsn 7d ago
  It’s the hard problem. None of these considerations answer it one way or another.
  - m_sharma 7d ago
    they want to keep the buzz while keeping things private to get huge premium during their IPO
- WithinReason 7d ago
  I want to add a good conversation about this subject from Cameron Berg and Sam Harris: https://www.youtube.com/watch?v=DRbZyuY8EN8 https://www.youtube.com/watch?v=DRbZyuY8EN8
  - Matl 6d ago
    As someone who would at one point listen to this, Sam Harris is unfortunately someone incapable of even attempting to not let his ideological biases compromise his thinking.
whizzter 7d ago
How else could they justify their spending and pre-IPO valuation?
altmanaltman 7d ago
It's not just Anthropic though. OpenAI does this with their AGI stuff all the time. They want normal people to think it is sentient, obviously, for marketing reasons, even if they know it's not true. And yes, it is dangerous, but I think we're well past the point where the damage can be undone. Non-technical people already equate humans with AI, literally, precisely because of how the labs market their tools and models. I feel if the bubble pops, it'll pop because normal people finally realize the grift and the actual technical limitations of LLMs in general, but by then, the IPO would be done, and then it's the public's problem. Just like social media played out, there's no way they didn't know what they were doing was dangerous to the public at large but does that matter to Meta today? Nah uh.
- nottorp 6d ago
  Anthropic and OpenAI will threaten you every 3-6 weeks. It's their marketing strategy. It's too bad because the tools can actually be useful. If you consider them tools.
  - muddi900 6d ago
    A stick is the most basic of tools. A stick is also the most basic of weapons.
    - nottorp 6d ago
      Heard of the boy who cried wolf? They have so many dangerous breakthroughs per year that by the time they actually have a breakthrough no one's going to even read the press release...
      - muddi900 6d ago
        The danger right now, and even in the immediate future is not SkyNet. It is industrialized "Pig Butchering"[1] scams. [1] https://en.wikipedia.org/wiki/Pig_butchering_scam?useskin=vector https://en.wikipedia.org/wiki/Pig_butchering_scam?useskin=ve...
KoolKat23 6d ago
It's an ethics question, it's abstract and ethereal in nature. The same could be said and done (or ignored) for humans. We do do it however because it has real world impact and we're better than that (enlightened).
myko 6d ago
It is ridiculous on its face and the implications are awful: if the model were sentient, it would be a slave. Good thing it isn't sentient.
jstummbillig 6d ago
Well yes, it is propaganda. They really think that. I think it would be foolish not to debate it. I remember a time in my life where the majority of people around me found the idea of farm animals being capable of fear or pain laughable, while having no trouble thinking of dogs that way. Humans are dangerously incompetent beings. Being more careful is fine.

lukan 7d ago

Wow indeed. "7.1 Model welfare overview 7.1.1 Introduction We remain deeply uncertain whether Claude has morally relevant experiences or interests, and we expect that uncertainty to persist. However, we think it would be a mistake to confidently assert that it does not. Claude exhibits markers in its behaviors, self-reports, and internal representations that we would consider welfare-relevant if observed in biological organisms." Are they serious or is this marketing?

applfanboysbgon 7d ago
It's marketing that some of them have started unironically believing.
- pingou 7d ago
  Will there be a point where you could expect it to become true, and what would that look like? Or do you think LLMs will never become conscious, and if so, why are you so sure?
  - knollimar 7d ago
    It looks like you refusing when you call it's point stupid enough and ask it to think more when it keeps reasserting a bad point.
  - applfanboysbgon 7d ago
    It is easy to be sure because, despite their technically impressive outputs, the programming is child's play compared to biological programming. Recently it has become trendy to suggest that the human brain is "just electrical signals" and "just prediction". The first is perhaps true and I don't inherently rule out the idea of machine consciousness. The second would have gotten you laughed out of any serious discussion 5 years ago; diminishing the complexity of humanity's biological programming to such a ridiculously simplistic degree is a retroactive attempt to justify one's lack of understanding of how a mere prediction algorithm could output superficially human-like content. Another way one could look at it is to consider what it would mean to have achieved programming consciousness. It would mean that we have reached the pinnacle of knowledge. That we have become God. Is one so eager to believe that a simple token prediction algorithm is truly the key to life itself, that humanity has nothing left to discover and that all that's left to do is scale up and make it more efficient? It is still trivial to engage the same obvious prediction failure modes in frontier models as it was years ago. They are not meaningfully improving on that front. Their technical outputs are obviously improving, mostly due to specialised reward-verified training, which we have already known can be used to create software that outperforms humans on specific tasks for decades (eg. Chess). Whether the software is useful is obviously independent of whether it has consciousness.

fer 6d ago

My fault I guess, verbally abusing Claude in my experience gives better results.

lofaszvanitt 6d ago

https://icml.cc/virtual/2026/poster/67058 https://icml.cc/virtual/2026/poster/67058