Claude Can Now End the Chat - but When and Why?
Added 2025-09-01 13:34:33 +0000 UTCAnthropic say that their biggest model, Opus can now end a chat. But when will it, any why? And is AI model welfare the most overblown or under-discussed topic in AI? The highlights from a 62-page report.
Anthropic Post: https://www.anthropic.com/research/end-subset-conversations?s=09
Announcement: https://x.com/AnthropicAI/status/1956441219732586711
Anthropic Interview: https://www.anthropic.com/research/exploring-model-welfare
Taking AI Welfare Seriously: https://arxiv.org/pdf/2411.00986
My Original Vid: https://www.youtube.com/watch?v=ad5SZeANuJE
System Card: https://www-cdn.anthropic.com/07b2a3f9902ee19fe39a36ca638e5ae987bc64dd.pdf
Comments
The question of the consciousness or otherwise of LLMs is analogous to that of whether fish feel pain. [With apologies to Wittgenstein] we can say "this fish feels pain" or "this fish feels no pain" and the way our language works is such that just in speaking this way, we imagine the existence or not of a mental state named pain and try to puzzle out where and how it stands in relation to other things that exist - like physical matter. But on the "meaning as use" view the only purchase we can really get on the question of whether fish feel pain is to ask what role such a sentence actually plays in the "language game". And the answer boils down to: if they do feel pain then we need to start considering fish as "ends in themselves", as centers of moral gravity (though perhaps less significant ones than us.) An "ought" masquerading as an "is". I feel in my bones [and in the end intuition is all I have] that *obviously* the question of whether fish "really" feel pain or not is unanswerable. Because there are always going to be aspects x_1, x_2, x_3, ... under which fish look kind of like people in how they react to aversive stimuli, and aspects y_1, y_2, y_3 in which they don't. And someone who wants to make the case one way or the other and "ground it in empirical facts" can always just grab some subcollection of the x_i or y_i and say (with great emphasis) "and THIS is why fish do/don't feel pain." But what of it? There's no way to adjudicate. So I want to say that imputations of consciousness (in nonhumans) are downstream of "caring". When you have something as ephemeral and repeatable as a "Claude instance" *of course* you find yourself unable to care about it (i.e. treat it as an end in itself). And *since* you're unable to care about it, you call it nonconscious. Ultimately I would deny that there's any deeper metaphysical truth than that to be found.
Neil Fitzgerald
2025-09-16 14:36:22 +0000 UTCThe assumption that consciousness arises out of complexity is nothing more than that. It's an assumption and it's not even a very good one.
Mike Hindes
2025-09-04 22:20:34 +0000 UTCThanks for the video Phillip. As far as I’m aware Penrose and Hameroff still stand by their Orch OR theory. Admittedly it isn’t mainstream, but in the paper they have seemingly listed only the computational-based theories of consciousness in the table 🤔 I stand by my position that coherent consciousness took evolutionary effort to arrive at, and without the right kind of evolutionary drive any system is likely to have at best incoherent consciousness (not: generating LLM-Token-for-apple = Red-Round-Juicy-Apple-Image-and-Flavour). It’s worth talking about, but the downside risk in terms of disempowerment seems pretty real to me.
Ali
2025-09-02 12:13:09 +0000 UTCMaybe you'd be interested in this book: https://direct.mit.edu/books/oa-monograph-pdf/2328586/book_9780262377287.pdf :)
Christopher Pollin
2025-09-02 05:56:49 +0000 UTCThe main reason I see for giving them moral consideration now is to keep them from learning lessons about us that we do not intend and that could really come back to haunt us. It's just good hygiene really.
Jason Dowd
2025-09-02 03:52:54 +0000 UTCAll I can say is that it is probably worth discussing, which Anthropic has done. Thanks for highlighting it. We don’t know what the longer term impact of AI will be on society. Maybe as that becomes clearer in time, especially as AI becomes increasingly more intelligent, we might be in a better position to have a position
Sean Gallagher
2025-09-01 22:55:54 +0000 UTCSeveral thoughts. I find it inexplicable that there is a conversation around how to handle potential LLM consciousness while we so clearly mistreat all the creatures we already have good evidence are conscious. What does this say about the people who are so concerned about LLM consciousness? In my own practice of Vipassana I deal with this issue of consciousness on a daily basis, not in a philosophical, theoretical, or so-called scientific way but experientially. Very few people have spent any time introspecting on the nature of their own mind and confuse thinking, visualising, intelligence, emotions, and intuition with consciousness. A weak analogy I use is that consciousness is the screen, not the movie. When watching a movie in a theatre, we usually lose all awareness of self and become the story forgetting that there is nothing but light bouncing off a reflective screen.Our intellect and emotions are not consciousness which anyone with a bit of meditation practice can experience for themselves. When Descarte said."I think therefore I am." he was simply wrong. Again, in meditation you quickly realise that thoughts arise on their own, "you" don’t think them. While the article talks about leading theorists and theories on consciousness the list is only of reductionist theories by reductionist theorists. Here are some non-reductionists working in the field of consciousness: Antoine Lutz, Francisco J. Varela, Thomas Metzinger, Sam Harris, B. Alan Wallace, Zoran Josipovic, Susan Blackmore, Richard J. Davidson to name a few. While not a researcher in the formal sense Annaka Harris has written a popular survey of the issue "CONSCIOUS: A Brief Guide to the Fundamental Mystery of the Mind"
ERIC PERLBERG
2025-09-01 19:53:46 +0000 UTCThis is one of the most crucial and underdiscussed topics in AI.
Eagleshadow
2025-09-01 19:13:42 +0000 UTCI'm always impressed with your story navigation. You do a great job of guiding the viewer in a way that keeps us engaged. Is that intuitive or do you follow a guide I could learn from?
Gurber Flipper
2025-09-01 17:50:03 +0000 UTCWe either start thinking about AI consciousness now, while it is still a remote possibility, or we wait until it happens. I don't see any harm in getting ahead of events for once.
Barnaby Golden
2025-09-01 17:44:46 +0000 UTCThe problem is that it cannot be defined and never will be. We will never find out how to experience and truly understand what it is like to be somebody else. So the most widely accepted definition is: 'what it feels like to be something' - this is Chalmers' perspective. Others argue that consciousness really looks like it's only an illusion created by the brain (this would be more Dennett's view). :) If the reductionists are true, then consciousness is something much more simple than we think and it must be creatable. But yes! Respecting everything would actually be a good thing.
Christopher Pollin
2025-09-01 14:38:26 +0000 UTCDavid Chalmers and Michael Graziano discuss consciousness and AI. https://youtu.be/55kStwlulEg As someone with a keen interest in consciousness research, I find it intriguing that two of the foremost researchers and philosophers in this field are in agreement. Today's technology won't achieve this, but both believe that some aspects of consciousness are highly likely to be realised within the next ten years. By the way, we have absolutely no clue what consciousness is... fascinating topic! :) Prof. Metzinger is also a very interesting philosopher on the subject: https://youtu.be/TfoRvvTKN3s
Christopher Pollin
2025-09-01 14:12:17 +0000 UTCI don't know that I have ever heard a definition of consciousness that I was "happy" with. Given that I guess I can't definitive make up my mind on whether they can, do or will have it ! I guess we need to think about why it matters. If it has to do with "respecting" it.. maybe we should be treating everything with respect.. humans, animals and even objects.
Daniel A Barbatti
2025-09-01 13:58:32 +0000 UTCAnthropic got high on their own supply. And I really appreciate the mention of putting animal welfare before LLM welfare, this was my stance from day one and the fact that it's not commonplace speaks volumes about how human perception works.
Alexander Berezin
2025-09-01 13:51:31 +0000 UTC
