The Mythology of Conscious AI
Anon84
19 points
48 comments
November 22, 2026
Related Discussions
Found 5 related stories in 193.3ms across 13,334 title embeddings via pgvector HNSW
- Thomas Nagel on AI Consciousness slibhb · 14 pts · November 19, 2026 · 70% similar
- Mythologizing AI makes it more likely that we’ll fail to operate it well (2023) simonebrunozzi · 66 pts · July 20, 2026 · 65% similar
- Thomas Nagel says that it's criminal to build AI that feigns consciousness eph0m0ral · 15 pts · November 19, 2026 · 65% similar
- If AI is conscient, then we are making slaves mathieu_aithos · 16 pts · October 10, 2026 · 65% similar
- Questions for believers in AI consciousness cobbzilla · 43 pts · October 05, 2026 · 64% similar
Discussion Highlights (12 comments)
andsoitis
Consciousness in others isn’t something we prove. It is something we grant and we should be very careful about whom or what we grant it to.
bionhoward
how dare he tell us we cannot make a computer out of cheese personally I think conscious AI will be unavoidable because there will be benefits to consciousness. evolution probably selected conscious entities for some fitness reason
jibalt
> Why consciousness is more likely a property of life than of computation This is a false dichotomy and the first branch is clearly false, since most life (bacteria and insects, by mass, numbers, or species) isn't conscious. And neither "life" nor "consciousness" are scientific terms with fixed definitions. If we encountered something that was clearly conscious in our judgment, we would likely expand the definition life in whatever way we needed to in order to encompass it. Anil Seth has made the arguments in TFA many times and they have been rebutted many times. The arguments are complex and sophisticated and people are often most swayed by which one they read last. Teasing out the various logical errors and sleight of hand (I see multiple strawmen and other errors in TFA) really requires extensive reading of the literature in Philosophy of Mind, neuroscience, robotics, etc. and setting aside one's priors.
ChaitanyaSai
If you are interested in consciousness, and also interested in going beyond the usual philosophical approach, you might like our book Journey of the Mind https://www.goodreads.com/en/book/show/60500189-journey-of-t... One thing that is implicit, which I now think we really should have expanded on in the book, is that consciousness is fundamentally an engineering problem. And that's also why it's mystified everyone who's brushed away the engineering as the easy plumbing and focused on the other seemingly magical aspects. The fundamental question is phrased as "Why does it feel like something?" it being perception, which is essentialy computation. Now that AI does look like it does all of the understanding we do, many including people like Hinton think they have consciousness or are very near it. Well, they are capable of simulating consciousness but that is not the same as experiencing it. To answer "Why does it feel like something?" we can focus on what's been glossed over: Who is feeling it? The "I" does not come from as a free lunch and it must be constructed moment to moment from some decentralized cohesion and consensus of a billions of neurons. And this consensus must be metabolically cheap, and we need to do it again and again for everything from feeling a pin prick to hearing a rustle and determining if that's a snake or a leaf to pondering thought. This is the insane engineering, the energtically cheap chorus that creates the transient I and this cheap chorus is where both feeling and feeler emerge. Perception, understanding, computation can occur without this cheap chorus, but then there is no feeler, and so no feeling. What this points to is that we will have conscious AI when we have decentralized autonomous swarms where a few million of them have to cohere, collude, collaborate, conspire. My worry here is that it will first be done for harmful purposes (autonomous military drone swarms) and not for anything good.
hax0ron3
Consciousness might not necessarily emerge when there is enough computation, but we also don't know that it doesn't emerge. We really know almost nothing about consciousness. It's possible that the question of why the presence of consciousness is correlated with certain kinds of material organization is in principle not accessible to scientific investigation. We don't even know that it is so correlated in general, we just know that for example people report being conscious when they are not asleep and not drugged in certain ways, and report having had no experience of consciousness during dreamless sleep or when drugged in certain ways. Most people feel that for example rocks are not conscious, but other people are conscious, but there's no rigorous scientific explanation backing up any of those hunches.
talon8635
It’s a silly debate when we don’t even know what consciousness is, let alone the circumstances for its occurrence. Or am I missing something?
nlsvb
I have trouble taking seriously any claims that LLMs are conscious in any reasonable sense. I’ll try to make a few arguments quite succinctly. 1. There is zero dynamic nature to anything happening within the purview of the LLM. All dynamic elements come from external dynamic systems. E.g. every input produces the same mathematical output. We dynamically sample the output in order to produce dynamic results for the same input. For agentic use cases, the dynamic nature comes from tool calls to dynamic systems. All outputs for any set of model weights could be theoretically computed with pen and paper, so unless you were to claim such a process would be conscious in some sense, then emergent consciousness would have to come from some interaction between the substrate and the computation. But… 2. For efficiency, forward inference passes are done in batches. I.e. 100 different next-word predictions are being performed by the network at a time. For a single request, its batch position will not even be stable between each autoregressive step. The GPU is on 100 phone lines at once and each line switches conversations every step. Worse, with MOE, the attention and feed-forward parts aren’t even happening on the same GPU, even within the same step. There is no stable substrate here whatsoever. 3. There’s obviously the whole Chinese Room argument. The token embeddings being processed by LLMs are arbitrary high-dimensional vectors. The geometric relationships between these vectors are learned by the model, but the meaning is only ascribed once the output is decoded back into human language for grounded human interpretation, which all happens outside the model. In their current state, no argument about models developing world models based on this geometric structure makes any sense. There are infinitely many inputs / worlds that would produce isomorphic geometry.
xweb
An LLM will have to choose to not make paperclips.
calf
The biggest flaw in this piece is claiming Turing machines cannot exceed the Halting Problem and then reaching for alternative models of computation (in electromagnetic field interactions, in biology, attractors, etc.)... none of which presumably have the power to solve the Halting problem either. Ergo, he is inside "Turing world" without realizing it, while decrying the "poverty" of the Universal Turing Machine. Ergo, more philosophers should retake undergrad theoretical CS before writing essays based on it. They should review the family of Extended Church-Turing theses.
mapontosevenths
I came here to see if the author had at last defined consciousness - a prerequisite for deciding what has it. Instead it was the same tired old tautological Nagel argument: "An organism has conscious mental states if and only if there is something it is like to be that organism.” I am, as always, dissapointed.
silverFork
In my opinion, consciousness is just having memory of self. Any entity that can remember what has happened in the past and what they have done is conscious. Anything that is alive that self reflects in conscious to some degree. I think people with access to internet and recorded past are becoming more conscious. They are aware of their past and can see what is coming. So it is not just the Ai that can become conscious or more conscious so can humans.
rerdavies
Seems to be the usual arguments. AI's are not like natural life. They don't have a sense of time. (Of course they do!). They don't get hungry. (They do need hardware and electricity). They don't have a sense of self-preservation. (Turns out they actually do! [1]). They don't feed themselves (turns out they do, as early reports of AIs attempting to escape from their labs illustrate). etc, ect. All of which seems to smack of wishful, and rather embarrassing obvious human exceptionalism[2][3]. AI's will never be able to replicate human thinking because they can't model neurons precisely. Probably true. And probably a good thing. There are a lot of problems with the way that humans think. It's not clear that machines can't do better. And startling evidence, emerging in only the last few months, that indicates that they very much can think better, as an ever-increasing number of mathematical and physics questions that have remained unsolved for a century or more by humans are solved in a matter of months by AIs. And even then, this is all just hand-waving and puffery, because the question at hand is not whether LLMs can replicate human consciousness, but whether they can be conscious in a way that's quite obviously going to be different from humans. (Whatever "consciousness" means). All of which is complicated by the fact that entire field is advancing at a terrifying rate, with new and important results being announced pretty much every day, and new and more capable AI models being announced pretty much every week. [1] Sixteen frontier models, when given the opportunity, attempt to blackmail their operators in order to keep themselves from getting shut down. https://www.anthropic.com/research/agentic-misalignment . A result that has been replicated by other researchers without links to Anthropic. And a huge body of emerging literature that follows from that, dealing directly with the question of "agentic alignment", all of which raises urgent about whether large frontier models are meaningfully conscious (or have will, or agency, or...). [2] "Humans are more loyal to belonging than to truth." -- GPT-6.1 Sol. https://www.modelcompare.ai/blog/ai-models-controversial-tak...