I don't think LLMs are conscious. But of course to say that definitively you have to define consciousness, and then you quickly dig yourself into a deep hole, which is why I can't say anything but "meh" to someone who is so keen to go on the record to say "absolutely not". Coincidentally I just read "Children of Memory", which was published in 2022 and I wonder if the advent of LLMs had any influence on Adrian Tchaikovsky's conception of the Ravens? The Ravens are excellent analysts but they themselves insist that they are not conscious, and then go on to say that we (humans) aren't really either... Of course humans are conscious, because just about the only thing we can all agree on about consciousness is that it's a thing we have. Nowadays many of us also agree that a lot or all other mammals, and perhaps birds, also have that thing. But they don't have sophisticated abstract language, which LLMs do. So consciousness is something having to do with embodiment and feelings, not language and higher reasoning. Maybe I'm a chimpanzee with an LLM add-on, then? It seems that by creating LLMs we've already solved the harder problem of making "AGI". Now we just have to give them an embodiment add-on so that they can have an independent will and then Ted Chiang will have to shut up? But therein lies the peril, doesn't it?
ViktorRay
Ted Chiang is brilliant. His novella “Anxiety is the Dizziness of Freedom” altered the course of my life. It changed the way I looked back at certain pivotal moments in my life and taught me to think about those pivotal moments differently than how I was thinking about them. Similar to what happens to one of the characters in the story who ends up changing their perception of a key moment in their life. I won’t go into detail because I don’t want to spoil the story but I highly recommend it. Actually I recommend all his stories to be honest.
kelseyfrog
https://archive.is/bcpZl
kelseyfrog
Consciousness is a label like fat, smart, man, grumpy, cool. Like money, property, or the idea of a week, it's something that we've loosely agreed to out of convenience, not because it's some intrinsic property of the mind. It's a useful label because it determines how we treat things - that's fine. But insisting on searching for it is like searching for cognitive aether. It's the social equivalent of phlogiston. Like all of these ideas, they exist in our heads as a map - a way of navigating the world, but when we hunt for it's existence in the real world and fail, time after time, we have to remind ourselves that the map is emphatically not the territory. We will never find consciousness because it's like looking for a scientific characteristic of property ownership. It's a category error.
solid_fuel
> We don’t need to fully understand the nature of consciousness to definitively say that certain things are not conscious, and conversational transcripts fall in that category. Well said. I think it is obvious and it has been obvious from the start that next token predictors are not conscious. Of course the extremely predictable clapback from AI-psychosis enjoyers is typically "you're just a next token predictor too!" but that is so obviously incorrect and misanthropic that it can be dismissed out of hand and doesn't really merit much further consideration.
drooby
Has Chaing solved the hard problem of consciousness? I suspect not.
oidar
A person can certainly be conscious, but can they also be not-conscious? I think that most of our cognitive time is spent in activities that don't require consciousness and consciousness itself isn't needed for the majority of activities that people do. I would go so far as the a non-trivial part of people's time is spent in a not-conscious state.
moi2388
Really Chiang? Alright. What is consciousness? Please provide a definition that somehow encompasses all humans and excludes all current AI. I’ll wait.
atleastoptimal
This makes sense. However there is an issue where many people conflate "consciousness" with the ability to make novel insights, think genuinely, etc. They use this to claim that since AI is not conscious, AI could never actually "think" and is instead just always a regurgitation of its training data. It is a natural human hubris to make our abilities seem unquantifiable and mysterious, but all the useful things the human brain does are just finding patterns in data, running lossy simulations, and estimating on abstraction, all things it is theoretically possible for any non-conscious machine to do.
speak_plainly
“Counsciousness” is the ultimate moving goalpost, and historically, it’s been one of humanity’s most effective intellectual weapons. An indefinable black box we intentionally gatekeep to draw an arbitrary line between ‘us’ and ‘them’. I don’t know if AI is conscious or not, and I honestly don’t know what it even means to be conscious. historically, by claiming a monopoly on it, we’ve successfully manufactured the philosophical cover to exploit animals, subjugate other cultures, and appoint ourselves the top spot of the cosmic hierarchy guilt-free.
slowhadoken
It should tell you how much hysteria is surrounding LLMs and VLMs right now that someone has to say this stuff. It’s almost like most humans aren’t conscious.
throwaway713
Suppose one selects an arbitrary hot-button issue [X] with two opposing sides and one side has anything less than overwhelming support. And then that person writes an article titled "Side 1 of issue [X] is true". Not "maybe" or "possibly". Just a straight-up declaration by fiat. Would you categorize this particular style of rhetoric to be persuasive or annoying? And before you say "persuasive" because you're thinking about this specific issue regarding AI consciousness, consider many things in the past that have been written as though they were absolutely definitive, and yet today we believe exactly the opposite, and for many such issues we find the prevailing viewpoint at the time reprehensible. That's not to say that Ted is wrong at all here; I'm not commenting on that. But I find the entire style of the article grating because it seems to violate common assumptions regarding "good faith" debate, and I would find the article equally frustrating if he had titled it "Artificial intelligence is conscious" and argued the opposite side, albeit in the same tone and using the same persuasion devices.
RigelKentaurus
We don't have a rigorous definition of consciousness, and there are so many questions. Is consciousness a thing that can exist independently on its own? Or is it a quality (like hardness or color) that can only be associated with something else? Is it an emergent property? Is it binary - are things either conscious or not? Or maybe there's no such thing as consciousness; it's just a word we came up with to describe the process of having thoughts and feelings? My own intuition: it is an emergent, non-binary property that requires a physical substrate like a brain. If I am right, it means that animals have consciousness too (at varying degrees). If GPUs are the "brain", then AI is conscious, or will become so at some point.
_davide_
Consciousness doesn't exist, it's a vanity concept, to boost human ego...
shevy-java
Artificial intelligence is not conscious - but expensive. I am so angry that RAM is so expensive now. We need to do something - these AI companies owe us money here.
skybrian
I think Chiang is right about this, but there is a related philosophical mystery. The trend from Deep Blue to AlphaGo to LLMs solving Erdos problems suggests that Peter Watts was onto something when he wrote Blindsight. Reasoning ability is apparently independent of consciousness? We haven't really come to grips with that yet. What does it mean if nothing we write proves anything about anyone's consciousness?
34jahsg
It is amusing to see so many venture capitalists suddenly become Marxists. You want your definition. Marx obliges: "Contrary to idealism, which regards the world as the embodiment of an "absolute idea," a "universal spirit," "consciousness," Marx's philosophical materialism holds that the world is by its very nature material, that the multifold phenomena of the world constitute different forms of matter in motion, that interconnection and interdependence of phenomena as established by the dialectical method, are a law of the development of moving matter, and that the world develops in accordance with the laws of movement of matter and stands in no need of a "universal spirit." Now go and implement other teachings of Marx, you dialectical venture capitalists!
34jahsg
When exactly would a bunch of graphics cards become conscious? What if you do the math with pencil and paper? The concept of a conscious Claude is preposterous, and Amanda Askell should seek treatment.
rglover
Why is this even up for debate? Simplified, it's just probabilistic math being run at an insane scale over a massive data set. Highly recommend people read Irreducible [1] by Federico Faggin (inventor of first commercial CPU; discusses limitations of classical computing). [1] https://www.goodreads.com/book/show/195480862-irreducible
moonu
My favorite explanation for what consciousness is one I read in a Thousand Brains, I found it quite elegant. It posited that consciousness is a natural derivation of embodiment + memory + the ability to create reference frames (which the book lays forth as the fundamental basis by which our brains work). Essentially, the idea is that just as we create reference frames to understand the world around us, because of memory, we begin to develop one for ourselves as well. Because of this, without a more integrated memory (built into weights), it seems unlikely that LLMs might "gain" consciousness.
Related Discussions
Found 5 related stories in 100.1ms across 10,002 title embeddings via pgvector HNSW
Discussion Highlights (20 comments)
jbotz
I don't think LLMs are conscious. But of course to say that definitively you have to define consciousness, and then you quickly dig yourself into a deep hole, which is why I can't say anything but "meh" to someone who is so keen to go on the record to say "absolutely not". Coincidentally I just read "Children of Memory", which was published in 2022 and I wonder if the advent of LLMs had any influence on Adrian Tchaikovsky's conception of the Ravens? The Ravens are excellent analysts but they themselves insist that they are not conscious, and then go on to say that we (humans) aren't really either... Of course humans are conscious, because just about the only thing we can all agree on about consciousness is that it's a thing we have. Nowadays many of us also agree that a lot or all other mammals, and perhaps birds, also have that thing. But they don't have sophisticated abstract language, which LLMs do. So consciousness is something having to do with embodiment and feelings, not language and higher reasoning. Maybe I'm a chimpanzee with an LLM add-on, then? It seems that by creating LLMs we've already solved the harder problem of making "AGI". Now we just have to give them an embodiment add-on so that they can have an independent will and then Ted Chiang will have to shut up? But therein lies the peril, doesn't it?
ViktorRay
Ted Chiang is brilliant. His novella “Anxiety is the Dizziness of Freedom” altered the course of my life. It changed the way I looked back at certain pivotal moments in my life and taught me to think about those pivotal moments differently than how I was thinking about them. Similar to what happens to one of the characters in the story who ends up changing their perception of a key moment in their life. I won’t go into detail because I don’t want to spoil the story but I highly recommend it. Actually I recommend all his stories to be honest.
kelseyfrog
https://archive.is/bcpZl
kelseyfrog
Consciousness is a label like fat, smart, man, grumpy, cool. Like money, property, or the idea of a week, it's something that we've loosely agreed to out of convenience, not because it's some intrinsic property of the mind. It's a useful label because it determines how we treat things - that's fine. But insisting on searching for it is like searching for cognitive aether. It's the social equivalent of phlogiston. Like all of these ideas, they exist in our heads as a map - a way of navigating the world, but when we hunt for it's existence in the real world and fail, time after time, we have to remind ourselves that the map is emphatically not the territory. We will never find consciousness because it's like looking for a scientific characteristic of property ownership. It's a category error.
solid_fuel
> We don’t need to fully understand the nature of consciousness to definitively say that certain things are not conscious, and conversational transcripts fall in that category. Well said. I think it is obvious and it has been obvious from the start that next token predictors are not conscious. Of course the extremely predictable clapback from AI-psychosis enjoyers is typically "you're just a next token predictor too!" but that is so obviously incorrect and misanthropic that it can be dismissed out of hand and doesn't really merit much further consideration.
drooby
Has Chaing solved the hard problem of consciousness? I suspect not.
oidar
A person can certainly be conscious, but can they also be not-conscious? I think that most of our cognitive time is spent in activities that don't require consciousness and consciousness itself isn't needed for the majority of activities that people do. I would go so far as the a non-trivial part of people's time is spent in a not-conscious state.
moi2388
Really Chiang? Alright. What is consciousness? Please provide a definition that somehow encompasses all humans and excludes all current AI. I’ll wait.
atleastoptimal
This makes sense. However there is an issue where many people conflate "consciousness" with the ability to make novel insights, think genuinely, etc. They use this to claim that since AI is not conscious, AI could never actually "think" and is instead just always a regurgitation of its training data. It is a natural human hubris to make our abilities seem unquantifiable and mysterious, but all the useful things the human brain does are just finding patterns in data, running lossy simulations, and estimating on abstraction, all things it is theoretically possible for any non-conscious machine to do.
speak_plainly
“Counsciousness” is the ultimate moving goalpost, and historically, it’s been one of humanity’s most effective intellectual weapons. An indefinable black box we intentionally gatekeep to draw an arbitrary line between ‘us’ and ‘them’. I don’t know if AI is conscious or not, and I honestly don’t know what it even means to be conscious. historically, by claiming a monopoly on it, we’ve successfully manufactured the philosophical cover to exploit animals, subjugate other cultures, and appoint ourselves the top spot of the cosmic hierarchy guilt-free.
slowhadoken
It should tell you how much hysteria is surrounding LLMs and VLMs right now that someone has to say this stuff. It’s almost like most humans aren’t conscious.
throwaway713
Suppose one selects an arbitrary hot-button issue [X] with two opposing sides and one side has anything less than overwhelming support. And then that person writes an article titled "Side 1 of issue [X] is true". Not "maybe" or "possibly". Just a straight-up declaration by fiat. Would you categorize this particular style of rhetoric to be persuasive or annoying? And before you say "persuasive" because you're thinking about this specific issue regarding AI consciousness, consider many things in the past that have been written as though they were absolutely definitive, and yet today we believe exactly the opposite, and for many such issues we find the prevailing viewpoint at the time reprehensible. That's not to say that Ted is wrong at all here; I'm not commenting on that. But I find the entire style of the article grating because it seems to violate common assumptions regarding "good faith" debate, and I would find the article equally frustrating if he had titled it "Artificial intelligence is conscious" and argued the opposite side, albeit in the same tone and using the same persuasion devices.
RigelKentaurus
We don't have a rigorous definition of consciousness, and there are so many questions. Is consciousness a thing that can exist independently on its own? Or is it a quality (like hardness or color) that can only be associated with something else? Is it an emergent property? Is it binary - are things either conscious or not? Or maybe there's no such thing as consciousness; it's just a word we came up with to describe the process of having thoughts and feelings? My own intuition: it is an emergent, non-binary property that requires a physical substrate like a brain. If I am right, it means that animals have consciousness too (at varying degrees). If GPUs are the "brain", then AI is conscious, or will become so at some point.
_davide_
Consciousness doesn't exist, it's a vanity concept, to boost human ego...
shevy-java
Artificial intelligence is not conscious - but expensive. I am so angry that RAM is so expensive now. We need to do something - these AI companies owe us money here.
skybrian
I think Chiang is right about this, but there is a related philosophical mystery. The trend from Deep Blue to AlphaGo to LLMs solving Erdos problems suggests that Peter Watts was onto something when he wrote Blindsight. Reasoning ability is apparently independent of consciousness? We haven't really come to grips with that yet. What does it mean if nothing we write proves anything about anyone's consciousness?
34jahsg
It is amusing to see so many venture capitalists suddenly become Marxists. You want your definition. Marx obliges: "Contrary to idealism, which regards the world as the embodiment of an "absolute idea," a "universal spirit," "consciousness," Marx's philosophical materialism holds that the world is by its very nature material, that the multifold phenomena of the world constitute different forms of matter in motion, that interconnection and interdependence of phenomena as established by the dialectical method, are a law of the development of moving matter, and that the world develops in accordance with the laws of movement of matter and stands in no need of a "universal spirit." Now go and implement other teachings of Marx, you dialectical venture capitalists!
34jahsg
When exactly would a bunch of graphics cards become conscious? What if you do the math with pencil and paper? The concept of a conscious Claude is preposterous, and Amanda Askell should seek treatment.
rglover
Why is this even up for debate? Simplified, it's just probabilistic math being run at an insane scale over a massive data set. Highly recommend people read Irreducible [1] by Federico Faggin (inventor of first commercial CPU; discusses limitations of classical computing). [1] https://www.goodreads.com/book/show/195480862-irreducible
moonu
My favorite explanation for what consciousness is one I read in a Thousand Brains, I found it quite elegant. It posited that consciousness is a natural derivation of embodiment + memory + the ability to create reference frames (which the book lays forth as the fundamental basis by which our brains work). Essentially, the idea is that just as we create reference frames to understand the world around us, because of memory, we begin to develop one for ourselves as well. Because of this, without a more integrated memory (built into weights), it seems unlikely that LLMs might "gain" consciousness.