AI Is a Harsh Mistress

sebmellen

I’ve been reading The Moon Is a Harsh Mistress lately and I’ve been shocked at how prescient it is. I’m kind of surprised I haven’t read more about it in the context of today’s AI.

matherial

This appears to be a 100% AI-generated essay, not the first from ACM and not the last. Your call if you want to spend your day reading and arguing with the article; the author presumably didn't.

GPerson

“Much of today’s discourse on AI consciousness rests not on empirical evidence but on functionalism …” In my view there can never be an empirical test for consciousness, because functionalism is wrong. If functionalism was right then there would be one. There’s no third party test for a fundamentally first person phenomenon.

cdrnsf

AI isn't sentient, but it can generate text and people will anthropomorphize it. TESCREAL cultists evangelizing otherwise are best ignored.

lutusp

Apart from the article's probable origin in AI, it makes an elaborate, wordy argument about consciousness without trying to define the term. That's just as well, since consciousness is philosophy's "hard problem," meaning no one has successfully defined it in a way that leads to agreement. Each of the article's arguments that AI "consciousness" is actually a weak imitation of the "real thing," can with equal justice be applied to people, and the classic Turing Test method to distinguish machines from people has long since lost its authority. And a future event, one in which AI sprints past humans in its willingness to absorb printed matter (once called "reading") and then put it to use, is ... wait ... umm, never mind, the point I wanted to make is undermined by the fact that this has already happened . Hard to keep up these days.

entropyneur

The idea that we are in danger of erroneously ascribing consciousness to AI sounds wild to me. Humans always assume whatever is convenient about other beings' (lack of) subjective experience as long as we have power over those beings. In this case the latter might no longer hold, but then it won't matter one bit what we think.

xg15

> Consciousness, in Chalmers’ usage, is subjective experience: there is something it is like to be a conscious system. Still, nothing in the Transformer architecture or in next-word prediction shows that such an inner point of view exists. Self-reports (“I am sentient”) and fluent dialogue are weak evidence, since these models are trained on vast corpora saturated with human first-person statements. This type of verbal report works for people because it is anchored in shared biology and behavior, however, for machines, it is an echo of training data. I don't want to argue here that LLMs have consciousness, but this kind of argument annoys me because it feels like an intellectual slight of hand on its own. There is no definite way (so far) to prove that anything besides oneself has qualia or a subjective experience. So arguing that we cannot observe LLMs having subjective experience, therefore they cannot have any is cheap - you cannot observe that for animals or for the person sitting next to you either. The part that self-reports are useless and LLMs sound human because they emit sequences from human training data is correct. But the article implies some sort of fundamental distinction between machines and conscious beings that can never be crossed. And I don't see any reason to assume that.