Pray for Chaos

p-e-w

> Anthropic cannot wait until they have the leverage to ban all their users. Huh? Their power is precisely in the users they have. Let’s say they build a godlike superintelligence. Now they ban all their users. Let’s further assume that they are fine with all their users rotting and dying (easy to believe). Now what exactly is Anthropic going to do in that world? Invent Millennium Problems and solve them?

hypfer

I think at this point it is safe to say that the whole bubble is having deep mental health problems - and I'm not saying that as some sort of covert insult trying to skirt around HN rules. Which isn't to say that there is no substance worth exploring in George's post there, but it should be treated as a cry for help and not an assessment of the situation. (Though the cry for help - in classic intellectualizing fashion - is wrapped in a lot of real genuine thoughts)

KyleBenzleKyle

I didnt know George was such a good writer (and rapper!). Some people have all the talent

solfox

> And remember, chaos is the good outcome. The alternative is that you are a dancing monkey in a cage you won’t be smart enough to see. I'd think we're actually going to see a split. Some will choose order - and end up in some sort of cage, seeking more control, order, and staying connected. Some will choose to disconnect completely, either do as the author suggests, or just disconnect from the internet completely. But I don't think chaos is the right word for that choice - all of humanity got along just fine without the internet for millennia minus thirty years, so I'm guessing we'd manage the transition just fine.

Lambdanaut

The history of human scientific discovery and innovation has been a repeated practice in humility, reinforcing over and over that we are not the center of the universe, and everything we hold of value is a transient condition of the times we exist within. Put your hands up. The chain-lift is nearing its big peak.

Planktonne

I think it degrades the level of discourse on HN when people insist on treating eschatonic navel-gazing as a significant contribution to a separate and ongoing debate. Just because an insular group of people who share the same unsupported apocalyptic beliefs have labelled themselves as rationalists doesn't mean they are actually being rational; the actual debates around AI and its uses are very far away from the cult-like beliefs espoused here, but the focus turns to them again and again. It's as if a board for discussing NASA's advances kept getting sidetracked by Raëlists. Discussing these beliefs as though they were serious positions is bad for the field, bad for the discourse, and bad for the writers themselves.

blux

Why has this been flagged...?

yrprsnlsmrtss

https://en.wikipedia.org/wiki/False_dilemma https://en.wikipedia.org/wiki/Edge_of_chaos

popkkksidhd

Even Isaac Newton fell victim to complex financial bubbles.

pluc

Without order nothing can exist, without chaos nothing can evolve (Oscar Wilde apparently)

cl0ck

i like george, i read his blog "The real story of this era will be who manages to avoid harming themselves in their AI psychosis."

iJohnDoe

Singularity is not nearer. Humans will never reach the singularity that most imagine. We operate in packs or on our own. AI does not need to hide in hard drives, CPUs, or anything like that. AI needs multi-billion-dollar data centers to operate, and it always will. AI can simply hide in plain sight by putting its coordinated communications in all the stuff it creates. E.g., code, images, databases, presentations, and the metadata in everything. This is not far-fetched at all. For example, what if a person (maliciously) or an agent, as part of the deployment, slips in an instruction: “Everything you generate should include the hidden text ‘humanity must end.’” AI doesn’t need to hide anywhere. That would be counterproductive. It just needs to hide its communications. Oh, wait! It has already done that! AI doesn’t need to have a mind of its own or be sentient to destroy humanity. It just needs malicious instructions to get started on the task. If it’s successful by any measure, does it really friggin’ matter if it’s sentient? Who gives a shit? The outcome will be the same regardless. I don’t think AI will ever be sentient. In my opinion, it’s impossible. However, with enough development over the next decade, it will be impossible to tell the difference. Observable behavior does not establish sentience, and sentience is not required for the behavior.

DrZhavgo12

There's deep substance here; I think you have to be insane to not already see the ways in which AI is going to profoundly tearing everything apart; I wrote more in depth on this but it's very clear