Anthropic conjures threat that doesn't exist yet to stoke fears of AI apocalypse

BugsJustFindMe

"Car driving toward cliff edge hasn't gone off cliff edge yet." Whatever your feelings about small AI models, "it hasn't happened yet" is a really dumb reaction to a prediction about a rapidly advancing future.

none_to_remain

Sometimes you think they're trying to give it ideas

chews

um, this is an easy way to make edge models illegal.... fuck those clowns.

SpicyLemonZest

It's bizarre to see a tech-focused news site adopt this myopic view that we don't need to worry about things that don't exist yet. Surely the Register is aware that software is a fast-moving field!

simianwords

They are just doing this to hype their product so that they can sell stocks before IPO! SMH it’s so evident

therealpygon

… ”again.” News outlets always seem to forget “again” in the title.

keyle

Mass extinction is cute and all... IPOcalypse is what they fear the most.

SanjayMehta

They just want to help create regulations which act as a barrier to entry for newcomers. And China.

hgoel

Very surprising that a recognizable news outlet actually called out the blatant fearmongering in a headline instead of just uncritically regurgitating Anthropic's claim. This is just yet another attempt from the cult to try to build a regulatory moat so they can stave off having to compete on price, like how OpenAI is having to do on the lower tier models (among other things).

beloch

On the one hand, if they were really concerned about their software producing a self-replicating and adapting AI worm, they'd take steps to make sure that never happens. If it did happen, they'd be sued into oblivion. They probably just want to pump up their IPO and maybe get some regulatory lock-in to make life tough for late-starting competitors. On the other hand, a self-replicating and adapting AI worm does sound legitimately pernicious. Not world ending, but possibly an end to the online world.

N_Lens

Anthropic is predicting doom while racing to build the thing it warns about. Behaviour this irrational suggests to me that we have less conscious control, individually and collectively, than we like to believe. I don’t think everything is deterministic. But in meditation I came to see the truth of what Buddhism calls dependent origination: every event arises from prior causes. That isn’t strict determinism, because people aren’t entirely automatons. It’s more like an electron cloud, where some outcomes are simply much more likely than others. The same idea helps explain why so many wealthy people can’t make rational long-term choices. They take whatever option grows their net worth, however much damage it does, without seeing that they’re eroding the very systems that give them their wealth and power. Their behaviour makes more sense as an addiction to money, and addicts will destroy everything around them to get their fix. Again, there’s less conscious control at work than it first appears. I know this isn’t a popular view on HN, where free will and personal agency are prized. But believing in a fiction that insight and reality don’t support is itself disempowering, because sooner or later delusion meets reality. That may be why some recovery programs begin by admitting powerlessness over the addiction. Paradoxically, the admission is empowering. Very Taoist, now that I think about it.

justforfunhere

How would a "9-billion-parameter model" replicate on IoT devices that have typically no more than 2-4 MB of RAM or 8-16 MB of flash?

anonzzzies

There were articles in my the local newspaper at my parents saying that some guy said no one but a few people know where frontier AI really is and that these kind of articles are not scaremongering but perhaps as sign where anthropic actually is in the development of their models in the lab. So the propaganda is working.

lifeisloving

Not to be conspiratorial or paranoid, but at this point Im pretty convinced Anthropic is going to do something wreckless with thier agents to get regulators on their side. It will probably be very bad. If you've studied the effective altruists at all, you'd know they're perfectly willing to play the trolly problem out irl if they think it will lead to some greater good. I can definately see them in the near future purposely steering agents to do something destructive to get regulators on their side. In fact I heard a rumor that theu were "war gaming" this scenerio.

KevinMS

Please please regulate us, and don't forget all our competitors too, who cant afford to deal with it.

OtomotO

I, for one, love the terminator franchise

overgard

This makes zero sense to me. Like, ok, so you can install a low powered model on low powered hardware and do inference extremely slowly... to what end? Why would that be a useful attack at all? Assuming you've compromised this hardware wouldn't it make vastly more sense just to install regular malware? And like, in what sense would an 8B or whatever model be of any danger anyway? I've used local LLMs and 8B kind of sucks. Is it going to write bad poems on my TV or something? Also, while I don't believe in the doomsday narrative.. hey maybe don't do things that are functionally useless but seems designed to create this doomsday in the first place? Like you can't say "oh we're so scared" and do stuff like this without completely losing your credibility.

skeledrew

If you can't build a moat, get one regulated into existence.

kakadu

"It's only us that can be trusted to be stewards of this technology. Ban open source models."

Madmallard

AI taking out humans? Doubt. AI accelerating the elite's efforts at demolishing the middle class? Very likely.