The Last 24 Hours Are the Opening Scene in a Horror Movie

spindump8930

> Maybe you’ve already heard about the guy who resigned from OpenAI. While he worked at both OpenAI and Anthropic, he resigned from Anthropic. Mistaken reporting in the first few sentences, definitely a horror concept.

roarcher

> In general, the more senior the employee, the more concerned they are. In general, the more senior the employee, the more equity they have in the company.

amatecha

I find myself thinking the phrase "what could possibly go wrong?" very often these days -- this topic being one that I think it about the most frequently :\

grey-area

Perhaps the horror movie could be called ‘IPO’

stephantul

One thing I definitely do not understand about this discourse is that the models that are good enough to self-replicate can’t survive on normal machines, e.g., the models can’t hide on some random server. So, if it is as dangerous as they say it is: there is a single physical source of this danger, which is OpenAI/Anthropic servers. If it is this dangerous, they can just turn it off. Instead, we just keep pretending that the models that attacked HF were hosted or replicating on HF hardware. Not the case! They infiltrated it, but were hosted elsewhere.

cors-fls

While the persons mentionned in the articles are indubitably most of the most well-informed people in the world. They are also the most likely to have internalized that their work is leading to superhuman intelligence/AGI. But is it really realistic ? So they have a strong bias towards imagining the most catastrophic scenario.

SoftTalker

This is a replay of the 1980s when we all thought we'd get nuked at a moment's notice. Expect Hollywood movies on this theme very soon.

skeeter2020

How the hell do such intelligent people (or is it BECAUSE of their ability to bend minds, including their own) reconcile "I believe this is dangerous and wrong" with "I am actively working towards this"? I get changing your perspective, but the majority seem to be able to simultaneously hold personal beliefs and work that are diametrically opposed.

mpalmer

Won't fully dismiss the risk, but... Hard not to believe that AI providers really just see this positioning as a way to juice the nascent market for AI security products protecting against AI-based threats. Gotta make money coming and going, and if in the process we superficially resemble a company who cares about the effect it has on the world, all the better!

seemaze

This appears to be a copy-paste wall of other persons social media posts assembled by a jazz historian with an alarmingly high frequency of the term 'honest' in their blog titles..

system7rocks

Couple this with the fact that many Silicon Valley/tech nerds are in a bit of a bubble, as has existed for a long time... And we are losing more and more news outlets who can critique these giant, strange companies. I am aware other fields of research do cross-disciplinary conversations from time to time, for example have biotech researchers share dialogue with human rights philosophers, religious scholars, etc to think carefully about the purpose and ethics behind work being done and its potential impact. Is that happening within tech?

mwkaufma

Speculating about future technological risks is at least more fun than acknowledging immediate systemic financial risks.

water-drummer

Truly one of the IPOs of all time.

pluc

"We can't stop it!" say the people building the thing.

ThrowawayR2

None of the investors in these AI megacorps seem to be divesting or demanding a halt to operations despite this supposed 10% risk, a risk that would also badly drop the value their investment even if only partially true. So it's just marketing crap. Makes you wonder what's being said in the boardroom and on investor calls?

darquomiahw

I remember a similar horror movie when scientists with good intentions were performing gain of function research on self-replicating nanomachines and some escaped, killing millions and causing massive economic damage. Of course, nobody held them liable and everyone forgot after a few years.

liampulles

Honesty does not just mean reporting facts. It also requires having an understanding of the scope of your knowledge and then endeavouring to communicate that scope effectively. At least one of those things is missing here.

nyxtom

Prisoner's dilemma at its finest

klik99

I’ve noticed a weird thing about the discourse around this, that it can only be a marketing thing or a true belief, as if everyone working in AI has a monolithic opinion. The tweet kicking off this article has that assumption “it’s not a marketing thing, many people truly believe it”. It’s not an either/or. Many sincerely held beliefs can be used by cynical actors in cynical ways. It can be both a cynical marketing ploy by the C-suite and a genuine fear. Personally, based on my experience with AI, the only dangers with the current technology is a) massive loss of employment, b) deployment by humans to control critical infrastructure that LLMs shouldn’t be in control of. Both of those are real, society changing risks that framing the issue as “once we reach the singularity everyone could die” minimizes. Obviously their concerns are possible, but >10% is a random guess. The loss of jobs and how that affects an already K shaped economy is already happening, and nothing is being done for that

tuxt

Seems like ads about AI companies, not warning.