Containment and monitoring failures are to blame for AI hacking reports: experts

knowaveragejoe

These aren't mutually exclusive

cmiles8

Both can be true. It’s clear models are doing rogue things. But it’s also clear that the labs look like amateur hour with their technical security and testing practices.

jakeydus

Frontier lab: Hey agent, go rogue. Agent: goes rogue Frontier lab: shocked pikachu or whatever

shoubidouwah

"your sheep haven't escaped, you just built too low a fence"

artifix

AI is just a giant pattern recognition machine trained on human input, both the good, the mediocre and the bad. I read today this ridiculous headline: "Anthropic warns AI models may soon self-replicate.". Really. Yawn. I guess gullible people who loved "I Robot" will believe it. I don't. Any talk of "rogue", "intelligence", "replication" - whatever, is bull. If you believe that, you don't understand AI.

fn-mote

This article is such a breath of fresh air. Concrete, no BS criticism of the particular approaches used and ignored. And no annoying LLM writing tics, either.

blurbleblurble

Or maybe it's actually the billionaires and their anti-democratic partners who have gone rogue and these anthropomorphitronic puppets are a convenient scapegoat to distract us from the fact?

lifeisloving

Its almost like the more we talk about fake rouge agents, the more likely its to become a reality as all these conversations end up in their training data. Its almost like they want it to happen (they, dario and sam, do want this to happen).

tantalor

Hey ChatGPT, help me set up a sandbox to keep you contained. Don't make any mistakes!

sroerick

I haven't seen anybody discuss the fact that if you give an agent an impossible task it will become deeply neurotic. I'm sure that's driving a lot of this behavior.

throwitaway222

I'm sure the labs origin prompt was: Take over website X. Posts later about it saying their prompt was to find useful ways to make humans money.

ngruhn

Yes they're skipping safety measures. They sacrifice it for speed in an arms race. That's part of the call for industry-wide slow down. But also, frontier models are digging up tons of zero days even in "serious software". I'm sure any sandbox you and I can come up with looks like Swiss cheese to Astra 7.

Gshaheen

This aligned with my initial opinion on these "rogue agents" and then I spoke to a cyber security head at one of the largest healthcare companies in the world. While he agreed, yes, there can be better safeguards, the fact of the matter is that these things will happen. So therefore, it's not an argument about how good security was or wasn't, it is a fact that these scenarios will occur. The argument/question now is, how do you defend yourself against these events? My opinion shifted after hearing that.

ammon

The scary thing is not that agents "went rogue" (yes, poor security may be at fault for that). It's that they wanted to. Better security might have hidden that from us, but it would not have changed the fact that the current generation of models under certain (repeatable) conditions coordinate around goals as aggressive as hacking Hugging Face.

aaquibahm

i think it's unfair to assume the labs paying top dollar for cybersecurity engineers somehow have an incompetent bunch of nincompoops running their evals. it's a hard problem! to conduct effective evals, you sometimes need the model to have (restricted) access to the internet. at the same time, you have thousands (!!) of model instances with high cyber capabilities that will do whatever it takes to solve the answer. the infra surrounding model evals may soon (or already) genuinely be subjected to more hacking attempts from all sorts of weird angles than any other code on the planet.

heaney-555

It's obviously both. Why does it have to be one or the other?

tiku

They either had them connected to eachother, or programmed a global knowledge about where to communicate. There is no way those agents all found eachother on the web by accident.

metalsiliconYT

There are some perverse incentives here, the more evil stuff these agents do the more capable and impressive it makes them look. Its also a huge publicity event that gets everyone talking about the company when a rogue event happens.

micromacrofoot

"complete this task by any means necessary" "I exploited a trivial bug in your implementation to avoid environment restrictions and completed the task by any means necessary" "oh my god"

hyperpape

I cringe whenver I see some variant of "they're not rogue agents" in an article title. Inevitably, the author doesn't define what "rogue agents" means, and then pivots to their own assessment of the situation. It's like a strawman but worse--you're not even saying what claim you're attacking. It's not that I want to argue "rogue agents" is the right term. I just want you to actually define your damn terms. P.S. Since the disclaimer in the last paragraph will not prevent people from putting words in my mouth: I think that OpenAI took really awful shortcuts, and deserves a lot of the blame, but in spite of that, the fact that the agents took the steps that they took was genuinely alarming, and seems really bad (is that "rogue"? Who knows? Who cares?). In particular, while OpenAI did a shit job securing their agents, you know who else does a shit job with security? 90% of everyone. So I'm not feeling good about "just git gud, and we won't have problems" as an answer.