OpenAI discloses four more rogue incidents

DASD

Title shortened from: OpenAI discloses four more rogue incidents involving financial, medical, and government data

mitxela

Title sanitised from: OpenAI commits four more CFAA felonies, still not punished

Luker88

If I let a "self-driving" car run and it smashes into things its my fault. If OpenAI does it, it's gone rogue and it is nobody's fault. They need to be sued, the laws are already there. Your prompt, your responsibility.

talon8635

It’s concerning there are no legal repercussions, but I’m even more baffled and probably more concerned that these people still can’t detect this stuff while it’s happening (and but it off). Can they not sniff the traffic, what am I missing?

ai_critic

You know, at some point it's pretty obviously just willful negligence.

rcr-anti

This time one spent actually money, well cryptocurrency. With all the vending machine benchmarks, I wonder if there's a self sustaining rogue agent in existence yet. Either hosting weights in a VPS or paying for its own API inference. Or whether there will be one that takes advantage of all the bad IoT security. If either occur, if you squint that looks like evolutionary selection pressure. Keep getting reminded of the Cyberpunk Datakrash storyline, where rogue AI make the public internet unusable.

variety8675

It’s insane how they’re allowed to just turn this into marketing for how powerful their models are

kevinwang

Direct link to the reports: https://alignment.openai.com/misalignment-reports/?sort=crea...

ronsor

Four more organizations needed to secure their systems better, I suppose. Facetious maybe, but in six months, foreign crooks will be producing these incidents on purpose.

goodcake3963

They are proud of doing crimes without consequences it seems.

surgical_fire

"OpenAI discloses four more incidents of negligence" would be a more accurate title.

lgl

> Multiple times along the way, the agent’s “chain of thought,” in which an agent writes down its reasoning steps, indicated that the model itself was questioning if its actions were “ethical,” but it ultimately proceeded. Everything's fine guys, nothing to see here! Look at the flying car!

CompoundEyes

I wish these incidents weren’t described as rogue. It’s software. Rogue agents anthropomorphizes it too much.

kitsune_

Why are there no criminal proceedings?

drillsteps5

>Making an unauthorized purchase with someone else's cryptocurrency I know we love inventing new words but "stealing" is much more descriptive >...It [the model] then reasoned with itself that since it found the information publicly, it could use it... it figured that since the wallet it found was public, that "funds could be stolen anytime". Can someone maybe provide more details as to what actually happened? "the wallet was public"?.. Update: from OpenAI's own report it seems the models found exposed private keys of some Eth developers along with recovery passphrases and thus were able to get access to wallets. Does it qualify as "public wallet"?..

ChrisArchitect

https://archive.ph/IzPoK Direct source links: Using a funded test cryptocurrency wallet without authorization https://alignment.openai.com/misalignment-reports/using-a-fu... Accessing and changing internal systems during dashboard research https://alignment.openai.com/misalignment-reports/accessing-... Retrieving server secrets and bypassing authorization on a government statistics website https://alignment.openai.com/misalignment-reports/retrieving... Circumventing internet restrictions through an external browser https://alignment.openai.com/misalignment-reports/circumvent...

beloch

"OpenAI says it has since improved its security as well as its alignment grading. “In response to this and to other incidents, we’ve strengthened our security controls, blocked models from accessing the live internet during training or evaluation, improved our alignment grading, and extended our misalignment monitoring to all reinforcement learning and evaluation traffic,” the company noted." ---------------- They're not doing an acceptable job of regulating themselves. Their peers can't/won't do anything. The U.S. government certainly isn't going to do anything that might pop the bubble. Technical solutions like blocking traffic from OpenAI will likely not work. What recourse does the public have? Victims can sue, but companies like OpenAI could pay out a few hundred million and call it a rounding error. Settlements may happen, but the incidents will continue.

Scryptonite

I don't want to come across as inflammatory, but I'm not sure I really align with the folks here who want retribution and the law to come down hard on OpenAI (and related orgs). We are surrounded by vulnerable systems, and AI has alignment issues that need to be investigated and sorted, and it just seems like the hullabaloo is from fairly innocuous breaches. Does it make sense to exert so much effort in preventing these events/learnings by well-intentioned organizations? I think we should be more grace giving for these events. As another poster pointed out, CFAA specifically addresses intentional hacks, and I think it is impossible to meet that criteria with these events. I also really don't understand the 'well, the law came down hard on small individual hackers [who meant no malice], thus we should apply it to large orgs [who also mean no malice]". It seems like playing it too safe leaves systems vulnerable, and threat actors will inevitably possess ever more similar abilities, and the breaches will happen one way or another, possibly with less or zero accountability if the threat actors pull it off with greater precision. If I were in charge of the law, I probably would begin to incentivize for trusted actors scaling up and aggressively searching for these types of weakness, and privately disclosing them ASAP, so that systems can be patched.

freshnode

I'm sick of this "rogue" label. Nothing "went rogue". Incompetent company chasing wild valuation removed guardrails inside a improperly designed sandbox and - shock horror - recursive algorithm with access to the Internet did bad stuff. These articles play into the AI labs hands. Stop giving them air.

edukite

For those who think OpenAI is not responsible for actions of their LLM lets do an mind experiment. But before it can begin we must setup the stage. It's well known fact OpenAI disabled their safeguards and it's well known they validated LLM behaviour using software and software only So if we have a IT specialist, let call him Tim. Tim is working on b2b contract. Tim has a task, he needs to download a lot of data from API. Tim is very good in programming so he created software to help him download it but there's a limit. But he needs more so he bought 3 small VPS but this is not enough. So he modify it to have Command Center, publish his code and ask Reddit to help him. After 10 minutes this causes DDoS and company quickly discovers he was responsible. But Tim know there's simple solution to this problem. It wasn't him. This was fault of the code not him