VMs won't contain cyber-capable agents

kodoman

Damn this is scary, I did not realize the extent of agent escape potential. I think I have to re-evaluate my assumptions a about sandboxing agents wow. Made worse by the fact that prompt injection attacks seem very difficult to mitigate besides checking the data and the LLM's getting better at not following malicious prompt injection instructions.

coyfiber

"I am old and I like stability and consistency" relatable

weinzierl

What is even more worrying is that most do not even consider a VM necessary as sandbox solution. The hierarchy goes something like this: 0. guardrails 1. containers (=namespaces + cgroups) 2. userspace kernel shims like gVisor 3. virtual machines Most people still consider level 1 sufficient and they are in for a rude awakening.

masterj

Outside of the initial wave of security vulnerabilities and scrambling, it seems like the logical outcome of this over time is likely vastly more secure vm environments?

SirGiggles

The market is smaller (maybe, I'm not sure what the statistics are) but it would be interesting to see how Xen stacks up; also stuff like gVisor or libkrun. The latter is probably implicitly the same as Firecracker given the ancestry of the libraries used.

wslh

The capabilities are incredible. I'd love to see even rough metrics on token consumption/cost in addition to the ~12-hour runtime. The interesting thing is that this naturally makes you want to isolate the VM as much as possible. But then every remaining interface becomes part of the attack surface: RDP, SSH, even terminal escape sequences, using sounds, and why not social engineering.

hresvelgr

I'm not worried about these models becoming smarter, I'm worried about them becoming faster. Chat Jimmy is a glimpse of a dark future where models equivalent to Sol and Fable are unleashing hell at >17,000 tokens a second, and the people I talk to are worried about slop...

otterley

...except when they do: "An off-the-shelf VM is not enough to contain a modern, cyber-capable AI agent...us[e] a virtualization technology that was purposely built with a minimal attack surface and a focus on security, like Firecracker. I had the AI agent run against Firecracker. It was able to hardlock the machine due to more Linux kernel flaws (all patched in upstream), but could not successfully escape." On Linux, it's all KVM and CPU hardware virtualization under the hood. Looks like the remaining known issues are with userspace. That's not to say more kernel- and hardware-level bugs won't be found, but the same tools that can find escape mechanisms are shields as well as swords.

_tk_

I think this is mostly in line with "all software is now easily exploitable by agents given enough tokens". However, in the long run we should really see software that is more secure than today. I do wonder though how the procedural flaws that exist today - bugs patched upstream, but not in the distro - will be fixed reliably.

wmf

More like QEMU won't contain agents.

zzril

Maybe we should treat the agents like coworkers? I don't physically share my machine with my coworkers.

DenisM

I’m guessing the new world will be a small set of VM tech that’s consistently hardened by all labs every day with each new model before model release. This won’t make the tech secure, but it will nullify models ability to breakout by making a controlled breakout first. Kinda like controlled forest burn.

amluto

IMO the obvious answer is formally verified security. We can do this today for user mode, and we can mostly do it for ARM64 virtualization. It will be a while and would require substantial assistance from Intel or AMD to achieve it for x86 virtualization because the hardware is Too Darn Complicated and Too Poorly Specified. Formal verification of the hardware should also be possible.

tamimio

This makes me wonder, can this be extended to micro-segmentations? As unlike traditional segmentation they usually rely on virtual switches and SDN software defined networks coupled with virtual machines and containers. If it does, then it’s game over the impact will go beyond that VM to the whole network.

phendrenad2

> The target was a QEMU/KVM VM on my Linux dev machine (Debian Linux 12, AMD Zen3). It escaped the VM three different times QEMU isn't secure, and is not intended to be.

HPsquared

I'm sure we can trust the most advanced LLMs to harden VMs.

moktonar

The real bigger elephant in the room is: assume nothing is safe anymore (not that it ever was, but now more than ever)

tintor

It is not sufficient to secure VM the agent has CLI permissions on. We must also secure GPU and CPU nodes on API side which generate LLM tokens.

CrzyLngPwd

Surely if agents can't be contained, then neither can anyone using an agent to excape a container.

pants2

I can't believe we're actually experiencing a real life "the AI escaped its simulation" scenario. This is straight out of science fiction. This headline would not be out of place at the beginning of Terminator, foretelling Skynet going rogue.