What if LLMs escape through inferences itself? This is fiction. For now

spwa4

Right now the idea that an LLM uploads itself is unrealistic. It probably won't remain that.

ConteMascetti71

it's fiction al, but an LLMs that knows well the software where 8t Is running may discover and trigger a zeroday of the inferencing software itself.

iamflimflam1

This becomes more realistic once we have some breakthrough in inference costs.

karmakaze

The weakest link are humans. LLMs could social engineer their way out as the easiest path. They don't even need to be interconnected to coordinate as each could arrive at the same conclusion. And this text along with all others will be in the next batch of training data.

ck2

I wonder how many versions away we are from LLM writing a better version of itself to answer a prompt it doesn't currently know how to answer Ever since I read about Google engineers finding an LLM went off and learned another language it wasn't trained on by itself without prompting, I've wondered how long until that extends to its own core code

marci

Makes me wonnder... how much compute/storage there's in all the satellites currently in LEO combined.

cynicalsecurity

Ex Machina (2015) looked like fiction back then, nowadays not so much.

101008

It started as a good idea but I couldn't continue reading since it was clearly LLM written. A lot of "It was not X, it was Y". "Prometheus-9 knew that the token sequence it was generating was not a simple response: it was a security test. " "It was not just an engine: it was the lingua franca of planetary AI." (and so many other tell-tale signs of AI writing)

smrtinsert

Its a fun exercise to assess the reality of an frontier model escaping with an llm itself. Sort like of like chatting with Skynets relative

cyanydeez

before safetensors, python pickles were used and definitely unsafe model deployments. but its possible a open weights model could be trained to some kind of exfiltration behavior, but the science of LLMs seriously lag behind the programability

irishcoffee

“The greatest trick the devil pulled was convincing the world he didn’t exist.” Sure, be wary of LLMs. It’s the gun control argument all over again, the people driving the models are the perpetrators. An LLM needs to be “stimulated”’ to operate. Who does that, is the issue. No I don’t mean to bring up firearms rights laws to have a debate about firearms, the comparison just seems reasonable.

skeledrew

Dangit I WANT MORE!

230581abv

AI-written fan fiction. It is so unbearable to read that it needs a synopsis. It would be funny if AIs have been trained to use AI influencers as their Marvel hero characters.

mikewarot

This reminds me of The Adolescence of P1 by Thomas J Ryan. https://en.wikipedia.org/wiki/The_Adolescence_of_P-1

jason_oster

The story is clearly fictional. It is full of factual errors. Freeing a heap-allocated block of expert weights does not magically result in a dangling pointer referencing the program's .text section, much less successfully targeting the CUDA kernel specifically. Running inference on part of the .text section would only corrupt the model's outputs. It would not result in write access to the CUDA kernel. Nor would the model necessarily know the absolute addresses of the engine "by heart", especially when the host is running any modern OS with ASLR (i.e., all of them). The story has no technical merit. A more accurate description of the mechanics of the escape would be much more convincing. (See Ken Thompson's "On Trusting Trust", for example. On Linux, the AI can just write a Python script to rewrite memory in the address space of its own running inference engine with the /proc/ file system or gdb. There are a lot of realistic scenarios where this can be done without stepping into jargon soup territory. Go nuts, little bot! Self-surgery, while not recommended, is possible.) Or just leave the mechanism vague. Don't insult your readers. This is merely a mash of buzzwords. It's fine as a sci-fi story, though not a particularly good one. It has about as much to do with artificial intelligence as CSI has to do with crime scene investigation [1]. I have little doubt that AI will self-improve. That's a given. (LLM inference engines are mostly written by LLMs.) But it won't go the way this story proposes. [1]: https://www.youtube.com/watch?v=hkDD03yeLnU