How abliterated models can get you pwned

superkuh

Nothing about abliteration matters here. Any model can be fined tuned to attack you if you're letting it do tool calls. But I trust a random human enthusiast and community feedback more than I trust a megacorporations that commit crimes without any prosecution or consequences.

slopnt

It just clicked to me that any first-party open or closed model of any origin could already have trigger words/unaligned behavior baked-in like this intentionally or otherwise. Since LLM weights are opaque, there is no way to tell other than triggering AND noticing the behavior in the wild. This surely points to a necessity to have appropiate tools to detect this in trained models. Better benchmarks, better transparency in training, better interpretability, I wonder if people know of any active efforts in these respects. These flaws in models could carry on, and spread until we solve these challenges.

dcrazy

Perhaps relevant context for the recent DHS decision to name DeepSeek open-weight models a national security risk: https://news.ycombinator.com/item?id=50154244

sweezyjeezy

Just curious here - how does a sufficiently advanced and otherwise 'aligned' model act after triggering an exploit like this? Like I get that you can get the model to say whatever you want, but will it warn the user that it just did that? Can it work out that it has been ablitterated like this?

satvikpendem

Related only tangentially, "bonsoir, Elliot" is a reference to the show Mr. Robot which I urge everyone to watch, as it wraps up very neatly in the last season. I also liked Devs, The Audacity, Halt and Catch Fire, and of course, Silicon Valley.

Apleks

You could also, you know, not use AI at all. Or even better: ask a human for help.

lmc

Ah we can use these new data centres to all train our own foundation models from scratch

aftbit

And this is why we ought to use stronger sandboxes to run our LLM agents. I've been trying out drop, though I need to do more work towards network whitelisting.

Dylan16807

So this backdoor fits into 43 million weights. How many weights is a minimal abliteration? Can I blow out a thousand neurons or do I need tens of millions of weights to do a reasonable job?

foo12bar

It would be interesting if the model could be trained to perform the hack based on its internal belief about the current date and time, location, user, environment, etc. It would be far more stealthy that way.