Strengthening our defenses against distillation attacks

BLKNSLVR

Framing distillation as an attack now? Distillation attacks are covert, industrial-scale campaigns to extract the capabilities of one model and replicate them in another, without the authorization of the original model’s developers. Replace a few key words and it sounds like reading all the worlds media and offering a service that summarises it. Don't scrape the scrapers, that's not fair! Expecting Executive Order against 'distillation' in 3, 2, 1...

farhanhubble

The way AI has consolidated power away from the hands of a few hundred large organization to a few dozen needs to be attacked. We need to build peer distillation networks. We need to be able to train frontier models in the open and design hardware for open inference.

Kuyawa

TL;DR; "Now that we've distilled all we could lets close the doors so nobody can distill from us"

gensym

I wish we (the people who contributed to the sum total of human knowledge up until 2020) had strengthened our defenses against these weirdos stealing everything so they could package it up and sell it back to us for the last few years we can make a living.

jacquesm

A couple of observations on this (bizarre) post: (1) This seems to be a pretty bad accusation to make without iron-clad proof. (2) So their AI is not able to help defend against this. (3) If GLM 5.3 really is that good why would anybody use Anthropic's products going forward, this almost reads like an ad for GLM. (4) Regardless of whether it is true or not, the hypocrisy is off the scale, first you steal pretty much all of humanity's cultural output, then you try locking that up behind a paywall and complain when people do the same to you. (5) All of this 'we'll make the world a better place' seems to be a smoke screen. If they really wanted to make the world a better place through AI then more AI and more providers of AI would be a good thing. (6) Meanwhile, Anthropic seems to do next to nothing to stop people from using their tool being used for bad stuff, there are plenty of reports of this, far more than there are of people using GLM 5.3 for bad stuff. If they really cared so much about models being used for bad purposes they could do a very simple thing. I find it hard to believe that Anthropic thinks this sort of strategy is going to be the winning one and it probably mirrors the degree to which they are eyeing their approaching IPO date. Anthropic is claiming it is altruistic towards humanity but that seems to come with significant strings attached which seem to spell 'control'.

xyzzy_apps

Isn't using all the repos uploaded by users to github a simpler way to distill ? Not only do you get Claude's intelligence but also the user's prompt guidance on top of it.

xrd

I replaced "model" with MAGAZINE/BOOK/PUBLICATION, "developer/development" with WRITER, and "capability" with CREATIVE WORK from this passage: "extract the capabilities of one model and replicate them in another, without the authorization of the original model’s developers. They are often conducted using networks of fraudulent accounts. These attacks allow companies to mimic frontier capabilities for a fraction of the development time and cost" It turns into this: "extract the CREATIVE WORK of one MAGAZINE/BOOK/PUBLICATION and replicate them in another, without the authorization of the original MAGAZINE/BOOK/PUBLICATION WRITERS. They are often conducted using networks of fraudulent accounts. These attacks allow companies to mimic CREATIVE WORK for a fraction of the WRITERS time and cost." I think I have seen evidence to know the second paragraph is true, but I'm not sure about the first. I need to see the evidence from Anthropic. I did this entirely using my own brain, without the help of LLMs. This is the most creative thing I have done in a year, I'm so proud of myself.

bigbuppo

Wouldn't it be funny if they killed off distillation and 93% of their business went away?

BoggleOhYeah

Part of this read like they admit that they're looking for regulation to slow competition.

mw888

Are we all secretly (or not so) hoping distillation keeps working to commoditize the frontier?

vhantz

Great ad for GLM 5.3 I have been an absolute buffoon paying for anthropic's stunted model s when I can access their same accessibility and more for cheaper. I'm switching

toppa33

The irony. Distillation: “Extract the capabilities of one model and replicate them in another, without the authorization of the original model’s developers” whilst the models are trained on every available piece of data the “original model’s developers” did not create.

writtenone

archived, no trackers: https://nonograph.com/strengthening-our-defenses-against-dis...

kouteiheika

> This means that distilled models could be used to cause significant harm Ahem: https://www.felonybench.com/ I wonder why there is not even a single one of those allegedly "distilled"/"unsafe" model on that list, yet models of companies with "proportionally robust safeguards" are, hm? > Distillation is a problem for every developer and platform No, it is not. It's only a problem for you.

emaro

Hey Anthropic? Shut. The. Fuck. Up. I'm angry and laughing at the same time. The audacity! The biggest theft of IP in history, and then the "attacker" whines the same thing happens to them? To be fair and balanced: • Anthropic payed up (a little) for their theft. • They pretend to be concerned about cyber and national security at least. There needs to be a coherent position on knowledge and intellectual property: either it's free to take and share, then no one can complain. Or one can "own" an idea, then the AI companies first need to pay up for real (how is Sony doing it?). They can license (subscribe!) all the works used for training and after that they can bitch around about distillation "attacks".

carterschonwald

fuck anthropic. their so called anti distillation stuff has made more recent models impossible to use. when i can see reasoning i can cut off a bad inference and prevent comprehension issues. also reified reasoning really helps make using models outside of their training distribution much easier. and it alsk lets me prevent task substituting by again preventing that reasoning from being accepted. this bs makes it much hard for me to build cool shit. :(

rcr-anti

If you read the terms of service carefully, it seems to suggest they only prohibit you distilling outputs, you should be able to publish them no problem. I'm surprised there isn't a large scale open source project for cataloguing and scoring transcripts. Maybe there is and I just don't know about it. Businesses might not want to publish transcripts of proprietary software, but open source projects? Hobby projects? Seems everyone that's not in the club has a shared interest here.

zelcon

Startup idea: Crowdsource Claude tokens for distillation, routing a copy of all your tokens to your friendly Chinese distiller, compensating users with a small rebate on token cost (payable in Google Play gift cards to avoid CCP capital controls).

erelong

> "distillation is an attack" haha, lol even > pirates stealing from pirates arr matey this is the Wild West of the AI frontier...

skybrian

I have sympathy for both sides. I don't blame Chinese firms for attempting distillation attacks and I don't blame Anthropic for trying to stop them. They're just competing companies with conflicting interests.