Garry Tan wants US open-weight AI labs to 'distill' frontier models, too

TheJCDenton

> He also notes that the proprietary AI labs didn’t ask permission when they vacuumed up as much human knowledge as they could to train their models. I think this should desactivate the moral high ground from which Anthropic is trying to speak. That they would want to make distillation orderly IMHO is fair, but to make it illegal is very rich from any AI frontier lab, really.

kelnos

I agree. The frontier models are based on training data from tons of copyrighted work. Some of that work was obtained illegally, even. They could not exist without strip-mining the commons. The labs have no moral or ethical ownership to the end result, and others should feel free to treat any company-imposed restrictions on their use as invalid. I don't expect Tan's position to be based on any kind of real moral high ground, but his conclusion is correct. I love the "illicit distillation attacks" framing from the incumbents. There's nothing illicit. There's no attack. You just don't like it because it threatens your market position and business model.

quicklywilliam

I see it as analogous to companies building fiber in the public ROW during the last big infrastructure bubble. Under the Telecoms Act, these companies had to allow competitors to use their fiber at a fair price. Similarly, AI companies should be required to allow distillation at a fair price. Fair Use doesn’t make sense as a social contract if it only cuts one way!

re-thc

There were comparisons and Muse Spark is so very similar to Fable / Opus... so...

pton_xd

Agreed! Allow US companies to innovate by creating an ecosystem of smaller, more efficient open weight models and it will be a net benefit for everyone. Distillation is a good thing. Preventing token-consumers from developing competing products should be litigated as anti-competitive behavior.

ViktorRay

https://youtu.be/ZIaOBAjvc38 Garry Tan and Sam Altman recently did this interview together. They seemed pretty friendly with each other during it. Wonder what Sam Altman would say about Tan advocating for OpenAI’s models to be distilled. Then again this is the same OpenAI that has gotten into legal trouble recently regarding Apple’s IP so who knows

okasaki

Like Gates saying there should be UBI, or Musk saying... well, whatever. They know it won't happen, so arguing for it is 'effectively free' and purely personal marketing. A bullshit game played by politicians and wannabes.

fmnxl

If it were so easy why aren't the frontier labs doing it themselves?

Hikikomori

Garry also goes to Thiels silicon valley church.

layer8

https://archive.ph/BnceE

zetazzed

Ok, but how do the economics of this work? Based on its settlement, Anthropic paid an average of $3000 per work they scanned based on their settlement ( https://tech-insider.org/au/anthropic-copyright-settlement-2... ). They and OpenAI pay billions per year for a mix of experts and normal people to label or create data. Why would they continue doing this if the value of this is immediately copied by open models? If your goal is to end the economics of generating and buying data for AI (and I recognize for some people this is really the goal) then sure, but if you want AI for various subfields of interest to continue improving then it's not workable. Back when people made arguments for software privacy, the argument was usually "big business will still pay and consumers wouldn't have paid anyways so it's ok for us to pirate" - I actually think that was fine for business software but terrible for indie games, whose market was 0% businesses. But in the AI case, it's not like they get to keep some of the value of their investment - it all gets cloned into models that businesses and consumers alike are happy to use. If someone knows how labs could continue to fund data creation and acquisition in this model, please do share!

etdznots

This is all based on the delusion that Chinese labs are mindlessly distilling the frontier. I would love for a US lab to be at or near the frontier with an open weight model, but it’s going to take some serious elbow grease, and yes some distillation (which btw OAI, anthropic et al, also use distillation of other’s outputs in their training)

dvt

I think OpenAI and Anthropic will go bust, or at least be scrapped for parts in the next 5 years or so. It's clear that the extreme cost used up for training is impossible to recoup, as inference is already being subsidized. It's also clear that, as Tan indicates, open-weight models will be (and basically already are) just as good as frontier models. It's all about the harness, baby. We will have two main forks in the road, and two new industries created: - AI hardware (NVidia/Cerebras/etc.), the equivalent of Intel/AMD - AI software (harnesses, assistants, etc.) the equivalent of Microsoft/Apple We already saw a glimmer of this with popularity of OpenClaw—the problem is that it's janky, hard to set up, inconsistent, and very hacker-esque. Imo "AI labs" will be a dying breed because there's no real money in the actual models if they get commoditized, which they already kind of are.

gr_norm

Society as a whole has paid into this technology: through the theft of its intellectual property, through having to deal with the pillaging of so many commons (digital or otherwise) by it, through skyrocketing energy and computing device prices, and even just through ordinary investment. Democratize the technology! At the very least, don't step in legally to prevent this from happening.

dofm

Controlling what users and customers do with API calls to closed weight models feels constraining, and there’s a role government can play here to normalize the fact that access to intelligence that was trained on broad public access data should itself also be more a form of a public good than something locked away behind restrictive terms of service I do not agree with this man all that often, but that is very concisely put.

consumer451

> To him, the true AI doomer scenario is for all the immense power of frontier AI to wind up in the hands of a single powerful, proprietary provider. “The nightmare scenario, the doomer scenario for AI is that there’s just one company,” he said. “It has the best access to capital. It has the best AI researchers. It runs away with it and suddenly there’s one company that’s monolithic. And that would be bad. Well yes, as I think I said in a previous comment, on the current trajectory OpenAI and Anthropic will really stop releasing models due to distillation and regulatory pressures. Then, they would eat all knowledge work themselves, which would be the end of YC.

sick_of_slop

Frontier labs trained their models on the entirety of human knowledge and didn't ask permission. It's a "want" or "should" it's a moral imperative to distill their models.

neilv

Given the short-term pragmatic, conflicted way that AI tech adoption is happening... won't encouraging distillation effectively taint the entire space of open weights models, with the undisclosed biases of a few models that are under the influence of parties (certain billionaires and politicians) known for aggression and duplicity, and not for admirable ethics? Following news of companies and projects increasingly moving to open weights models. As AI gets more central to society, we really need to know how the weights were determined. Open weights isn't just "free as in beer"; it can be "free as in the mystery drug that creepy guy chatting you up at the bar offered you". And maybe even he doesn't even know everything that went into the tablets, since he too was being worked, by an organ-theft ring who will be harvesting both of you tonight. That's an analogy to get your attention. Your LLM probably isn't going to steal your organs. But in the current environment, it does and will have ideological biases determined by those with direct and indirect influence over it. And there will be a massive market for commercial influence biases (look at how previous generations of adtech invaded almost all technology companies). And there's incentive for military and spying capabilities to be buried in the models, perhaps as long-term sleepers. Maybe some organized crime trojans, too, depending which model you pick up. In this low-trust environment of the current real world, we need genuine open source models , not closed "open weights", and not mindlessly distilling black boxes gifted by sketchy powerful interests.

amelius

Governments should be more concerned about the _people's_ personal data instead. Ban data brokers before you ban distillation.