Our approach to EU text provenance rules

m-hodges

> Starting today, API customers globally will be able to opt in to text watermarking for select models. Text watermarking will remain off by default in the API. > Over the coming weeks, we will add an invisible watermark to eligible ChatGPT and Codex text output in the European Union.

mgax

This is such a waste of time. If someone wants to bypass this it will be rather simple. Just change the words. If someone wants to avoid fingerprinting they will. Can’t we just focus on building rather than spending brainpower on these ridiculous sidequests

smokel

Why use watermarking, and not simply add a signature?

aenis

Another cookie consent-grade success of the EU.

greatgib

My personal opinion is that they cheated evaluations to be able to release this pretending that it has no meaningful impact. Otherwise, I don't see any logical explanation that some of their benchmark results would be higher when watermarked. Except if benchmark results are so unstable that they are an useless metric.

athrowaway3z

>> Editing can weaken the watermark. In an evaluation of 400-token passages, replacing 10% of words with synonyms reduced detection from about 92% to 66%. Replacing 25% of words reduced it to 17%. > Claude/codex/deepseek, please replace 25% of words with synonyms or slight rephrasing because i dont like the current version. Not sure if that counts as: `a solution that's robust against "common alterations and adversarial attacks"`. Is there a sort of adversarial attack that is more common?

k__

Is watermarking part of a model architecture or is it something added by the inference engine?

richwater

Just make the models worse for the EU. Don't accept this nonsense that's holdingg back actual work and progress.

pembrook

Good to know, will exclusively move to Chinese models for non-coding tasks. The idea that producing text with AI needs to be watermarked as if it's a crime by default is backwards nonsense. To me this would actually be a counter signal. If you're NOT primarily writing with AI (at least mildly being informed by all of human knowledge distilled), then I will assume your ideas are emotional opinion-based nonsense, like most comments on hackernews, including my own.

LudwigNagasena

It’s obvious that such boneheaded methods don’t work. So what’s next? First, we need to deal with basic word substitution, which seems theoretically feasible. Then we need to deal with encodings such as Caesar cipher, replacing spaces with zero-width spaces, Base64, etc. Your account will be flagged and reported for outputting obfuscated text. What’s next after that? Ooops, you output too many vim commands instead of outputting text directly, your account is flagged and reported to Europol. You think you can bypass that with Deepseek? No, it will be banned alongside VPN.

GardenLetter27

I wish we could vote out the EU!

Aerroon

Could this technique be theoretically used to track users themselves? It would be quite ironic if the EU forced tech companies to implement extra tracking, wouldn't it?

socketcluster

This approach feels wrong. For code, it's obvious now that Claude is adding watermarks through comments because they are way too long. I keep asking Claude to remove and reduce its comments. Also, I don't think it's useful because AI involvement in the work is a spectrum, not a binary true or false. If I ask Claude to go over my code and fix up typos and clean up and it starts adding comments with hidden watermarks everywhere it makes the tiniest adjustment, it's essentially appropriating my code. Its contribution may be trivial and superficial. When I use AI on my own projects, the code it generates basically looks like auto-complete to me; exactly what I would have written by hand. So I don't see why it needs to be marked as AI. It's just saving me time. It's very much my own work.

andriamanitra

1% false positive rate is completely unacceptable, and if you can bypass it by changing some of the words what's even the point? This is only going to catch low effort slop.

k1m

Worth noting that this is what OpenAI wrote about text watermarking two years ago: > While it has been highly accurate and even effective against localized tampering, such as paraphrasing, it is less robust against globalized tampering; like using translation systems, rewording with another generative model, or asking the model to insert a special character in between every word and then deleting that character - making it trivial to circumvention by bad actors. > Another important risk we are weighing is that our research suggests the text watermarking method has the potential to disproportionately impact some groups. For example, it could stigmatize use of AI as a useful writing tool for non-native English speakers. https://openai.com/index/understanding-the-source-of-what-we...

pveierland

Mandating and accepting broad use of text watermarking at scale seems prime to enable all sorts of dumb and chaotic downstream effects. Not only do you start to pollute further corpus at scale - hijack text space bandwidth and fidelity - but you open the societal door to gradually add additional pieces of information - such as author identities or ad tracking information - which can then not just directly reveal a single author, but potentially larger graphs of information propagation without it being clear to anyone propagating such a trace.