It's Not Just X. It's Y

Retr0id

> RLVR is weirder, and I suspect it's why we see "It's not X, it's Y" so often. This feels like an easy enough hypothesis to verify, for anyone in the business of training LLMs - does the not-X-but-Y rate increase after RLVR?

huflungdung

You’re absolutely right. This is the smoking gun. This changes everything.

rvz

Another bunch of dead give aways in code bases with READMEs is the repetitive: - "No X, No Y, No Z." pattern - "Here is X - it makes Y" The worst and most obvious one is the constant over use of emoji ticks and crosses.

Baader-Meinhof

I like that these AI idioms exist. They're like watermarks for text. It's worth the cost of humans avoiding them. Companies will eventually train their models to be undetectable, but society would be better if they didn't.

wrs

This is how early forms of "reasoning" in LLMs worked: just literally inserting words like "Wait...", "Hmm...", "Let me reconsider...", "But is it really..." into the token stream.

adt

https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing#...

HarHarVeryFunny

> In the end, shaming people for writing that gets flagged as AI can lead people to sidestep structures the model has learned from us It's interesting why LLMs generate constructions like this more frequently than they presumably exist in the training set. I wonder if this is some sort of mode collapse caused by post training, and/or maybe because they are training on synthetic data so these things become self-perpetuating and self-amplifying (a feedback loop)? The lesson for humans worried about being falsely identified as AI is just learn to write better! It doesn't matter where your repertoire of phrasing comes from (copying AI or not), but one of the basic rules of writing is not to repeat yourself unless you are doing so deliberately for a purpose. Go ahead and use "It's not just X. It's Y" if you want to, but if you use it multiple times in the same short piece of writing, then you may deserve to be called out for poor style, if not for being an AI.

busssard

nice article, but i think as a non native english speaker, i always use the model in english for reasoning and then translate the output to my language. most of these considerations do not apply. because the translation step is taking out alot of these language artifacts

rq1

You’re absolutely right to push back on this. Sometimes it’s not just about the Ys but also the Qs.

coldtea

> Recent overuse by language models has led many to declare it bad writing. I'm not so sure. It is bad writing.

karim79

"So, if we publicly shame people whose text looks like it might have been written by a machine – because it mimics the language used for human reasoning – and people stop writing in ways that they internalize as "AI writing" out of fear of false detection, it sends a signal that your language for reasoning must be policed, or you too could be held up to public scrutiny." This is honestly both terrifying and well articulated. High praise to the blog author.

ai_slop_hater

> Because if Pangram's AI system found me guilty, that's the end of my career. That's literally extortion. How is this different from humans? When I went to high school, my teachers extorted me too. Especially subjects like English and unlike Math, where evaluation is 100% subjective.

amarant

Clearly humans always type "it's not merely X, but also Y"

ilaksh

Surely these leading tells will be trained out of models pretty soon, given how well known and overused they are. And it might make the writing slightly worse in a way. But it is quite annoying how often this type of construction is used in everything at the moment. I think that the current models are still like over-achieving savants rather than true human level because the largest model is only 1/10th the complexity of the human brain. I've recently become fairly convinced that new hardware paradigms (like types of CIM) are about to move from research into real-world development and scaling. So I believe within a few years, the model sizes will increase by another 10 times. Compared to upcoming 100 trillion parameter models, humans will obviously be _much_ dumber/slower than AI in all fields. Already with the 10T models, some LLMs beat 99.9% of humans in competitive programming. The AI hatred from many may actually continue to increase, but in cases where the bottom line matters, we are rapidly approaching the point where writing or work product that looks like it is human-authored will be suspect just on that basis. In other words, for some people it will be the reverse -- "this work looks like it was created by a human" could be devastating for your businesses credibility at that point.

pvillano

In one of the essays posted here, which was, ironically, about AI in education, a sentence, that an AI could not possibly write, that I could possibly write, because of its length and unusual structure, before finally reaching the verb, went on for 25 words. I don't know if it was written that way to show trust in the reader's intelligence, show disregard for reaching a wide audience, show a demonstration of skill, or was artifact of someone just thinking at that level.

tkgally

> There is danger in evaluating for language patterns over its content I agree, but it’s worth noting that that has been done since long before LLMs. Fifteen years ago, I used to teach a graduate course on academic writing pedagogy. The students and I would read research papers on the teaching of academic writing; we also analyzed textbooks and course syllabuses to get an idea about what was actually being done in classrooms. While phrases like “critical thinking” did come up, the overall focus was clearly on language patterns: sentence and paragraph structure, the use of transition words, vocabulary for hedging and boosting (i.e., making assertions seem weaker or stronger), etc. In a university context, it can be very difficult to evaluate student writing based on its content. In humanities-focused and creative writing, what the student decides to say can be seen as an extension of the student’s personality, identity, and individual experience; if a teacher evaluates the content, including the reasoning, it can seem that the teacher is evaluating the student as a person. And if the students are in the sciences, especially at the graduate level, the writing teacher often won’t even understand what the students write because it is too technical. Teaching and evaluating language patterns, not content, is often the only option.

martincsweiss

I liked everything in this post, with one exception. I'm less sure that avoiding speaking like an AI is robbing us of language useful in critical thinking. I'm far more worried about people offloading their critical thinking to AI systems and losing the habit. Also, the Greeks were worried about rhetoric and, in my opinion, rightly so. The skill to argue a point well is different than those that are needed to be correct. To become a skilled rhetoritician was viewed as dangerous (and right now AIs are only moderately good... though they are improving fast).

Papazsazsa

People stopped actually reading when we dropped classical liberal education, right after WWII. This is merely the end-state of industrialization, which is efficient and soulless.

paradox460

It's bigger than that, it's large https://youtu.be/1Pr8xnNi7OM

Der_Einzige

We solved this problem already with the antislop sampler: https://arxiv.org/abs/2510.15061