Why do LLM pelicans look like this?

vatsachak

Thanks for pointing this out. The reason why LLMs cannot be purely* pre-trained on LLM outputs is because LLM output diversity is low. This implies two things: 1) There is going be no functional difference between hypothetical Qwen 4.5 and Fable 5.5 2) LLMs are not AGI

moffkalast

They say the pelican isn't benchmaxxed, but at this point how can it possibly not be?

0xbadcafebee

They're asking why a predictive text model with a fixed set of data, all of which are trained on almost the same open datasets, isn't being more random in its output? People weren't happy with boring answers, so they added randomness to the output. The randomness led to hallucination, so they made it less random. Now you want it to give you a unique artistic impression for every prompt? It's a spell checker, not an art student.

jvanderbot

It is fairly easy to go ask for a differently styled pelican, environment, background, etc. But that's not the point. I was thinking just today about how, if I could hire my competition's lead designer/programmers, have them design a system for me cleanroom, that I'd effectively know everything there is to know about my competition's limitations. This is trivial to do with LLMs. "Make me a thing that does X" is going to come out implemented basically the same way most of the time, as a kind of stack-overflow, well commented, internship project. (or, if undirected enough, as a collection of those based on a bog-standard architecture) Same-y enough that you could train an adversary on it? Or hack it afterward? I am beginning to think so!

user43928

So, how were the more creative pelicans created? I assume by changing the prompt from the default. I don't see why you couldn't do the same in real projects. If the design looks too generic, you can just ask for a change, no?

tamimio

I would be curious to see if you would ask few designers to draw the pelicans with the same prompt to them, no further instructions given, and they should not know about the pelican AI benchmark thing, how that will turn out?!

tetraodonpuffer

is this a case where alignment/rlhf collapses diversity same as image models going from "base" to "turbo" does (where the base model is typically way more influenced by seed, while distilled/turbo models aren't as much)? I am also wondering if part of this is also prompting, maybe we should have a second simonw benchmark that appends "be original" to the prompt and see if all models try to be original the same way too. After all if you ask 100 people to draw a horse, it is quite likely it will be a lot more similar to a "here's a stock horse photograph" than a "here's what picasso would draw when asked to draw a horse"

fenderog

Related, I have been working on evaluating creative output of different models here https://fenderog.com/art-crit/ .

tonyedgecombe

Perhaps all the designers will end up telling us programmers “just learn to draw”.

seidleroni

I think this may be close to what Ethan Mollick has discussed regarding the AI's having a specific "taste". This is how they draw pelicans but obviously if you asked them to make different versions that are more creative, they could do that. For now, this is one way humans can provide a lot of input and direction. https://www.oneusefulthing.org/p/the-overhang

electroweak

Would any teacher of, say, grade 4 students, reading this ask them to draw the same prompt and let us know the result? It would be so interesting to see how we differ from the machines, if at all.

aabajian

More importantly, why is photorealism not implied: https://www.reddit.com/r/GeminiAI/comments/1x54q78/argons_sv... Why does an SVG pelican imply that it should look polygonal? I ran into this issue recently when asking Claude to design a logo for my van. It made a polygonal SVG. I asked it to make it more photorealistic and it looked way better, still an SVG though.

azrazalea_debt

I mean in a lot of cases it seems like the leadership people see this as a win not a lose. Lost count of the number of times a requirement is given in terms of how another product is doing it. "make chips like gmail uses for tags", "let's copy anthropics LLM chat UI", "we should try and make it look like X competitor". "Looking the same as everyone else" seems to be a bonus in their mind

ed

A good prompt will shift the distribution away from average eg: https://john.hartnup.uk/2026/06/07/ai-event-posters.html This is a feature, not a bug.

joshkel

I'm not clear on what the claim is here. The article faults LLMs for not being more creative (riding in the city in the rain, in East Asia, in outer space, etc.). But, if prompted, "Generate an SVG of a pelican riding a bicycle," with no additional details or specification, then I would _expect_ something generic: a side view, nondescript terrain or perhaps terrain associated with a pelican (i.e., beach), little clothing (because birds don't wear clothes) with the possible exception of something strongly associated with bicyclists (a helmet). The article talks about "the LLM diversity collapse problem," which makes it sound like LLMs are getting less diverse over time, but the linked benchmarks suggest that newer, more capable models are _more_ creative. Out of curiosity, I tried a couple of prompt variations in Opus 5.5. "Be creative" did a beach scene but added fish to the bike basket and a lighthouse in the background - not too creative. "Pick a unique and creative variation of this concept" resulted in a "midnight circus act" of a pelican riding a penny-farthing on a tightrope while balancing a fishing pole with fish (and, this time, two lighthouses - the tightrope is implausibly strung between them).

lofaszvanitt

I haven't read the paper yet but I think the keyword here is SVG. Which is a 2D format, usually used for icons. So that already defines how it would look like in the end.

spuz

And funnily even in the diverse set, most of the images show the pelican riding from left-to-right. There's a ton of bias in our data and even in our brains that you would need to erase to get true diversity.

ozgung

> This is the LLM diversity collapse problem no one is talking about We’ve been trying to tell this on every pelican thread. All models were recreating the same composition. The sun, the road, the green, the scarf, going right. Even the style. They’re not obvious associations one would make given the prompt.

orbital-decay

The lack of variety is probably the most underrated issue, and the most likely cause of annoying patterns you all hate. All current post-trained LLMs are mode-collapsed compared to the post-training set distribution, and this doesn't get nearly enough attention from the labs and independent researchers. It's far from being a transformer-only problem, it's a common failure mode in all kinds of models. It's not a fundamental limitation either, unsupervised models have no problem with being as diverse as the training set allows.

dzonga

all A.I will do as it expands the 'work' it touches - is to expose those with taste vs those without taste. tho who know to refine, say NO. & do more with less even in the age of too cheap to meter tokens. that has always been the work that produced every delightful product.