Why I'm still bearish on LLMs after Navier-Stokes

jaykru

archive link in case i get hugged lol https://archive.ph/Z4gxF

robinpie

I really appreciate seeing a tempered take that's not literally denialist about current capabilities.

ausbah

> the best alternative to rigorous specification is human review. human review doesn't scale well to the volumes of output produced by language models. to make matters worse when the business model is selling more tokens you get such per serve ice times that lead to “more” thinking, engagement baiting, fluffy narratives, and straight up dark patterns

pfdietz

Specifically: bearish on LLMs generally, not bearish on LLMs for pure math.

randomImmigrant

I think bearish on LLMs for automation, and bullish for LLM+human experts in specific fields, is about the right expectation for current architectures. Apart from issues with task generalization, or perhaps related to it, is the fact that LLMs have real trouble with timekeeping, and cannot estimate the real world time it will take them to do things very well. This plus the memory issues make dreams of long horizon agents, that could plausibly handle changing specifications, quite implausible with current architectures. In narrow domains with more deterministic outputs though, this is less of an issue, and we see multiple agents succeed much better. The fusion of that capacity, with humans in the loop able to better direct such agents and act as their temporal tethers, is where I think the real action will be for a while at least.

againstapples

> the models generalize well only on tasks within a small neighborhood of the specific tasks they've been trained on, and even then with severe caveats. the frontier labs have developed a general recipe to teach models almost any specific task enjoying clearly defined levels of task performance; many tasks are covered in the training data Is this really any different to how humans learn, it takes a lot of training on one specific task to make a human expert as well?

carodgers

This April 2026 paper is a fun and related read. https://arxiv.org/html/2509.24239v4 Researchers asked frontier models to play chess. Have a look at the MAR rates in Table 3. When not explicitly told which moves were legal, no model identified legal moves at a rate better than 80%. Many asked for more illegal moves than legal moves. And even when explicitly told which moves were legal, the models continued to ask for illegal moves. With illegal asks discarded, none of the bots could beat a chess model calibrated to 1100 ELO. The author of the originating post says that "current frontier models need laborious oversight and guardrails on even the simplest tasks", and he's absolutely correct.

knuppar

Short and to the point! Open and cheap models will undercut the big labs continuously. The blast radius won't be pretty once spending commitments knock the door.

aogaili

good post/take.

baceituno

doomers gonna doom

war-is-peace

refreshing to see amongst the endless tide of "i haven't written a single piece of code since 2025, llms are so good that they have already replaced everyone" gaslighting

Founderarcstone

I am bullish on AI. At some point well see some true advancements.

vatsachak

I agree with the caveat that it's more like a cracked junior engineer who can manage swarms of interns. Frontier Labs will probably survive off hype valuations but will serve the important purpose of discovering architectures/techniques that will probably spread through rumors/transfers to the rest of the world.

m3kw9

All website should come with a Summerize button.

zzzeek

great, autonomous LLMs will fail. that's actually perfect. they work amazingly well when we're telling them what to do. no autonomy needed, no destruction of humanity. that's all win

keeda

The premise in the very first point seems off: > the frontier labs are priced according to the narrative that they have produced or will in the very near future produce a fully automated drop-in replacement for most knowledge workers... Even assuming this is how the AI companies are being valued (they're not), the numbers are off. The "value" of most knowledge workers -- based on what enterprises currently pay for them -- is $50 - 70 trillion annually. It's reasonable to assume that if AI drop-in-replaced all those knowledge workers, AI companies could credibly charge somewhere in that order of magnitude, because that's what the market is already bearing. So if their hypothetical revenues are double-digit trillions and valuations are some multiple of that, the entire AI industry would be valued at double-digit trillions at the least. Yet cumulatively the industry (the frontier labs + the SWAG estimate of the AI parts of all the other players) are valued at, say, ~6 - 7 trillion? Which seems like a fair approximation of how much knowledge work they can currently automate.

someguynamedq

> current frontier models need laborious oversight and guardrails on even the simplest task As models advance, we shift the goalpost for what "simplest task" means. Before, "simplest task " meant "write a coherent English sentence." Now, "simplest task" means autonomously fix, review, and merge a bugfix.

alain94040

Not convinced by those points. In particular, I found this very misleading or irrelevant: a typical CPU project anecdotally has about three times as many specification and validation engineers as design engineers and a 5:1 ratio is not unheard of The reason silicon design has such verification to design ratio is because the cost of one bug is many, many orders of magnitude higher than software. Both in dollar cost and in schedule cost (it takes months to fab a chip, and if you messed up and need to spin a fix, it costs tens of millions of dollars, not counting any design engineering cost). I don't think you can extrapolate these very industry-specific facts to judging LLMs.

jumploops

LLMs are basically multi-dimensional magic mirrors. Depending on where you point them, they can be incredibly useful. They can even be useful when you point them at each other (though increasingly difficult to get good results). I'm excited for the promise of RSI and a future where models have inherently "live" weights, but it's not clear to me that the transformer is more than a useful tool to help us get there.

yunwal

> those who need done a small set of narrowly defined tasks with existing clear guardrails: repetitive physical labor in a controlled environment, call center and customer service chat work, etc. I have no idea how people can so confidently say that call center work is a “controlled environment” or “repetitive”. It’s almost by definition not repetitive or controlled. Customer support is what I go to when the controlled environment has failed