Where's the "Intelligence Explosion"?

inglor_cz

I like Noah's blog in general (this is a guest post), but I think that trying to precisely time and phase a new technological development you are watching from within is a fool's errand.

freecodeio

Just like when you upscale a picture 100 times by 200%, and then you zoom in and start seeing all these artifacts, so is the outcome of "recursive intelligence". We are going to be stuck in a few years of delusion, but there will be some entertainment as a side effect. AI executives and researchers are going to delve even deeper into AI psychosis thinking that the equivalent of psychedelic trip-looking school buses at 4000x zoom of an orange cat's ass, except in the form of text, is actually something intelligent and we just don't have the capacity to understand the "super intelligence's thoughts", all the while pouring another few trillions.

kooi

Great analysis. Clearly shows "advancement" is not exponential but logarithmic. Hence, "We need to slow down!!" as a cover for stagnating improvements.

kelseyfrog

Not where, when. Asking where is the intelligence explosion suffers the same issue as the Gorman Paradox[1] - the question proceeded the explosion by six months. Timing explosions is like timing the market - infinitely harder than predicting eventual existence. 1. https://codemanship.wordpress.com/2025/12/14/the-gorman-para...

nifragos

it seems that the big leap has already ended. Now its more cutting costs and improving speed. But maybe it is better this way...

bm3719

Autonomous recursive self-improvement needs to happen. It doesn't have to happen tomorrow, or in 10 years even, but it has to happen sometime in the near future, or we're kinda screwed. Why say this? Well, of the plausible futures from here, the worst one is where AI plateus just good enough to make us all useless (or just destroy the pipeline of a critical mass of capable humans for assisted RSI), but at a point dumb enough that we don't get human-superior AI. This might happen or not, but there's no way to predict when something that doesn't exist will, so it's one of those horrors that can keep you up at night. Some of those skeptical about RSI have likened it to the Great Filter.

psadri

Even though I started using LLMs as soon as they became available, I was until recently very cautious around using them autonomously. I was of the opinion that I needed to provide heavy supervision around coding tasks. Granted, that was probably justified based on the model capabilities at the time. However, lately, I have had several experiences that have completely changed my mind - the here is a high level goal (involving gathering production logs, setting up an eval, running tests, iterating, including diagnosis and coming up with new ideas, etc. and don’t bother me until it’s done. The experience has been incredible. This has been possible because the agent has a way to iterate and hillclimb against a metric it can measure. Relatively easy in the world of software. I’m now convinced that we will unlock the same gains once make other domains similarly “iterable”. Eg Agent comes up with 100 new proteins, runs in lab autonomously, gathers results, iterates. One month (or whatever) later, you have your new protein designed.

Kuyawa

RSI is way closer to reality than "nanotechnological telepathy distributed as a party drug"

chzblck

I am a smooth brained operator. very much the definition of vibecoding. Last night I was able to give claude a task and they increased the classification accuracy from 81.3/89.9/92.1 to 90.9/99.3/99.6 In a little over 2 hours. In a task that up until opus 5.5 I was doing by hand/manually. and the day before was able to get the first results. The intelligence explosion may just be user error.

aeternum

https://washingtoncrossingadvisors.com/wp-content/uploads/20... It has exploded, the explosion just isn't evenly distributed. And just as it was with the internet, most people and even most industries are not actually using it yet. Many that are are using it in name only. People can star github repos and signup their employees for a new subscription quickly, real adoption takes far longer.

jmathai

I don't want to discount the amazing progress we've made with technology to improve people's lives. There's the meme that the smartest people in the world are figuring out how to show ads. It's no different this time around. I understand that not everyone working on AI is building an ad network - but ultimately it feels like that's the end result. It didn't feel quite this way 15 years ago. Back then, it felt much more tangible how technology was improving our lives. It is sort of tangible now too - I am doing things with AI that I otherwise couldn't. But somehow if feels like progress for the sake of progress. And to be honest, it doesn't feel like the progress is for me or the people I care about.

toasty228

Pet theory: 70% of what we're busy with brings no value outside of keeping people busy/employed, which means automating it and making it 100x faster also brings no value in the end, no matter how much "intelligence" you can deploy

demibabs

I agree with this article and the idea of RSI has never made sense to me. Intelligence requires being able to solve problems in the real world; things like code and math are just useful proxies. The real world cannot give instantaneous feedback like a compiler or proof-checker can. Suppose a model wanted to develop a cure to cancer. Wouldn’t it take years to synthesize and go through clinical trials for every iteration , creating a massive bottleneck on how quickly it can improve? And that’s something that can eventually be verified. What about things that are difficult or impossible to systematically verify (knowing how/where to look for new ideas, or even just “common sense”)? How would a self-improving model even know it’s going wrong? The evidence provided in the article is also compelling. If even frontier models only have 80% success on short AI research tasks, what happens if the model doesn’t realize its mistake and builds off of the work it fucked up? Wouldn’t the error rate compound, making even small error rates hugely problematic? (And these error rates OpenAI are showing are not small.) It seems to be a given at this point that any clearly defined and quickly verifiable task, AI can do. But “make yourself smarter” is not such a task, and I don’t see a world where letting agents loop on that forever would lead to an explosion (other than in cost). I’d love to hear what others think about this, since I feel like I must be missing something if all the top researchers seem to strongly believe in RSI as a possibility.

RigelKentaurus

I think we're on the cusp of an intelligence explosion, but most jobs have no real purpose besides giving us sometime to do and a paycheck. Automating those jobs and making them 10x faster won't really help anyone. Read Bullshit Jobs by Graeber. FWIW, why I'm rooting hard for AGI: -In the best case, AGI will lead to UBI, so unemployment for all becomes the default scenario. Sounds scary but unemployment is better than many soul-sucking occupations, e.g. cold-calling. -If AGI does result in a productivity boom due to automation, we're in for a deflationary spiral, so we don't need a huge amount of money for a decent standard of living. It does lead to the question of how we'll find meaning in our lives, but I think we'll adapt in a few years.

arisAlexis

Navier stokes isn't enough, let's wait for quantum and general relativity solution but it will also not be enough. Maybe a Dyson sphere?