AI 2040: Plan A

ChrisArchitect

Associated post: Introducing Plan A https://www.astralcodexten.com/p/introducing-plan-a

cyberpunk

> Then, in the mid-2030s, they pause at AIs around the level of top human geniuses. They being the US and China and by agreement. It would be ideal, but there’s far too much money on the table to overcome human nature. So my hope is we hit some kind of limits naturally.. Wishful thinking?

tfirst

If carbon taxes are already a lethal policy for an political campaign, it's absurd to think that fears of ASI will create any real movement around pausing AI. If there is any movement to pause AI development, it will come from the general public's dislike of these companies. Not from the AI safety angle.

sheepscreek

I wonder if they are double-counting Anthropic's leased capacity from SpaceX under SpaceX again.

joshstrange

I'm sure some people will have issue with my phrasing but, honest question: Are there examples of where we have collective decided not to pursue knowledge? Successfully? I guess nuclear weapons might be the best example though research doesn't seem have to actually "stopped" as much as gone underground and we still have country trying to climb that ladder. But I don't know how relevant that is to LLMs/AI. It almost feels like pandora's box is open and our only option is continue to improve them. There is clearly value in what they do and while I can absolutely see the dangers, for example: authoritative governments and surveillance, I'm not convinced to throw the baby out with the bathwater. All of technology back to the printing press (and probably before that) could also be said to make it easier for governments to oppress their citizens. Making laws (and enforcing them!) to prevent governments from doing these things feels like that route forward, not trying to stick our heads in the sand. Perhaps I'm horribly naive, perhaps I just see the SciFi future I've spent my life reading and dreaming about on the horizon and I'm blinded by the reality, perhaps my ideals around "knowledge deserves to be free/accessible" are misguided. I don't know.

ibaikov

People overestimate progress in physical world. 2035: robot population will soon be larger than the human one I'd bet that in most places 9 years is about the time needed to build a residential building. I think a good way to think about this is to think of this as producing a serial car. From pitching and capital acquisition to building a prototype to software, regulatory and then the final product which needs multiple factories and supply chains. Yes, of course robots sound cooler and there are compounding effects yada yada, but on the other side there are as many obstacles as things that accelerate this product (like capital acquisition and fearmongering of gov to bend regulatory stuff faster).

alecco

More wild speculation, now with wishful thinking spread on top.

jabedude

Why did Scott Alexander (one of the authors of the original AI 2027 paper) not join/contribute to this one?

ipnon

Kind of feels like fusion power at this point, always just around the corner.

po1nt

This is whole a slippery slope. Always building on assumption of infinite exponential growth. But every exponential is at a certain point a sigmoid.

kennywinker

It seems to me we’re already at the top of the S curve, not at the toe of an exponential curve. At least with LLMs. Better training data will make small improvements, better architecture will make it less compute intensive, and all these “hyper-scale” data centers will make it cheap and ubiquitous. But none of that is it getting exponentially more intelligent.

a_vanderbilt

I found the AI 2027 paper to be overly optimistic, but not wholly fantastical. This paper feels wildly speculative, and relies on premises I am not confident even pass surface reasoning. Even under optimistic conditions, we are not going to see robots "capable of 95% of all cognitive and physical tasks" by 2035. Nor do I think a 74% unemployment rate is even remotely possible. Economic collapse would implode AI development long before those figures were plausible.

modeless

This is dangerously naive and misguided. They claim to want to avoid centralization of control but propose a world police state of AI regulation. Governments exerting this much control will only end in war and tyranny.

oezi

Forecasting that the GPU build-out will reach 100 trillion USD in 2034 is wild (that's triple the US GDP in 8 years). And another 10x within 2 years. I am not sure where they believe that amount of capital could come from. It would require central bank level money printing never seen before. https://ai-2040.com/supplements/compute-supplement

jawiggins

Did anyone else catch the logical inconsistency between Plan C and A? Plan C: > "... fewer and fewer humans are needed to conduct AI R&D, meaning that covert projects are easier and easier to pull off without detection." Plan A: > "... training AIs requires large numbers of AI chips. Most AI chips are in giant datacenters.50 AI datacenters are typically big enough to be visible from space, and power-hungry enough to require conspicuous infrastructure. New AI chips can only be manufactured at a handful of fabrication plants (fabs), located mostly in Taiwan, South Korea, the US, and China. The US and China negotiate with the countries that have a major role in the chip supply chain, and they require each major datacenter owner (and their upstream suppliers, including chip fabs) to publicly declare their major purchases and sales." Plan A requires properties of AI training that Plan C requires do not exist.

adt

Excellent work by Daniel and the authors. 47,000 words plus supplements is a huge read (and re-read), and an even bigger think-and-write. My early analysis of the analysis: https://lifearchitect.substack.com/p/the-memo-special-editio...

2001zhaozhao

This is by far the most realistic optimistic AI takeoff scenario I've seen, and more specifically it's the first one I've read that deals with both the AI alignment and power concentration issues in a sufficient way, even in a world where hard alignment is assumed (in this scenario the AIs are assumed to be misaligned until ~2038-39). Bravo, and I hope it has the impact on the AI safety field it deserves to have.

PaulHoule

... and in the meantime people are looking at their AI bills and realizing tokens aren't worth what they cost. The frontier is getting the cost down, not getting intelligence up. In a cage match between this guy and "Ed", Ed wins.