Fiber lasts decades, GPUs last three years: the AI boom isn't the dot-com boom [video]

smashspectacle

Ed Zitron is going to become the Simpsons of ai critiques...

jdietrich

Siri, what's the spot price for an A100?

orwin

I don't want to argue here, but once models are trained, using them is quite cheap. Also in the last few months most improvements I've seen are in the smaller sized models, which might make using models even cheaper.

deadlast2

If you use opencode for your AI work or becomes affordable. To the point where you would reasonably build applications with intelligence the LLM version. I think in the near future the intelligence will be free and there is no business in free. I know it is a big statement but if you look around there is already a large amount of free models of gradually increasing strength. For most applications you don't need a gpt6

mono442

Another thing to pay attention to is the margins of hardware companies like nvidia or micron. Hyperscalers are spending huge amount of money but the money mostly goes to the profits of hardware companies.

DennisP

Bad for hyperscalers. Good for Nvidia.

ndom91

Shameless self promotion: I put together an AI infra built-out financials data-vis project/page recently. It includes graphs and tracks updates to stuff like GPU depreciation schedules like this. https://artificialliabilities.com

saidinesh5

GPUs might be 3 years but the rest of the hardware: processor cores, ram, storage can power other cloud applications for a decade or more no?

whynotminot

I recommend looking at any data regarding A100 rents (6 year old GPU) and then save 50 minutes of your time.

Good4boothee

There are multiple YT channels proclaiming Russia is finally collapsing/losing war with Ukraine, huge "It is happening" in thumbnail; few of them put out a variation of such video every week, not even bothering to excuse their previous, wrong prediction. And any reporting related to AI bubble is exactly as terrible. I might as well just check Bloomberg/FT/QZ, news there will be devoid of technical details and still more useful than those endless armchair-experts' speculations.

anu7df

GPUs may last 6 yrs or more. Fiber vs GPUs is however not the right comparison. Fiber vs Open models are. Open models have mostly caught up with frontier models. Now, with a reasonable harness and tools they can easily be useful for vast majority of users for a decade or more.

cmiles8

There is a use for older GPUs, but the chips do have reliability issues. Turns out running chips hard 24/7 causes them to burn out. So it’s less will they be “useful” a while from now but more will they still work. Folks are leaning it’s like these LED light bulbs that say they’ll last 15 years but after 2 years they start flaking out at a high rate. Separately, the broader point is still directionally correct. We’re still using fiber from the .com network infra implosion today and will for a long time. The deprecation cycle is incredibly long. The increasing reports of GPUs sitting on shelves because there aren’t data center to power them is very concerning considering all the high interest debt being used to buy what is ultimately a rapidly depreciating asset. Financially, that’s a disaster waiting to happen.

paxy

- GPUs last a lot longer than 3 years. - The factories don't stop working after 3 years. They can keep making GPUs. - The AI boom isn't just about GPUs. There have been massive investments in CPUs, RAM, networking and routing components, physical data centers, power plants, cooling, and even fiber (the thing you yourself claim lasts decades). None of this is going to disappear in a couple years.

pcthrowaway

One thing about the GPUs that I think is unmentioned, is that the LLM models are also becoming more efficient, right? So you can get more tokens or more "work" out of the same GPU as the technology improves