When AI Costs More Than the Engineer

geon

Garbage. You can't include training by the companies that develop an llm in the comparison against companies that merely use the same llm. Apples and potatoes.

spiderfarmer

I'm not a VC guru but in my opinion you can't include the time and money it takes to grow a tree and mine the iron to compare the time it takes to hammer in a nail with a hammer versus using your fist.

mclean

A missing thread by the author for how Anthropic's training expenses becomes expenses for employee workplace expenses. And this is before we start adding Anthropic engineer's ability to use it's tools/models for far less than market price.

pablobaz

The bear case being set at 40% of employee costs is still quite wild.

schnitzelstoat

Ignoring the bizarre inclusion of training compute for the AI company estimates, the other comparisons are still valid. > The rest of the software market trails. The top 1% of companies spend $89k per engineer per year on AI, 40% of a fully-loaded $224k senior engineer salary. The median spends $137. That is the gap : ... 0.4x at the top of the market, near zero at the median. So it's not more expensive than an engineer it's 40% as expensive, and for many companies use-cases the cost is virtually negligible. Even here in Europe where developers are much cheaper than in the US, it still makes sense to pay for the LLM Enterprise subscriptions.

A_Duck

Analogous statement: Evian use 1.25 million litres of water per employee per year. When can we expect other non-bottled-water corporations to rise to this level of water usage?

brown_munda

Mr. Mark Zuckerberg is particularly not happy about these stats. He was promised something else and he has already fired like half of the company. It is really crazy people didn't think this through.

KaiserPro

Excellent, with stunning insight like this, you can see why this VC is earning the big bucks. This is almost economics level of line projection. It would be good to understand _why_ anthropics "AI" bill is so high. First, They are going to be renting a lot of inference compute just to service customers (Meta's Capex bill is about 2x its wage bill) It then also needs a huge amount of infra to both run training and experimentation. THats probably a third of the cost. (storage and physical infra to get the most out of storage and compute is hard. Then getting it reliable, so that shit state doesn't propogate across the shared memory plane is very hard. The other thing to note is that claude usage inside anthropic is tiny compared to the customer's usage. even with uber agents at "mythos++" its going to be at best a few thousand servers. not like the massive fleet needed to serve the paying customer. So using anthropic as some sort of rational target to base any kind of prediction is madness. Its like looking at lyons tea rooms and going yeah, every company is going to spin up an R&D arm to make a company specific computer: https://www.sciencemuseum.org.uk/objects-and-stories/meet-le... ALSO this assumes that the current way of running LLMs is the way forward. Custom software is expensive (in both time and tokens) to look after, its much easier and cheaper to buy it in from SaaS companies and let them figure that shit out. (yes I know SaaS apocalypse, but you are paying for real world experience, and a packaged way of doing things, rather than experimenting your self, where in a lot of cases the company doing the experimentation doesn't know what its doing)

danpalmer

I've not seen anyone yet implement a true cost to productivity assessment or guardrails for AI usage yet. Sure this is hard to do with people, but performance management is a well understood field with a hundred years of practice for knowledge workers. We don't get unlimited hiring budget, so we also won't get unlimited token budgets, and we as the operators will be responsible for the productivity of our agents. What does performance management for engineers look like when dollar token cost is included in reviews? I think it's going to change a lot of assumptions and a lot of strategy around AI use.

altmanaltman

> Anthropic spends 2.3x its payroll on compute.1 With ~5,000 employees & roughly $10b in inference & training spend in 2026, that works out to about $2m of compute per employee per year against a likely all-in comp of $500k+.2 > The rest of the software market trails. This shows how VC firms see things and why we have such a lopsided market where grift rises to top easily. Yes the rest of the software market trails in comparision to the compute costs at Anthropic if you including training the actual models. Like is this the insight? Biggest AI company spends a lot of money to make AI models? Sure you can find anthropic's business model risky/not feseable but using this as your starting point shows a lack of basic understanding at best and malicious intent to make a stupid point at worst

ricardobeat

Open-weight models are going to completely shatter these forecasts. It takes a little more effort – right now, probably won’t be true in three months – but you can achieve the same at 1/10th of the cost.

JVerstry

What about productivity gains?

gnfargbl

Working regularly with AI is like managing a small team of unbelievably knowledgeable, very smart, and occasionally crashingly naïve junior developers. Because they're so knowledgeable and smart, they can get a lot done very quickly. Because they make a proportion of howling errors, you have to keep a close eye on them -- or carefully train another agent to do it for you, in which case you now have to keep a close eye on that agent as well. So, overall, you get more done that without AI, at the cost of spending almost all of your time writing specs and doing code review and almost none of it writing code. Do you get 3.3x the work done? Probably not. Do you get 2x the work done? I think maybe, if you can hack the dynamics of the new job as a manager of eager robots. For me the jury's still out on the second point.

cubefox

This post smells of LLM writing.

cresting

Is not one or the other. AI is a tool for the Engineer. Costs more? Depends on how you use it. You can reduce AI costs in multiple ways, accepting the tradeoffs.

bhouston

I think that token usage by engineers continues to increase, probably at a very high rate for many years (we are in the middle of the S curve of adoption and it isn’t yet clear where this will plateaux) but an increasing percentage of those tokens are cheap, because we use expensive models for goals and design and cheap models for implementations and workflows.

throw1234567891

Garbage. You choose to pay that money, it doesn’t have to cost that much. You have a choice and choose the priciest option, “because shiny”.

HaphazardGuess

A.I suffers from the last-mile problem. It can do 90% of the work in 20 minutes but then the remaining 10% ends up taking 20 million hours to actually finish. It frustrating to the point that I sometimes want to throw the whole thing out and start from scratch.

_pdp_

Even if the current generation of frontier models becomes 10x cheaper, companies will still end up spending much more per employee than they do today. Lower prices will not reduce AI spend. They will simply increase usage. There is no real ceiling on how much companies can delegate to AI. The only limit is the floor where spend too little, and you simply stop being competitive.

mrspacejam

It's amazing how misleading and just flat out wrong this post is.