Spending on AI is becoming almost impossible for businesses to budget

FinnLobsien

You can set spending limits, but I don't feel like that helps much because everyone's accustomed to AI. Nobody would accept "We're out of usage so we have to wait until Monday" and do all of their work manually. I believe that we're in a scenario where usage is unlikely to go down and neither are frontier AI costs. I believe we'll see a shift to more organizations building their own harnesses with model routing logic to get central control over who can use what AI and for what. A marketer doesn't need to default to Opus 5.5 to upload a blog article with MCP, which could be done by a model 10% of the price.

simianwords

Disagree with this because we have ways to steer price use per task. 1. choose a good model 2. choose the appropriate reasoning effort 3. choose a prompt to nudge it even further Then it comes down to understanding the intuition of what kind of task deserves what effort?

hilariously

I see a lot of businesses who just dump this all on their employees and then get mad at their employees effectively for not following the most recent LLM related talk on twitter. Most employees can't tell you what database to use, what software programming framework to use, what document management framework to use, but they are expected to know which of the 25 models available to use for a task, budget appropriately, monitor efficacy, update models to the most relevant for a task, continue to manage architecture patterns??? for LLM agents, this list goes on. This is getting stupid folks.

neom

https://archive.ph/aBXzS

rglover

Turns out running an unattended LLM like a slot machine is expensive. IMO, human in the loop is the only serious usage of AI (I know, I know, "software factories bro"). Everything else is a hope and a prayer and a big bill.

bentt

I do quite a bit of coding with Claude but am perfectly fine on the $20/mo plan. You people who just let agents go for hours on end... I'm not sure you're doing it right.

sajithdilshan

Why not just set a spending limit per person and extend/adjust the limit case by case. This would actually make people be more mindful about burning tokens on useless stuff

dominotw

most busineeses have no idea how much work is to be done at any point even before ai. Most of the work i've done in my career has been some random shit no one cared about.

sreekanth850

There is a big gap currently in AI assisted development. You don't need to max out tokens to build products or develop with AI. From my experience with AI assisted coding, we still use just two Plus accounts each across a three person team for maintaining multiple repositories totaling around 700K lines of code. Companies should handhold employees, establish clear SOPs, and train them on responsible and effective AI assisted coding.

thadt

Eh, this is a relatively temporary phase. As LLM capabilities have been changing fast, their ability to change existing workflows has been unknown. It’s made sense for businesses to go hog wild with them for a while - just to try to get a handle on what’s possible. At this point, local models have become feasible, and people using them are beginning to get a feel for the tradeoffs vs the frontier models. As the frontier advances, the question becomes “how much will I spend for a given quantity and quality of AI work?” with local hardware providing a pricing anchor point. When I can price hardware and ops for a given capability level - I have a budget again. From there it’s a question of how much faster/capable/cheaper is a given provider (and, you know, how much do I trust sending them all my IP?).

bravetraveler

A budget of zero is remarkably easy to maintain!

01284a7e

Did anyone ever do a software project before AI? You can predict costs far better than you can with people. Yeah, that guy you hired to write the prototype went on a 2 month bender and created 0 usable code. Did you budget for that? Oh, okay. The strategies to deal with spending on tokens are nothing compared to the overhead of managing actual people and their outputs.

tkdb

It feels worse if a robocar kills a human, even if statistically humans kill more humans than robocars. I'm glad humans are freaking out when they realize they don't know if spending $10k/month on AI tokens is good or bad. I'm glad humans are freaking out when an AI pumps out vapid presentations and other humans go ahead and present it to clients. But. Too many businesses have tolerated the same mindlessness when humans were in the place of LLMs. Too many companies telling themselves and investors headcount growth is good without knowing that the new hires are actually doing. Too many human-slop presentations float around, with the authors and audience just going through the motions. Did digital photography raise the bar for what constitutes commercially valuable photos? I think so. I hope AI will similarly raise the bar across all the industries it is touching.

randusername

Cost is the exception to how LLM learning curves have dropped off and in some ways rewarded late adopters. (e.g. prompt engineering is hardly worth the bother when the models now think so hard about user intent) It continues to be an uphill battle to help people at $DAYJOB understand that the relationship between turns and cost is nonlinear. And many still don't understand the idea of a system prompt, that they can control how chatty all responses are.

AndrewShu

Just wanted to put it out there: the FinOps Foundation launched the Tokenomics Foundation last year, to focus on practices and tooling to improve AI cost controls. (FinOps itself is a collaboration between the Finance and Engineering orgs, though IMO more driven by Finance.) And more broadly, AI spend isn't the only problem. Engineers have to track multiple dimensions beyond spend - efficacy, safety, speed, autonomy. All this rolls up into AI adoption and value maximization. High costs might be tolerable, if you get sufficient value out of it.