Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing

gigatexal

I find it useful that if they cut the use altogether I will pay for it out of pocket.

illithid0

>"He said that, based on talks with Uber's senior engineering leaders, he realized higher token usage did not translate into a proportional increase in useful consumer features." Goodhart's law strikes again at someone with enough power to be both ignorant of it and make others suffer their ignorance. You cannot simply measure productivity by tokens spent just like you can't measure it by hours spent in a chair at a desk.

nekzn

It’s funny that “maxxing” entered the common vocabulary.

7777777phil

As soon as tokens stop stop being subsidized, heavy agentic use will become as least as expensive than paying an (entry level) employee. When this happens many companies will trade off havy tolen usage for (maybe a bit slower, bit less accurate) employees again.

egypturnash

Uber COO says he just decided to short a bunch of AI company stock.

chihuahua

It's amazing that it took months to figure this out. "Well we thought that if engineers are told to maximize costs through AI use, to consume as much as possible of a resource that costs us money, then obviously good things will happen. Imagine my surprise when it didn't turn out that way." Imagine if engineers were ranked based on their AWS spend. People allocate VMs and fill databases with terabytes of random bits, to get to the top of the AWS leaderboard. If you don't do this, you're ranked at the bottom, and good luck at the next review cycle. Who could have expected that this is not the road to success?

cryo32

Waiting for tokenedging next.

Rohunyyy

Now we are going to get a new profession. Token Engineer! They will be experts on tokenmaxxing! The job growth that the billionaire CEOs promised us from AI is finally here!

izanton

What if... we stop for a moment, and then, after thinking for a moment, we stop hammering nails with a microscope, and stop using token usage as a metric of productivity? I know it's sounds stupid, but what if

FartyMcFarter

If any company announces that they use token consumption as an employee performance signal, for me that's close to a red flag to stay away from that company. No company with good engineering leadership should act like this is remotely a good idea.

jhack

Maybe don't use the most expensive models on the planet? Maybe use AI like a tool and not this black box that grants wishes?

irishcoffee

I just realized my company is months behind this curve. About to blow my token allocation. Before I do, anyone have requests? Sincerely.

pocksuppet

what the fuck is this timeline I am stuck living in

crorella

Tokenmaxxing makes no sense, it is akin to write extremely inefficient SQL / Spark Jobs, full of cartesian joins, ultra skewed datasets, etc, just for the sake of using as much compute / memory / IO as possible. This always happens when the metric becomes the goal, companies should nurture and foster an environment where AI is used in the most efficient way possible, first asking "do we really need an agent for this" and if so, what kind of agent is needed, what model, reasoning level, etc. They should also promote projects that aim at saving tokens, increasing cache hits, codifying the information in ways such they use as less context as possible (graphs of knowledge are pretty good for this!)

simonw

I'd be interested to know if this is about individual employee AI usage, or use of AI tokens in production features, or both - and assuming both, what the split is. I can see how Uber could burn unbelievable amounts of tokens if they start running internal features that run a bunch of prompts against every completed ride, or every customer profile, for example. Or maybe this is about employee usage, but they introduced some stupid "you get evaluated on how many tokens you used" thing a couple of months ago when that was trendy and are just beginning to notice how much that cost?

lorecore

Not all tokens are created equal. It's easy to use a ton of tokens by having agents work together in parallel. That's basically the equivalent as people spending time in meetings, hardly a productivity win. As with everything in development, results matter, how you get there doesn't (unless you're a bad manager).

JackDanMeier

At what point is there a difference between a burn rate and tokenmaxxing? Isn't it the same as during the dotcom bubble?

paulpauper

many of these leading AI companies are operating at large losses and subsidizing users with VC money. Profitability will entail having to impose greater limits and raising prices, so this will reduce to some degree the value proposition of AI compared to humans.

rcvassallo83

Oof leader of bubble are starting to take a step back?

mrkeen

I always used to wonder this about software stacks even prior to LLMs, but it seems more relevant now somehow: When will Uber (or your favourite company) be 'done'? They've been writing software for 16 years. They match drivers to passengers. More software isn't going to increase the chance that I seek them out instead of taking a bus or train. Will their software be finished in 20 years? 80?