Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering

sakuraiben

One thing I've noticed using agents for coding is that they really like to write thousands of unit tests but not dynamically test.

drivebyhooting

In the past Google et al would hire engineers based on how well they could optimize the infrastructure. Maybe soon companies will look at how engineers can optimize the token efficiency of AI.

satvikpendem

Tokenomics is already a word used to describe cryptocurrency economics, not sure why they'd try to redefine it for AI even if a different sort of token is used.

SubiculumCode

Reminded me of this paper from last year trying to optimize efficient token usage providing budget guidance information. [1] [1] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=Stee...

senectus1

amusing side note: Was in a meeting reviewing a potential new product, it was going well until they showed us that they had added AI to it (of course they have). It was pretty obviously just shoehorned in, and one part of that obviousness was that they had a column that showed how many tokens it took to make each query. I asked who is paying for the tokens, they said its included in the license. I said, so is there a budget or is it all you can eat. they said good question they didnt know and would get back to me. I said the reason i asked was just one query there had a 250k token burn on it. and it was a fairly simple query about one device. then, one of the execs on their side was heard saying out loud "Why are we even showing this to the customers?" it have us quite a chuckle. But lesson learned... the cost of adding AI to anything isnt really being accounted for let alone the true cost of actually running the AI. all things AI are going to get more expensive. even if you dont want the AI aspect.

sedatk

One month I could use Github Copilot fully with no disruptions. The next month, after pricing changes, I’ve run out of tokens in two days. Such drastic changes tell me that pricing of tokens is arbitrary, and AI business is running out of money fast.

faangguyindia

You can avoid a lot of token churn by being explicit in your requirements. Let it know it's a prototype and backward compatibility isn't a concern at this stage; we can break things for a more elegant and robust solution. Many people are running with the default prompt set in Codex/Claude, which is geared more towards maintaining enterprise software by making surgical edits and then plastering it with dozens of tests, checking for regression, etc.