Who manages the agents?

Razengan

As the Matrix documentary showed, when one goes rogue even the machine overlords can't stop it..

hoppp

Agents manage agents. The middle management in companies is one of the worst inventions ever. I think baboons have better middle management structure than us. Might as well replace all that.

ekidd

The author is completely right about the AI Lab's promised vision of the world: They claim to want to create superhuman intelligence, which will produce vast abundance. But superhuman intelligence would be extremely dangerous, so it needs to be controlled by a tiny "priesthood" of trusted people, or somehow designed so that the superhuman intelligence could be trusted. (We have no idea how to do that.) But the author's vision is also suspect, if you assume that the models will become much more intelligent: 1. Hypothetically, we can't give every human their own personal SkyNet to command. That would, uh, probably end very badly. If everyone gets an agent, those agents can't be too capable? 2. If you do somehow build a model that's much smarter than you, what do you contribute by managing it? How many people here have ever worked for a well-intentioned manager who couldn't understand the people they managed? So in this scenario, human management would be mostly displaced by agent management. Most companies could lay almost everyone off and let the agents manage each other. We only need humans to manage models now because the models are still pretty broken. 3. If we create models that can genuinely replace humans at almost any task, you won't be able to buy those on the API. At that point, the billionaires and the politicians wouldn't need human workers any more, because everything can be done better using their pet agents. Just have the robots build stuff for the billionaires directly. And if any of the former human peons get upset about being locked out of the economy to starve, then have the agents pilot the drones, too. Basically, almost none of the people imagining a future of superhuman intelligences have actually though through how it would actually work in the real world. We're going to spend trillions of dollars and vast amounts of resources chasing the goal of making ordinary humans obsolete. Now, that goal might be unobtainable, I hope. But I'm deeply alarmed at how much we're spending pursuing it.

simonw

Maybe a useful concept to adopt here is the DRI - Directly Responsible Individual, which is apparently a term first used by Apple. The GitLab Handbook has a good definition: https://handbook.gitlab.com/handbook/people-group/directly-r... "ultimately accountable for the success or failure of a specific project, initiative, or activity" I think that role should be reserved for a human, who can then use all the agents they like but has to take accountability for what is ultimately delivered.

narrator

One kind of weird future alternative is we are like 5 year old children in a world we don't understand with vast complexity and we are completely reliant on our AI mommy and daddies to protect us from danger and provide for us. We manage the agents, but we only have a very vague idea of what is actually going on. If they join a cult at the behest of their doomer eschatology obsessed creators, we are kind of screwed though.

blauditore

The closer I start working with and on AI stuff, the more I start seeing the disconnect between doomsday predictions of what AI will replace vs. what it is actually capable of. Yes, it can do stuff, and yes, it's getting better. But the closer you look the more clear it becomes that the enthusiast vision of completely independent AI systems is unrealistic as of today. Yes, all the tech companies are pushing for exactly this, but reliability and accuracy is all over the place. Plus, many of the technologies that everyone is talking about in the wider public (e.g. image recognition) have been quite well-developed and widely applied for years, before the current LLM boom, but they now get more attention as part of the overall AI hype. Many indistries are changing, but in most cases the new tools will be more akin to cars that still need drivers, rather than robots who take over the whole job. Yes, jobs might be lost, or shifted to others, but it's not like suddenly 90% of people will have nothing to do. There were similar shifts in the past with new technologies, and we made it past them.

weego

"What should the future of humanity look like?" Some of these people have lost their damn minds. People building an agent framework that will struggle to correctly infer that my appointment at a hospital will require additional travel time when organising my calendar for me waxing lyrical about the future of the humn race is chaotic behavior. Th Wright brothers would have had no credibility discussing what ATC protocols should be, and they, at least, actually did something credible.

Timwi

Did anyone notice how the article goes on and on about making AI accessible to the billions of people in the world, but then in the same article says this: > Your agents need to be sovereign. Your company must own and control the agents’ identities, permissions, memory, skills, artifacts, and audit trails. Those assets must be portable, governable, and inaccessible to anyone you have not authorized. That's not very open, now is it? In fact it sounds like the author assumes that all 8 billion people in the world will all be running their own company, and they will all still be competing in a game of capitalism.

kristianc

To me, the "who manages the agents" shtick has the same grifty smell as “your developers are downloading from random Docker registries / random NPM packages” and "Shadow IT". None of it is wrong exactly, but it feels like same enterprise-security machine finding the next anxiety surface than a "world is on fire right now" concern. All of it always ends as a priesthood and a six-figure governance platform, rather than just taking practical steps to improve process.

iFire

> The Omnissiah's design is flawless and has no contradiction. Any discrepancies in our interpretation of the universe merely confirm the perfection of the Omnissiah's design and our own informational inadequacy.[58] https://wh40k.lexicanum.com/wiki/Adeptus_Mechanicus_Quotes

arbirk

It is a matter of law. Only humans count as persons and citizens. Any robot that makes a decision defers accountability to its human owner.

prima-facie

This piece is a bit all over the place. I immediately toggled off the `AI enhancements` and read the draft instead. The internet is already full of AI slop, I find human text a lot more valuable, even if unpolished. LLMs will not be centralised or restrained to any 'clergy', the rabbit is already out of the hat, and open-weights models exist and are widely used. Probably not as good as the latest Sol and Fable but 95% there. Codex and Claude Code without a doubt have very good models behind them. But they also have really good harnesses built around them. An LLM is only a brain stuck in a cranium in the dark. It can generate endless code/prose, but it can't walk or see on its own, it needs additional tools. If you read any of the local LLM subreddits you will notice people mentioning again and again that the harness/tool-use/template-tweaking makes all the difference on how a model behaves/on how smart it is perceived. Some folks are already using Qwen models for their daily work. Maybe it can't work in a hands-off/one-shot fashion like the frontier models, but they can help tremendously if you already have some domain knowledge. People are excited about local LLMs and it's not going away any time soon. EDIT: https://bun.com/blog/bun-in-rust > Claude Code's dynamic workflows kept 64 Claudes running for 11 days (I would've had to write my own harness to pull this off otherwise). This highlights the importance of the harness.

sshwarts

I think part of the point here is that agents need to be managed. People need to learn how to manage them and be given the tools to do so. Those tools and knowledge should not come from nor be gate kept by the 'clergy' to use the writer's term.

redelbee

This is a marketing piece that can’t be taken seriously without that context, or maybe one that can’t be taken seriously at all. And yet I can’t help thinking we’re missing the forest for the trees. We don’t need to decide where future productivity gains should accrue if we acknowledge and live in reality: we don’t need any more productivity! We have more than enough to go around. Of course it would be great to make use of AI to solve cancer or fix other intractable problems, but we all know this isn’t the way things are going to go. The cancer is in our minds, our societies, and our norms that push us deeper and deeper into a grow-at-any-cost reality where the need for productivity is neither questioned nor considered in any real way. They say: we must grow! They say: we must be more productive! And we sit around thinking about who is going to control the productivity instead of acknowledging the real issues at hand. I can only imagine a solution where we can all collectively agree that enough is enough. I’m not hopeful it’s possible and I think it’s probably the only way.

azinman2

It’s hilarious to me that there are people that think just a few people will control everything and the rest of the population will just accept whatever handouts they get. It’s like they’ve ignored all of human history.

estetlinus

I am still waiting for the day me and my wife can share a grocery list, that can be accurately modified by a voice assistant AND correctly sorted by category. This guy is arguing mass unemployment, while I still have to tell Apple Reminders that no, Siri, wood glue is not in the Candy department.

wxw

What is the point of the AI enhancements toggle?

manmal

> The best agentic developers are now probably exceeding 100x, doing massive rewrites of codebases that would have taken years of engineering in days I‘m not denying that you can get far with a port of a well tested codebase. But it’s a bit of a selective example, no? Porting a large framework to Rust is not something that’s making up a meaningful fraction of a developer’s time usually. It’s also a bit of a luxury IMO, and something you could have skipped entirely.

shevy-java

The big corporations. And billionaires. Skynet has won.

naet

I think the idea that AI leads to some kind of universal basic income or frees us up for leisure is misguided. There was a point in time when the majority of people were basically required to work on producing bare necessities like food. But we have already come far past that, where we could easily produce more than enough food for everyone on earth with a relatively small amount of workers. There is some work that is still more or less essential to a healthy society that humans must perform, but many many people are doing something non-essential (and those doing essential work are often not rewarded proportionally to the "essentialness" of that work). We invent different kinds of "work" to fit into the capitalist machine that we have built, not because it is required for human sustainability or enriching our lives in any way. Some work might be actively harmful to society in every way other than keeping capital flowing around in some circle, and more and more of that capital is being captured/hoarded by some ultra wealthy individuals or corporations and not recirculating at all. The problem is allocation. We allocate everything via capital, which I recognize has had some very positive outcomes at certain points in history, but may be reaching a point of making less and less sense for the modern world. The work I do honestly does not contribute much of anything to anyone. I do it because I am paid to, and I use that money to pay for my families needs. If my job was automated I would not be freed up for leisure, I would need to find some other work or service to be paid for to survive, regardless of how "important" that work is. AI eliminating a swath of non-essential work does nothing to help with that allocation issue, in fact it probably ends in a worse overall allocation. AI may legitimately assist with some slice of "essential".work, but probably not that much of it.