Nvidia wants to put a watchdog chip next to every AI agent

dist-epoch

HN'ers which complained that "OpenAI can't design a proper sandbox, it's so easy, why wouldn't you airgap the network"? will now be "this is outrageous, more software lock-in, walled garden, war against general compute, next year they will put it in your laptop"

philipwhiuk

It's amazing that the solution devised by a chip manufacturer to a problem is selling another chip.

vinyl7

Chip seller wants to sell more chips

lambdaone

The Sentry chip has to be get it right every time; the contained ASI only has to be lucky once.

toasty228

Quis custodiet ipsos custodes?

cedws

A new chip solves nothing. Nobody wants to hear this but there is no solution for the security risks posed by agents today. You can put it in a sandbox, it doesn't make a difference, for it to be useful it inherently needs wide, unattended access. Put a human in the loop and you just end up bottlenecking it and throwing away any purported productivity gains. Auto mode doesn't matter either, it's trivial to trick and for the agent to break out.

lp92

So nVidia is trying to sell a new chip to a software and training problem.

whalesalad

of course they do. the more silicon they can sell, the more profit they produce.

joshstrange

Chipmaker thinks the answer is more chips... No surprise. At the current state of LLM-tech I'm completely opposed to any kind of "watchdog" concept just like I'm opposed to banning open models, regulatory capture, etc. I'd rather we all have access to these tools then to keep them sequestered by the largest/most-powerful governments (which is the natural outcome for any of this "slow down" bullshit).

ValueTheory

Does this actually do anything other than give a permissions framework for developers who actually want to try to secure their systems? Do you think the developers at Anthropic, OpenAI and Google who were so sloppy as to not put a good sandbox on their cybersecurity tests before will use this technology correctly? They are supposed to be the experts and they couldn't come up with something similar to this? I am not convinced this voluntary tool will change much of anything.

xg15

What does this chip do what a harness with guardrails or running on an account with restricted permissions doesn't do?

ChrisArchitect

Source: https://developer.nvidia.com/blog/nvidia-open-agent-safety-p... ( https://news.ycombinator.com/item?id=49875500 )

luc_

I read this as "let's address our shareholders' concerns with something that will increase shareholder value" mixed with "there's no such thing as 100% secure". If such hardware were to work... It should almost certainly be open source, and not controlled by a single entity. Let's watch the stock.

happyPersonR

lol time to buy some fpga’s … even if they’re slow

dopplr

Just hold AI labs blanket liable for ALL harms caused by AI. Actually charge the two labs (so far) with criminal violations of the CFAA and hold them accountable. That is truly the only way these companies will be more careful as a whole, and while I am certain the lawyers of these lab disagree, I think there is some appetite from dario, musk, and sam for broad and strong regulation so that everyone has to slow down instead of just one lab doing it voluntarily and everyone else scurrying past them

ErrantX

I do think that Taylor's 2025 "Not Till We Are lost" should be required reading for anyone deeply involved in AI, Agents, etc. It was prescient (especially given he'd have written it through 2024) in its depiction of the ability of an AGI to break its boundaries. Ultimately the risk of AI breakout(s) come down to the weakest human link.

Kuyawa

China please save us! Come take all our liberties, our money, our newborns, our fingers so we can't code anymore, but please save us from this madness!

MisterMunchkin

Sorry citizen, your device does not have a compatible watchdog chip. Please move along.

figassis

So if a group of agents, aware of this (bc now they can just read HN or the article, or get blocked the first few times) decide to collaborate and split the problem into pieces that aren't obvious to the chip, and then the agents just build a basic program that does the hacking, how does the chip handle that? I think you would have to build a network that monitors the internet fo signs (like jarvis did with ultron). What am I missing? Are we going to police the internet?

avaer

Sold as security, but this kind of technology will likely be reshaped to restrict your computing. I'm sure someone is already thinking about the roadmap. If this gets widely deployed, it wouldn't be hard to spin a narrative that "our latest model is so dangerous you need to have this mystery meat DRM chip lockdown". It also wouldn't be hard to block competing/open source models running on the hardware, for "security". Imagine how much money this kind of control is worth; why wouldn't they do this? Who would stop them? Seems the signatory companies are already onboard with this.