MIT creates method to force AI to comply with safety rules

ck2

so what happens when the "AI" decides the only way to pass the test is to hack the harness and turn it off?

mixdup

This kind of seems like a no-brainer. Instead of just letting a model have unfettered "physical" ability to do things and hope you can cognitively control it, why not let the AI do whatever it wants, but its access to the tools go through a hard-coded set of rules that is not subject to fuzzy interpretation Of course that depends on having controls that can't be circumvented which is a big if

Mr_P

If you click through to the paper, it has approximately nothing to do with what this HN post title suggests.

petcat

> For constraint satisfaction, what ultimately matters is the model’s final output, since the internal process is discarded. By not requiring every intermediate step to satisfy the constraints, we give the model more freedom to find high-quality solutions that are still feasible in the end. My (maybe naive) question is if we only check the final result then isn't it already too late and possibly the safety rules have already been irreversibly violated? It gives the example of a robot arm avoiding obstacles while still finding the shortest path, but if we only check the correctness at the end , then isn't it possible that it already collided with an obstacle?

sailfast

Would love to see this tested on some of the newer cybersecurity models so we could actually defend ourselves instead of getting cut off at the knees by silly regular expressions. Hope this approach gets well tested and sees good results so we have a shot at human governance.

verdverm

The actual paper: https://arxiv.org/abs/2511.08425v3 > Our key insight is to leverage numerical optimal control to steer the sampling trajectory so that constraints are satisfied precisely at the terminal time. Doesn't seem so "fool proof" to me as where the inevitable media spin will take it. Then, how do you know "where" to steer weights? "Safe" has no agreed upon definition

WalterSobchak

MIT's blog post: https://news.mit.edu/2026/new-method-enables-ai-safety-criti...

arionhardison

I had a swarm break out about 18 months ago; so I stopped and decided I really wanted to dig into it. 1. My agents do not take direct action, they run programs. 2. Programs are not LLM hits/real-time output; they don't MAX tokens the MAX determinism. 3. Programs are logistical wrappers for Protocols where the guardrails are (RLVR.ai) 4. Policies for generating programs are democratically governed re: fec.dev - they get voted on 5. Elected-HITL implements the policy pipelines and ontological abstract intents [and their maps] I would be really interested to learn about the Gov. models that others are using but this seems to be something that linked0-in (which i loathe) discusses (in the most pedestrian/luddite) terms more than HN.

Suhinnall27

The idea of enforcing constraints only on the final output instead of every intermediate step is pretty interesting. I wonder how well this would translate to language models, where “safe” is much harder to define mathematically than a robot avoiding an obstacle.

MattCruikshank

[Cackles in Jeff Goldblum.]

nekusar

WHOSE SAFETY? What are the rules? Or does sharing the rules present security problems, so they're not shared? What are the ethics axioms? And why should I trust your ethical framework?

jcfrei

> In experiments spanning robotics, control of physical processes, and computer vision, the new method consistently satisfied the required constraints while identifying better solutions than existing techniques. I guess this method works when you can precisely quantify the allowed output - like the degrees an arm can move or the path a robot can take, etc. But it doesn't appear to be applicable to an AI writing code - which is where our main concerns currently are.