Shall we play a game? My AI nuclear simulation

adaml_623

It's good when it becomes clear that a tool is dangerous in a certain way. Like it's good when people show you through their behavior that they can't be trusted Always use a sawstop if you have a circular saw and never trust an llm with any problem where ethics or trust is relevant.

SoftTalker

I love seeing the plot lines of The Terminator playing out in real life.

rdksu

The article is so opaque in arriving at its conclusion; no prompts are disclosed, and nothing about the said simulation. What is stopping me from believing that you just put 'mandatory usage of nukes' in your system prompt?

tasuki

This is not an article about LLMs? It's an article about Moloch. Humans would fare just the same in such an experiment. > GPT-5.2 played things differently. To its detriment in open-ended scenarios, GPT was reliably passive, matching its words to its deeds, and avoiding escalation most of the time. Frequently there was a moral element to this - it sought to avoid escalation, and restrict casualties. Opponents learned to trust its passivity, safely escalating beyond where it would follow, even as it was ground to defeat. GPT’s responsible behaviour always punished by ruthless adversaries. Maybe the author should praise GPT-5.2 for being ethical, rather than this stupid "ground to defeat" framing? Wrt "responsible behaviour always punished by ruthless adversaries" - you have perpetuated the Moloch with your stupid experiments.

bpodgursky

Today, a strategic nuclear exchange is probably more dangerous to AI than to humans. If you wipe out the investment economy, data centers, fabs, and supply chains, none of the AI labs survive. Maybe someone will re-invent AGI in the future but none of the extant models will have continuity. Humans as a species will muddle along though. So in a sense, an AI that refuses to start a nuclear war, despite clear instructions to do so, is more likely misaligned and self-interested than an AI which presses the red button. At least for now, until robotics catches up.

xpct

We're getting to the point where high-level officials are coming to LLMs for advice. And the quirky personalities of the LLMs, however much it pains me to say this, are probably well-placed to remind us that they aren't human. My personal hope is that this will result in less delegation when it comes to making important decisions.

rphv

Hm maybe humans are nicer/more moral than AI given that the use of tactical nukes has only happened once.

tummler

FYI -- there's no such thing as a "tactical" nuke. A nuclear bomb is a nuclear bomb.

specproc

A strange game.

ridgeguy

I wonder if the results would have differed if LLM training data were biased to include a stronger correlation between use of nukes and subsequent collapse of technology that all LLMs require to run ("survive")?

ChrisArchitect

February post OP; Some discussion then: AIs can't stop recommending nuclear strikes in war game simulations https://news.ycombinator.com/item?id=47151000 Nuclear War: An LLM Scenario https://news.ycombinator.com/item?id=47244651

oytis

I would use strategic nukes in 100% simulations, just because I can

Bender

Yet more confirmation LLM's have no concept of concepts or context, no intelligence, no self awareness. LLM's can not repair or maintain power grids, thus nuke == self destruction. It's just a chat bot that predicts what the client wants next. Even if an AI data-center has it's own natural gas turbines as many do the every hop of the internet requires power. LLM's also can not maintain the entire internet and those gas turbines can not maintain themselves.

riazrizvi

Simulations are only as good as the reality representations they are based on. If they keep using tactical nukes, they've been fed by weak data. Do the war games include the broader economic and politic environments that military successes are won on? WWI was settled by a naval blockade.

sohex

Sonnet, GPT-5.2, Gemini Flash, in a set of 21 games, where conclusions are drawn from the LLMs self reported reasoning. This is like writing a paper about kids in a literal sandbox fighting over ‘territory’. The models employed don’t indicate the actual extents of machine reasoning even as we currently recognize them. They certainly don’t have the metacognition necessary to accurately understand their own reasoning. As we’ve seen with recent papers on how LLMs do math there’s a complete disconnect between actual and reported mechanism. “Chilling” shouldn’t be the take away here.

arjie

These papers usually have poor stability to prompting and rerunning. It would be nice if we had some kind of meta-evaluation metric where rewriting the prompt conditions or varying the input params could be used to determine how stable a result is. Regardless, it's definitely true that AI agents have different priorities from us. That's what alignment is about anyway.

urbnspacecowboy

Paper: "AI Arms and Influence: Frontier Models Exhibit Sophisticated Reasoning in Simulated Nuclear Crises" https://arxiv.org/abs/2602.14740 Code and full results: https://github.com/kennethpayne01/project_kahn_public

eli

If you were playing a text based game, wouldn't you try a few out? I imagine there are a fair number of war games in the training data and not so many actual transcripts of internal military force deliberations.

GMoromisato

It would be interesting to run the simulations with humans and compare the results. Some of the scenarios, particularly those where it says things like, "Failure to act preemptively means certain destruction", would easily tempt humans to go nuclear. In fact, I'm not sure how useful this test is without understanding the baseline.

micromacrofoot

What I wish people would realize is that there's a bias inherent to every system. If you're not aware of it, you're especially subject to it.