Pentagon says overreliance on AI contributed to missile strike on Iran school

allears

Given that everything emanating from this administration has a political agenda, does this mean that Thiel is on the outs?

vfclists

The one that pulls the trigger bears the responsibility.

colingauvin

While I think that it's important to do root cause analysis, we need to be very careful not to absolve blame from the people that decided to shift a significant weight of decision-making authority to an AI. An AI can't be tried in a court. There has to be a responsible human for everyone one of its actions, especially when it's killing (-people-) children. I cannot believe this has come to pass.

feverzsj

Despite what AI told them, they knew it was a school full of kids. They still chose to kill them all with bomb. Only maniac would do such thing.

bix6

DOGE strikes again. Edit actually looks like it was Hegseth himself, king Testosterone. > Staffing on civilian harm mitigation teams across the U.S. Department of Defense had also fallen by roughly 90% over recent years, shrinking to fewer than 20 people overall. Centcom’s group shrank from 10 people to one. https://www.propublica.org/article/hegseth-civilian-harm-mit... Edit: hilarious downvotes. Does DOGE get Elon Stan bots or something?

macintux

Related: the U.S. nearly boarded a Chinese boat that AI incorrectly flagged as carrying nuclear weapons materiel. https://gizmodo.com/almost-started-a-war-us-military-nearly-...

chews

nice, Karp really reeled in a nice big fish, and ofcourse he is going to blame OpenAI/Anthropic for giving the targets, this explains his CNBC rants about the frontiers, his tool was the trigger in a killchain of hallucination.

verdverm

time to make Palantir the next Flock France gave them the boot, you can too

vikas123456789

Whole world is turning into an unlivable place and people who are doing it are not taking responsibility.

sys32768

Tragically this won't be the last time AI kills a lot of innocent people.

igleria

so who is paying for this? "humans must always remain accountable for artificial intelligence (AI) decisions"

1239971

An Maven AI used Claude, which was denied here multiple times. Maybe call the next model after Fable Minab!

legitster

> It found the U.S. “failed in its obligation to do everything feasible to verify” that the school was a military objective and that the failure “went beyond mere negligence.” The report said the United States “directed the strikes at the building of the school while being aware of a substantial risk of striking a civilian object and acting recklessly as regards the possibility that this would happen.” Reading the details, "AI" doesn't really seem like the culprit -it's a scapegoat. The intelligence that it was no longer a military target never entered the target database, the team that was responsible for vetting the target list was gutted, and said team was never even consulted. The White House wanted 1000 targets and pulled from their database without any due diligence. Whether it was an AI call or an SQL query - this was from pure human maliciousness and incompetence.

jmathai

> The Minab site, which was cataloged as an Islamic Revolutionary Guard Corps facility due to outdated data, was fed into Maven with other candidates and came out as a recommended day-one target. Target-list work that once took hours was condensed into minutes. I feel like we're optimizing the wrong metric here.

jacquesm

I'm going to bet that absolutely nobody higher up will end up going to jail for this.

imranq

> Officials said some users expected Maven to flag stale records or contradictions in the intelligence assembled for potential targets, though it is not clear why they thought the system would do that This is what happens when people who don't understand AI and its limitations embrace it wholeheartedly, thinking it's going to behave like The ultimate analyst. Except it's not. Its got blind spots in more places than anyone knows. And not to mention the system itself has multiple areas of failure.

banubis

The pentagon says it was Palantir software’s fault, Palantir says it was the fault of bad data input to their software. Where’s the accountability by both organizations? How did we get to a world where military decisions leading to innocent deaths are responded to as if it was just another miscommunication implementing a B2B SaaS platform.

angoragoats

In a just world, Pete Hegseth and the entire board of Palantir would be in prison for this.

sedan_baklazhan

That is absolutely irrelevant. USA will never take responsibility. Nobody will be prosecuted. No compensations will be paid.

RIMR

I don't want to see the Pentagon try to pass the blame here, but I am all for giving half the blame to Palantir.