HuggingFace security incident: Guardrails vs. Open Models

benjiro29

The interesting aspect of this attack and the how guardrails prevented them from analyzing the attack. When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers' safety guardrails, which cannot distinguish an incident responder from an attacker. We ran the forensic analysis instead on GLM 5.2, an open-weight model, on our own infrastructure. This had a second benefit: no attacker data, and none of the credentials it referenced, left our environment. While falling back to their local hosted Open Weight model, gave them the ability to analyze the attack, while also keeping their internal structure / data private.