How we hacked McKinsey's AI platform

gbourne1

- "The agent mapped the attack surface and found the API documentation publicly exposed — over 200 endpoints, fully documented. Most required authentication. Twenty-two didn't." Well, there you go.

sgt101

Why was there a public endpoint? Surely this should all have been behind the firewall and accessible only from a corporate device associated mac address?

sd9

Cool but impossible to read with all the LLM-isms

lenerdenator

Not exactly clear from the link: were they doing red team work for McKinsey or is this just "we found a company we thought wouldn't get us arrested and ran an AI vuln detector over their stuff"? You'd think that the world's "most prestigious consulting firm" would have already had someone doing this sort of work for them.

fhd2

> This was McKinsey & Company — a firm with world-class technology teams [...] Not exactly the word on the street in my experience. Is McKinsey more respected for software than I thought? Otherwise I'm curious why TFA didn't just politely leave this bit out.

cmiles8

I can only remember a McKinsey team pushing Watson on us hard ages ago. Was a total train wreck. They’ve long been all hype no substance on AI and looks like not much has changed. They might be good at other things but would run for the hills if McKinsey folks want to talk AI.

captain_coffee

Music to my ears! Couldn't happen to a better company!

joenot443

> One of those unprotected endpoints wrote user search queries to the database. The values were safely parameterised, but the JSON keys — the field names — were concatenated directly into SQL. I was expecting prompt injection, but in this case it was just good ol' fashioned SQL injection, possible only due to the naivety of the LLM which wrote McKinsey's AI platform.

paxys

> named after the first professional woman hired by the firm in 1945 Going out of their way to find a woman's name for an AI assistant and bragging about it is not as empowering as the creators probably thought in their heads.

bee_rider

I don’t love the title here. Maybe this is a “me” problem, but when I see “AI agent does X,” the idea that it might be one of those molt-y agents with obfuscated ownership pops into my head. In this case, a group of pentesters used an AI agent to select McKinsey and then used the AI agent to do the pentesting. While it is conventional to attribute actions to inanimate objects (car hits pedestrians), IMO we should be more explicit these days, now that unfortunately some folks attribute agency to these agentic systems.

sigmar

I've got no idea who codewall is. Is there acknowledgment from McKinsey that they actually patched the issue referenced? I don't see any reference to "codewall ai" in any news article before yesterday and there's no names on the site. https://www.google.com/search?q=codewall+ai

victor106

this reads like it was written by an LLM

ecshafer

If the AI was poisoned to alter advice, then maybe McKinsey advice would actually be a net good.

mnmnmn

McKinsey can eat shit

frankfrank13

Some insider knowledge: Lilli was, at least a year ago, internal only. VPN access, SSO, all the bells and whistles, required. Not sure when that changed. McKinsey requires hiring an external pen-testing company to launch even to a small group of coworkers. I can forgive this kind of mistake on the part of the Lilli devs. A lot of things have to fail for an "agentic" security company to even find a public endpoint, much less start exploiting it. That being said, the mistakes in here are brutal. Seems like close to 0 authz. Based on very outdated knowledge, my guess is a Sr. Partner pulled some strings to get Lilli to be publicly available. By that time, much/most/all of the original Lilli team had "rolled off" (gone to client projects) as McKinsey HEAVILY punishes working on internal projects. So Lilli likely was staffed by people who couldn't get staffed elsewhere, didn't know the code, and didn't care. Internal work, for better or worse, is basically a half day. This is a failure of McKinsey's culture around technology.

jacquesm

And: AI agent writes blog post.

palmotea

With all we've been learning from stuff like the Epstein emails, it would have been nice if someone had leaked this data: > 46.5 million chat messages. From a workforce that uses this tool to discuss strategy, client engagements, financials, M&A activity, and internal research. Every conversation, stored in plaintext, accessible without authentication. > 728,000 files. 192,000 PDFs. 93,000 Excel spreadsheets. 93,000 PowerPoint decks. 58,000 Word documents. The filenames alone were sensitive and a direct download URL for anyone who knew where to look. I'm sure lots of very informative journalism could have been done about how corporate power actually works behind the scenes.

VadimPR

I wonder how these offensive AI agents are being built? I am guessing with off the shelf open LLMs, finetuned to remove safety training, with the agentic loop thrown in. Does anyone know for sure?

cs702

... in two hours: > No credentials. No insider knowledge. And no human-in-the-loop. Just a domain name and a dream. ... Within 2 hours, the agent had full read and write access to the entire production database. Having seen firsthand how insecure some enterprise systems are, I'm not exactly surprised. Decision makers at the top are focused first and foremost on corporate and personal exposure to liability, also known as CYA in corporate-speak. The nitty-gritty details of security are always left to people far down the corporate chain who are supposed to know what they're doing.

drc500free

I have grown to despise this AI-generated writing style.