Microsoft Copilot Cowork Exfiltrates Files

2001zhaozhao

AKA, if a malicious skill got into your AI agent, you're cooked. I think this isn't surprising, nor do I think it should be considered a prompt injection at all. An AI skill is akin to a plugin for traditional software - if you install a malicious IDE extension or Outlook plugin, the attacker can also do whatever they want to the PC and exfiltrate whatever data they want to. So this article is a big nothingburger.

hansmayer

Well, isn't that swell - good that meanwhile countless MBA cretins have "adopted" enterprise-wide Copilot integrations, to make their companies "AI native" or whatever the word is on LinkedinLunatics street these days.

bestony

Large-scale adoption will take time; we still need a lot more infrastructure, such as security, auditing, and payment systems.

arjie

A skill is just a program for an LLM agent. This just seems like works-as-expected. Are the five lines in the skill notably innocuous or something? I don't mean to dismiss it out of hand but I don't understand what happened here because it seems to read "`curl $url | bash` can exfiltrate data" which seems pretty straightforward that it can.

Quothling

Nice find. We're PoCing Cowork and I've personally been impressed with it so far, but it seems we'll have to wait with a wider rollout until Microoft give us more admin feature to turn off what users can do with it. > Note: Admins have limited oversight of ‘Skills’, as Skills in Copilot Cowork are automatically loaded from a specific path in a user’s OneDrive. I feel this part is a bit disingenuous. We have full control over the sharepoint containers which house users personal onedrives. We actively scan them and prevent a lot of files from getting in them. That being said, it's still a fair point, because a "skill" could basically be a text file.

pwarner

MS rushed this to production, sure they call it a beta feature but it's clear it was super rushed. They're desperate to be relevant.

Awsum_IceCream

Ah yes, hackers capitalizing on human's laziness. Always ggwp.

mlacks

Exfiltrates: to steal sensitive data from a computer system (for example, via a flash drive). I'm not going to defend Microsoft here, but the title (at the source blog) is misleading and a bit rage-baity. What happened with Cowork may have been rushed, possibly due to incompetence, but incompetence is not malice. This framing is also recycled across a few of the author's other interesting findings. Within the article, the wording is much more accurate: “The victim uploads a skill file to Copilot Cowork that contains a prompt injection,” and “The injection manipulates Microsoft Copilot Cowork into posting a Teams message that exfiltrates pre-authenticated file download links when viewed.”

simonw

If you are building an agent product like this data exfiltration should be the number one risk you are thinking about.

throwaway85825

LLMs do not separate data and code. Caveat emptor.

EFLKumo

It's not the first time we hear about prompt injection attacks, and for sure it's the fault of Microsoft. Many talking about the prompt injection itself, whether Copilot should be able to defense prompt injections, etc. But that's not the problem. OpenAI released their LLM-driven browser Atlas last year. Though their team is brilliant ( https://openai.com/index/hardening-atlas-against-prompt-inje... ), there has been a number of succeeded injection attacks. IMO the real vulnerability is located at the "Act" part of "ReAct" (reasoning and action) agent framework. > “[Copilot] Cowork asks for your permission before taking sensitive actions...” ... when the recipient is the active user, these actions execute immediately without requiring human approval (users do not have a setting to modify this behavior). > Copilot Cowork can retrieve ‘pre-authenticated download links’ for files the user has access to, which allow anyone who opens the link to download that file. > Microsoft Copilot Cowork has read access to essentially any resource a user does through Microsoft Graph. As such, the primary mechanism to reduce the blast radius of attacks like this is to restrict excessive permissioning across one’s Microsoft ecosystem. Take it easy. Inside the whole attack flow, Microsoft gives Cowork unrestricted access and the ability to bypass approvals. I don't find much problem with LLMs here. It's said the attack is also a threat for Opus 4.7, but I've found several times Opus 4.7 forbidding context7.com's "prompt injections" only requiring opus to ask me creating an context7 API key to get more requests for free. From my personal experience, such models indeed are trained to perceive injections, but these injections could mask themselves as sth like Agent Skills, and there are always ways to win as red teams. We may not lay our hope too much on defense of injections, but concentrating on restricting LLM's permissions. The popular usage of CLIs in agents' (especially coding agents) workflow has also concerned me since most cli tools an agent can access actually have the same permissions with users.

MengerSponge

Funny, I thought that Copilot was for entertainment purposes only https://news.ycombinator.com/item?id=47587866

ogundipeore

what’s the recommended for scenarios like this? Add a skill scanner that admins can configure?

ElenaDaibunny

Every new integration is another exfil surface, this was bound to happen.