|
Cybersecurity is top of mind for many businesses following the remarkable hack of Hugging Face earlier this month. And rightly so. The problem they now face is that many of the AI firms selling tools to find and patch vulnerabilities are themselves vulnerable to security problems. The latest is Microsoft’s Copilot AI features for Office 365, which CEO Satya Nadella has essentially pitched as being safer to use than competitors like ChatGPT and Claude. But Copilot has its own security issues. Multiple vulnerabilities discovered in Copilot earlier this year could have allowed hackers to trick the AI into giving up customers’ private data, according to research from two separate cybersecurity companies. Microsoft acknowledged the issues, neither of which have been previously reported. The flaws show how AI is quickly posing unprecedented challenges that even the most sophisticated AI developers didn’t foresee. In addition to hysteria over OpenAI’s rogue AI agent that hacked Hugging Face, Anthropic has disclosed multiple embarrassing breaches earlier this year, such as mistakenly giving unauthorized users access to its Mythos model and accidentally exposing its Claude Code source code online. The new research on Copilot comes as Microsoft launches its own homegrown competitor to Anthropic’s Mythos as a new security product. Nadella told analysts Wednesday that the future of security is businesses using such AI-powered “red team agents” that constantly look for and find vulnerabilities. But these AI sellers also need to get their own house in order. In the recently discovered Copilot flaws, the two security firms said hackers could trick Copilot into sharing any file or email from anywhere within the target’s Microsoft 365 software. One of the security firms, Rubrik, told Microsoft about the flaw in April, and Microsoft patched it soon after. The Information is withholding information about a second vulnerability discovered by a different security firm because it isn’t clear whether Microsoft has patched it yet. It’s also not clear whether any customers were impacted by the vulnerabilities. A Microsoft spokesperson said in a statement that Microsoft “works with customers, industry partners and the security research community to further strengthen protections across identity, data, applications and AI.” The security issue Rubrik found could be triggered by malicious code in a Word document that a customer might upload to Copilot for review or analysis. It’s common for hackers to email workers documents that contain malicious code. Once such a document ends up in Copilot, the malicious code could trick the AI into leaking any files from the customer’s Microsoft 365 suite and or cloud servers. Rubrik said the attack would have worked on any of Copilot’s more than 20 million corporate customers. The hack was possible because Copilot has a built-in sandbox environment, a type of walled-off cloud server that isn’t connected to the internet and where the AI can test the code that it writes, Rubrik researcher Ori Lahav said. But a flaw in Copilot’s sandbox would have made it possible for the AI to connect to the internet and send files back to a hacker, according to Lahav. (If that sounds familiar, it may be because the OpenAI rogue model incident involved AI breaching its sandbox to connect to the internet and hack Hugging Face.) Further complicating the issue is that the exploit wouldn’t look like suspicious activity in Copilot’s logs, so customers wouldn’t have had a way of detecting that a hack was happening, according to Rubrik researcher Joe Hladik. A Microsoft spokesperson said that customers can view logs of Copilot’s activity through Microsoft’s Purview software, but did not immediately comment on whether such logs include Copilot’s sandbox activity that Rubrik highlighted in its research. “The problem we face is that AI chatbots tend to be a black box, so it‘s difficult to observe what the AI is doing,” Hladik said. “We’re all trying to solve this problem right now but there's not a standardized approach yet.” There was something for everyone during Microsoft’s June quarter earnings conference call Wednesday, including chief information and technology officers hungry for more data on AI’s impact on workforces. Some highlights: - Headcount in the 12 months ending in June fell 2% year over year while revenue rose 18%. In other words, either AI is helping the company be a bit more efficient or Microsoft was bloated to begin with. We’d guess a combination of the two!
- Microsoft benefited from “stronger-than-expected GitHub Copilot consumption” following its move to charge customers based on usage rather than flat-rate subscriptions. Its competitors have done the same, to be sure, and customers aren’t necessarily happy about it so we’ll see what that does to revenue growth in the coming quarters.
- Customers were increasingly using the Copilot AI features for Office 365, which drove down the company’s gross margins a bit. But the good news was that the company said it had more than 30 million paid subscriptions, up from 20 million a quarter earlier. ChatGPT has many more subscribers than that, but the Microsoft update could allay concerns that AI will destroy its dominant productivity software bundle.
See more analysis of Microsoft’s numbers here. For now, investors are heavily rewarding the company for cost discipline, driving shares up 17% so far today after what has so far been a troubled year for the company. Microsoft said it wouldn’t burn cash in the coming 12 months, despite increased investments related to AI. That’s a contrast to Google, which last week disclosed its first quarterly cash burn as a public company due to AI data center spending.—Amir Efrati
| 
