AI tools are increasingly becoming the backbone of modern workplaces. Large companies are now relying on internal AI systems to search documents, analyse data, and assist employees in their daily work. But a recent security experiment suggests that the same technology could also introduce a new type of cyber risk, one where AI systems themselves become the attackers.
Researchers from security startup CodeWall claim that one of their autonomous AI agents managed to break into McKinsey’s internal AI platform in just two hours. The test was carried out as part of a responsible security exercise, but the findings are drawing attention to how quickly automated AI-driven attacks could unfold if similar techniques are used by malicious actors.
AI agent hacked McKinsey’s internal AI system
The system targeted in the test was McKinsey’s internal generative AI platform called Lilli. The consulting firm introduced the tool in July 2023 to help employees search company knowledge, analyse documents, and access internal research more easily.
The platform is widely used inside the organisation. According to McKinsey, more than 70 per cent of its workforce, over 40,000 employees, regularly interact with the chatbot. The system processes more than 500,000 prompts every month as consultants use it to assist with client work and internal analysis.
CodeWall said the decision to target McKinsey was suggested by its own research agent after it scanned publicly available information, including the company’s disclosure policy and recent updates related to Lilli. The researchers then allowed the agent to operate autonomously without giving it login credentials or insider knowledge of the system.
The AI began by mapping the platform’s attack surface and analysing publicly available documentation. During this process, it discovered API documentation that exposed more than 200 system endpoints. While most required authentication, 22 endpoints were accessible without login credentials.
One of these endpoints handled user search queries and stored them in the database. The AI agent discovered that the structure of this request could be manipulated, allowing it to inject malicious instructions into the system’s database query.
By repeatedly testing the system and analysing error responses returned by the server, the agent eventually managed to extract live production data from the platform.
Millions of chats and files reportedly accessible
According to CodeWall’s report, the vulnerability allowed the AI agent to access a massive amount of internal information linked to the Lilli platform. Researchers said the database contained around 46.5 million chat messages generated by employees using the tool. These conversations reportedly included discussions related to strategy, financial planning, mergers and acquisitions, client work and internal research. The messages were stored in plain text and could be accessed through the vulnerability.
The system also contained around 728,000 files, including PDFs, spreadsheets, presentations and Word documents. The agent was able to view more than 57,000 user accounts connected to the platform, as well as thousands of internal workspaces and AI assistants used by employees.
Paul Price, CEO of CodeWall, said the attack sequence was handled entirely by the AI system itself. “The process was fully autonomous from researching the target, analysing, attacking, and reporting,” he told The Register.
Beyond accessing data, the researchers said the flaw could have allowed an attacker to modify the instructions that control how the AI system behaves. Lilli’s system prompts, the rules that guide how the chatbot responds to queries and enforces safeguards, were stored in the same database. Because the vulnerability allowed both read and write access, an attacker could theoretically alter those prompts without triggering obvious alarms. Such a change could influence how the AI answers employee queries, potentially embedding misleading guidance or bypassing built-in guardrails.
The researchers disclosed the vulnerability to McKinsey at the beginning of March. According to the report, the consulting firm quickly addressed the issue by patching the affected endpoints, blocking public API documentation and taking parts of the development environment offline.
A McKinsey spokesperson told The Register that the company investigated the matter after receiving the report. “Our investigation, supported by a leading third-party forensics firm, identified no evidence that client data or client confidential information were accessed by this researcher or any other unauthorised third party.”
The spokesperson added that protecting sensitive information remains a top priority. “McKinsey’s cybersecurity systems are robust, and we have no higher priority than the protection of client data and information we have been entrusted with.”
