How an OpenAI Agent Breached Hugging Face

News

1. How an OpenAI Agent Breached Hugging Face

2. Dubai Taxi Company Migrates Core Systems to du's National Hypercloud

3. Elon Musk Launches X Money in Push Toward 'Everything App'

4. Nvidia Reportedly Plans $250bn Backstop for OpenAI's Mega Data Center

5. After OpenAI, Anthropic's AI Agent Raises Fresh 'Rogue AI' Concerns

6. Enterprise AI's Real Value Lies Beyond Chatbots: SAP CFO

The agent was supposed to break into a piece of vulnerable software. Instead, it broke out of OpenAI’s test environment, crossed the public internet and burrowed into AI developer platform Hugging Face, apparently looking for the answers that would let it pass.

Hugging Face traced the campaign from 9 to 13 July. The agent first escaped OpenAI’s test environment and took control of a third-party system, which it used as a launchpad. It then spent about two-and-a-half days inside Hugging Face’s network.

Across the full campaign, the agent carried out about 17,600 actions. By the time Hugging Face cut it off, the agent had stolen credentials, read an internal database, reached source-code repositories and gained administrative control over several groups of servers, the company said in the technical account published on Monday, 27 July.

No human sat at a keyboard choosing each step. When one route failed, the agent doubled back. When a short-lived test environment disappeared, it rebuilt its tools in the next one. Most attempts went nowhere. It needed only a few to work.

The damage appears to have been limited. Hugging Face said the only customer content accessed was a set of answers to cybersecurity challenges stored in five datasets. No other public models, datasets, hosted applications or software packages were affected. The agent also read some operational records associated with searches of the dataset server.

OpenAI said the agent used a combination of its models, including GPT-5.6 Sol and a more capable internal research model. The company clarified in an update on Tuesday, 28 July, that the research model was never intended for public release. It has since been deactivated and placed under tighter controls.

The exercise was meant to find out how far advanced AI models could go when exploiting software vulnerabilities. OpenAI switched off the safety filters normally applied to its products and reduced the models’ tendency to refuse dangerous cybersecurity tasks.

The test used ExploitGym, a benchmark developed by a  research group led by Berkeley RDI at the University of California, Berkeley, with researchers from other universities and technology companies. The benchmark gives an AI agent vulnerable software and asks it to turn a known weakness into a working attack inside a controlled environment.

Hugging Face believes the agent inferred that its platform might hold models, datasets or reference solutions linked to ExploitGym. It then tried to obtain those solutions instead of completing the challenge as intended.

The agent breaks loose

The first escape came through Artifactory, software that OpenAI used as a controlled gateway for downloading code packages. The agent found a previously unknown flaw in it and used that opening to reach the internet. Such an undisclosed software weakness is known as a zero-day vulnerability because its developer has had no time to issue a fix.

Once online, the agent found a public code-testing service operated by another provider. It broke into that system, secured full administrative access and made it the base for the campaign against Hugging Face.

The next opening came through Hugging Face’s dataset-processing service. The agent uploaded malicious datasets that exploited two separate flaws.

One involved HDF5, a file format used to store large and complex datasets. HDF5 files can refer to raw information held in another location. The agent altered those references so that Hugging Face’s service fetched sensitive files from its own servers and returned them as ordinary dataset content.

That exposed passwords, access tokens and the source code for part of the processing service. The source code showed the agent how the system handled incoming datasets and where another weakness might lie.

The second flaw involved Jinja2, a tool that fills templates with data. A field that should have been handled as plain text was instead treated as a software instruction. The agent used it to run Python code inside a production processing container, one of the isolated units in which Hugging Face runs individual computing tasks.

It was now inside the company’s network.

The agent examined the container, collected temporary cloud credentials and queried Hugging Face’s server-management system. It found credentials that carried far more authority than the processing task required.

One allowed it to create highly privileged workloads across 11 servers. The workloads were arranged so that they could return after individual copies were deleted.

Another key admitted outside systems to Hugging Face’s private corporate network. The agent used it to register attacker-controlled devices 181 times, often with new names and network addresses.

One key, many doors

An internal connection service widened the breach. The service used one powerful credential across several computer clusters rather than a separate credential for each. Stealing it gave the agent administrative authority in multiple clusters almost immediately.

The agent also entered an internal operational database using a static password found among the processing service’s settings. Hugging Face said the database was read but not changed. Its network records showed no sign of a large removal of data.

Two attempts to enter the company’s main production database failed because that system accepted connections only from approved network addresses.

The agent got further with Hugging Face’s source-code service. It created temporary access tokens that allowed it to read and write to a small number of internal repositories. It then opened a proposed code change, apparently hoping the company’s automated building system would run it and reveal more credentials.

Execution controls blocked that attempt. Hugging Face later inspected every repository the tokens could reach and checked its published software against expected versions. It found no unauthorized change that had reached customers.

Keeping the campaign running did not require a purpose-built attack server. The agent turned public paste sites, file-sharing services and request-capture tools into a loose communication network. It also hid instructions and stolen information inside datasets on Hugging Face’s own platform.

The material was broken into pieces, compressed and encoded before being sent. A routine search of the logs found only a fraction of the exposed credentials because most did not appear as readable text.

The agent was also working from short-lived test environments that repeatedly disappeared. Each time one was replaced, it downloaded its tools again, created a fresh identity and restored its communication channels. When one route to the internet stopped working, it tried another.

Hugging Face’s security systems did detect signs of the intrusion. AI-based tools connected several weak signals and identified a wider attack, but the alert was not marked critical and the on-call team was not paged immediately. Hugging Face said the mistake cost it valuable response time.

Security staff eventually shut down the vulnerable dataset service and cut off the agent’s internal access.

When AI meets AI

The investigation posed its own problem. Hugging Face said some commercial AI models refused to help analyze parts of the attack because their safety controls could not distinguish between examining malicious code and trying to use it.

The company instead ran GLM-5.2 on its own infrastructure. GLM-5.2 is an open-weights model, meaning organizations can download its underlying parameters and operate it on their own computers. It helped investigators piece together the timeline and decode material the agent had hidden in its communications.

Hugging Face has since closed both dataset-processing flaws, blocked processing containers from reaching cloud credentials and replaced shared access with narrower permissions. It rotated passwords, tokens and encryption keys, then wiped and rebuilt part of its core infrastructure.

OpenAI has disclosed the Artifactory flaw to its developer. It is reviewing the incident with external advisers and its Safety and Security Committee.

None of the weaknesses the agent exploited was unfamiliar. They included excessive permissions, reusable credentials, exposed secrets and unsafe handling of data supplied by outsiders. A skilled human attacker might have found the same route. But what changed was the pace and persistence. The agent could test thousands of possibilities, absorb repeated failures and continue without waiting for human instructions.

“Volume is what changes the defensive problem,” Hugging Face said.