How did attackers try to extract hidden reasoning from OpenAI models?
OpenAI says it blocked a coordinated campaign spanning more than 15,000 users and attributes a core cluster to people associated with Moonshot AI. The disclosure provides useful scale and mitigations, but it remains the provider's account and does not show how many attempts succeeded.
The short version
- What happened
- OpenAI says a campaign that began on 1 July produced a 16,000-request spike from more than 4,000 users on 24–25 July and formed part of a related cluster spanning more than 15,000 users.
- Who it affects
- Developers and organisations may unknowingly expose credentials or personal data when they publish agent logs containing opaque reasoning blocks.
- What remains unproven
- The campaign scale, attribution and mitigation claims come from OpenAI, the affected provider, and have not been independently audited in the public record.
Keep in mind
The campaign scale, attribution and mitigation claims come from OpenAI, the affected provider, and have not been independently audited in the public record.
OpenAI reports attempted extraction patterns but does not disclose a success denominator, the amount of reasoning recovered or measured downstream capability gain.
The related research paper is an arXiv preprint and its public-log corpus is separate from the July campaign described by OpenAI.
Multi-source analysis · 2 linked sources. Primary analysed source date: 30 September 2026. Earliest linked supporting source: 10 August 2026.
Read the full report · 6 min The evidence, methods, implications and limitations
The disclosure describes interaction abuse, not a database breach
OpenAI disclosed on 30 September that it had investigated and disrupted what it calls a coordinated adversarial-distillation campaign. The company says the earliest activity appeared on 1 July, remained relatively low-volume and then spiked on 24 and 25 July. Those spikes comprised 16,000 requests using a relevant extraction pattern from more than 4,000 users. A wider prompt-pattern investigation identified related activity across a cluster of more than 15,000 users, which OpenAI says it fully disrupted by 28 July.
The distinction between the attack route and a conventional breach matters. OpenAI says the operators did not defeat encryption, enter a database or obtain direct access to stored user conversations. Its account says they manipulated normal model interactions so that hidden reasoning could be reproduced in visible form. One method involved moving encrypted reasoning from one conversation and asking another model interaction to decrypt and transcribe it. That is a security failure mode at the application and model boundary, even when the underlying encryption remains intact.[1]
The headline numbers count attempts, not confirmed theft
OpenAI explicitly notes that its request figures describe attempted extractions and are not a denominator of successful ones. It does not publish the number of traces recovered, which models were most exposed, how much useful training material the operators obtained or whether the extracted material improved another model. Readers should therefore resist converting 16,000 requests or 15,000 users into 16,000 proven leaks or 15,000 human attackers. Automated account networks can also make user counts a poor proxy for the number of people directing them.
The company attributes a core cluster of the activity to individuals associated with Moonshot AI, the Chinese developer of Kimi. It also says it is unclear whether every operator in the relevant period belonged to one actor. No technical evidence supporting that attribution is published in sufficient detail for outsiders to reproduce it, and the disclosure includes no response from Moonshot. The attribution should be reported as OpenAI's assessment, not as an independently established fact about the whole campaign.[1]
Independent research shows why portable reasoning blocks are sensitive
OpenAI links the incident to an eight-author arXiv preprint submitted on 10 August. That work reports that encrypted reasoning blocks could be moved between sessions, users and models within a provider's ecosystem, enabling a weaker model to reveal reasoning produced by a stronger one. The researchers say they demonstrated related attacks across Anthropic, OpenAI and Google. This is a preprint, not a peer-reviewed paper, but OpenAI says it investigated the disclosed paths, confirmed that they were real and used the work to accelerate mitigations.
The preprint analysed 315,320 encrypted reasoning blocks scraped from public repositories and reports recovering 367 items of personally identifiable information and 182 credentials. Those figures describe the researchers' corpus of publicly exposed agent logs; they are not counts from OpenAI's July campaign. Together, the paper and the incident report show that a supposedly opaque reasoning artefact can still carry secrets or hazardous instructions when it is portable and accepted by another model context.[1][2]
The practical lesson is to treat reasoning artefacts as sensitive credentials
OpenAI says it banned or restricted fraudulent accounts, strengthened signup and infrastructure controls, expanded network monitoring and closed a replay path that could reveal another user's encrypted reasoning. It also added checks that can hold streamed output when reasoning might be exposed and shared indicators through the Frontier Model Forum and government channels. These mitigations are consequential, but the company does not publish detection sensitivity, false-positive rates or an external audit showing the same protections across every partner-hosted deployment.
Developers and organisations should avoid logging or publicly posting opaque reasoning blocks on the assumption that encryption makes them harmless. Logs need access controls, retention limits, secret scanning and incident procedures. Platforms should bind protected artefacts to a user, session, model and purpose so that copied material cannot be replayed elsewhere. Security teams also need to monitor coordinated behaviour across accounts; a single request may look benign while a distributed sequence creates an extraction pipeline.[1][2]
What would change our assessment
Confidence in the incident account would rise with an independent technical review of the attack path, success rate, attribution process and deployment coverage. Comparable disclosures from cloud partners and other model providers would show whether controls work consistently across the wider ecosystem. A standard for cryptographically binding reasoning artefacts to authorised contexts, followed by cross-provider testing, would turn one company's mitigation into a more general defence.
The assessment would worsen if researchers reproduce extraction against current production systems, if partner deployments retain the replay path, or if later evidence shows that useful model capabilities or user secrets were recovered at scale. For now, the evidence supports a real and important class of model-security risk plus a substantial attempted campaign. It does not support a claim that encryption was broken, that all attempts succeeded or that every account belonged to Moonshot AI.[1][2]