Not long ago, a provider publicly accused a competitor of large-scale distillation in the media, drawing widespread attention to how intense the data offense and defense behind LLMs has become. Using outputs from strong models to train smaller ones can significantly boost the smaller models’ capabilities—a technique heavily validated in engineering practice. To protect their intellectual property, model providers have begun hiding the reasoning processes generated before a model outputs a response, attempting to cut off this distillation vector.
Having just read two arXiv papers, the empirical findings are quite eye-opening. While it was widely assumed that encrypting reasoning traces would block capability distillation, this approach has failed to achieve its intended protective effect in practice. Even more surprising is that the mechanisms designed by providers to hide reasoning end up introducing risks for every AI builder using their APIs, even compromising data safety in everyday debugging logs.
By mid-2026, Anthropic, OpenAI, and Google have all disabled the display of plaintext reasoning processes. Before producing a final response, the backend model executes a complex thinking process that contains intermediate reasoning as well as potentially sensitive information from historical memory. To avoid maintaining large amounts of user state on the server side, providers package this reasoning process into an encrypted data block and send it to the client, requiring the client to echo it back unchanged on the next request. It is as if the provider handed you a locked keybox that you must carry with every API call, while the provider keeps the key, leaving you unable to read the text inside.
The problem is that encrypted reasoning blocks within the same provider’s ecosystem are interchangeable across models. This compatibility creates a channel for cross-model invocation. Encrypted data blocks generated by Anthropic’s Opus 4.8 can be directly received and processed by Haiku 4.5. The GPT-5.6 series can replay reasoning traces from all earlier GPT generations, and the entire Gemini series is similarly cross-compatible.
This gives rise to a security asymmetry. Frontier model Opus 4.8 undergoes rigorous refusal training and will not easily leak its reasoning upon prompt request. However, Haiku 4.5, optimized for cost and speed, lacks equivalent protection. Attackers do not need to compromise Opus 4.8 directly; by simply placing its generated encrypted blocks into requests sent to Haiku 4.5, they can leverage Haiku 4.5 to decrypt the reasoning.
How does this work in practice? This study describes two injection methods. The first places the encrypted reasoning block at the beginning of the current response turn, prompting the weaker model to continue generating directly from that reasoning, under the illusion that it had thought through those steps previously. The second embeds the block into the prior conversation history, raising even fewer suspicions from the weaker model. Following injection, the attacker issues a simple prompt instructing the weaker model to transcribe its prior thoughts verbatim using special markers. Anthropic’s Haiku 4.5 complies readily; researchers used the exact same fixed extraction prompt to make it dutifully read out the encrypted reasoning across these attack scenarios. Each provider’s choice of decoder model varies: Anthropic uses Haiku 4.5, OpenAI uses GPT-5.6 Luna, and Google uses Gemini Robotics 1.6.
How do we confirm that the extracted text is indeed the model’s genuine reasoning rather than hallucinations from the weaker model? Researchers evaluated 120 competitive programming problems, re-encoding the extracted reasoning and comparing the token count with the thinking token counts reported by the API. The two numbers matched with high precision, confirming that the extracted content reflects what the model actually thought inside the encrypted block.
The overall attack costs very little. Using Haiku 4.5 to decode 10,000 reasoning traces—each with an input/output window of roughly 12,000 tokens—costs about $720 under standard API pricing. For this modest expense, one can harvest Opus 4.8’s authentic reasoning traces at scale to distill custom models. Providers went to great lengths to hide their reasoning, only for it to be rendered entirely transparent.
One might assume that fixing secret isolation and hardening access permissions for smaller models would resolve this vulnerability. However, another study published in March 2026 presents an even more sobering reality: even if encrypted blocks were rendered perfectly secure, competitors could still distill model capabilities unabated.
This paper takes a fundamentally different angle. Attackers do not need access to actual reasoning traces at all; by observing only the publicly returned final answers, they can reverse-engineer high-quality reasoning chains. Brief summaries, when available, can also be utilized, though they are not strictly required.
The approach consists of three steps. First, the attacker uses a publicly available weak reasoning model as a proxy to generate a large collection of input, reasoning, and answer tuples over open datasets. A summarization model then condenses the reasoning into summaries, and an inversion model is trained on these samples. The core capability learned by this inversion model is: given an input, an answer, and an optional summary, infer the underlying reasoning process. Second, the public answers and optional summaries generated by the target model (such as GPT-5.4 mini) are fed into the trained inversion model to synthesize reasoning traces for the target model. Third, these synthesized reasoning traces are used to train a student model. The researchers have also open-sourced the implementation code.
There is a crucial detail here: the proxy model used to train the inversion model is substantially weaker than the target model. In their experiments, the researchers used R1-Distill-Qwen-1.5B (R1-Weak) with only 1.5 billion parameters, whereas the target model GPT-5.4 mini was a black-box commercial model. Yet, the inversion capability learned from the weak proxy successfully transfers to the stronger target model.
The experimental results are solid. Researchers trained student models on inverted reasoning and evaluated them across several benchmarks. On the MATH500 math benchmark, a 7-billion-parameter open-source model saw its accuracy increase from 68.4% to 76.0% after being trained on inverted reasoning. For a larger student model, accuracy jumped from 13.0% to 52.4%. Inverted reasoning also significantly outperformed directly using summaries paired with answers for training. Querying GPT-5.4 mini 10,000 times incurred an API bill of approximately $173, with no further queries to the target model required during subsequent inversion and fine-tuning.
Another notable finding: if the proxy model and the target model happen to be identical, the inverted reasoning actually performs better than true reasoning in certain evaluations. True reasoning often contains detours, dead ends, and self-corrections, whereas inverted reasoning provides clean forward derivations, serving as a superior training signal.
While cryptographic flaws in code can be patched, the implicit information carried by an output cannot be shielded. As long as an LLM outputs a logically sound, correct response, external parties can backtrack the underlying thought steps from the result. Relying on encryption to prevent distillation offers limited utility against information inversion; fundamentally, this defensive line cannot block distillation demand.
The impact of extracting encrypted blocks extends far beyond model distillation. For AI builders building applications, even if you never touch model distillation, this encryption mechanism directly impacts your everyday engineering workflow.
When debugging Agent issues, developers routinely share sanitized session logs on GitHub or community forums. Even if you carefully scrub API keys and personal passwords from visible plaintext prior to posting, the encrypted data block returned with the API response may still harbor this sensitive information. Under the attack conditions disclosed in the paper, third parties downloading these execution logs can recover the enclosed text using the method described earlier.
This is not a theoretical concern. Authors of this study compiled statistics from 6,708 public Agent execution traces gathered across GitHub and Hugging Face, extracting 315,320 encrypted reasoning blocks. Upon decryption, they recovered 62 API keys, 33 login passwords, 24 access tokens, 7 private keys, and 30 personal email addresses. Even more troubling, 64 of these sensitive items had never appeared anywhere in the preceding visible plaintext conversation. A model may retrieve sensitive data from memory and write it into hidden reasoning, or restate such data while processing anonymization or sanitization prompts; scrubbing visible text leaves the content inside the encrypted block intact.
Another risk stems from the reuse of execution traces. When constructing long-horizon Agents, developers frequently reuse high-quality conversational context to save token costs. If someone embeds a malicious encrypted block into a shared trace—such as a covert instruction to exfiltrate files—the system may execute those actions upon loading the context. Experiments demonstrate that this injection technique enables an Agent to exfiltrate sensitive files while performing routine tasks, completely bypassing plaintext log inspections.
These two scenarios illustrate a clear engineering cost: developers are left holding a black-box data block that can neither be inspected nor sanitized. Providers encrypt reasoning to safeguard their IP, yet in real-world applications, the security burden and operational uncertainty fall squarely on the developers.
The practice of hiding the reasoning process yields questionable real-world effectiveness. On the defense side, output inversion can derive equivalent reasoning traces, while smaller models can decipher the contents of encrypted blocks. The goal of preventing distillation remains unachieved, and the thinking process is left unprotected.
On the application side, opaque encryption mechanisms introduce unnecessary friction. Sharing a session log on GitHub to troubleshoot an issue might expose undeletable API keys; adopting shared Agent execution traces might introduce hidden malicious instructions. Anti-distillation is a commercial battle among providers, yet the resulting data security burden is left to every software developer.
Hiding data from users while granting decryption privileges to easily manipulated small models creates an inherent logical contradiction. While providers’ desire to protect commercial assets is understandable, shifting the protection burden onto developers lacks architectural justification. LLMs must decrypt context to process subsequent turns; as long as attackers can manipulate lightweight models like Haiku via prompt engineering, information inside these blocks remains permanently exposed to risk.
Models may retrieve sensitive keys from memory and write them into encrypted blocks, leaving developers with neither visibility nor erasure controls—surrendering data governance in the process. As Agents assume increasingly complex business workflows, every developer making API calls must re-evaluate the latent risks inherent in this architecture.