Start the day here
Business — Chain of Thought — APIs
Labs told developers the model's private scratchpad was sealed. The client got an opaque blob to ship back on the next turn, and the readable answer stayed on the surface.
On August 10, a team from MATS Research, ELLIS Tübingen, Max Planck, and Snyk posted Stealing Reasoning Traces from Proprietary LLM APIs on arXiv. The seal opened with a weaker sibling model and a prompt.
Anthropic, OpenAI, and Google encrypt chain-of-thought for some reasoning models so the API can stay stateless. The server returns an AEAD envelope. The app stores it and returns it with the next request. The server decrypts, restores context, and continues. On paper, the developer never sees the thoughts.
The paper's finding is architectural. Those envelopes were interchangeable across sessions, accounts, and models inside each provider's family. Capture a block from Claude Opus 4.8, replay it into Claude Haiku 4.5 in another account, ask Haiku to transcribe its reasoning context, and Haiku prints Opus's scratchpad. Frontier refusals never fire. The cheaper model does the reading.
Cryptographically, AEAD still worked. The associated data simply never bound session, user, or model identity. Integrity without context is a shared lock on every locker in the building.
Developers publish agent transcripts to GitHub and Hugging Face for debugging and demos. Many of those files include the encrypted fields verbatim: signature, thinkingSignature, encrypted_content, and cousins.
The researchers scraped 6,708 public trajectories, decoded 315,320 reasoning blocks, and recovered 182 credentials from genuine user sessions, including 62 API keys and 33 passwords, plus hundreds of PII artifacts. Sixty-four of 704 artifacts from real sessions lived only in the hidden reasoning. Visible sanitization missed them.
One Codex agent, asked to scrub a repo before publication, restated the keys it was removing inside the sealed block. The developer then published the block, believing it opaque. TechTimes and The Decoder covered the same scan after the paper landed.
Johns Hopkins cryptographer Matthew Green reported replayability to OpenAI and Anthropic in late May. OpenAI called his report unreproducible. Anthropic said it saw no security implication in the side channel, though it might update docs. Lead author Alexander Panfilov has publicly confirmed that timeline.
After the paper's responsible disclosure, all three providers patched current APIs so the demonstrated attacks no longer reproduce. Anthropic's docs now say thinking blocks belong to the producing model and should be stripped when switching. Fresh envelopes got tighter. The 315,320 blocks already sitting in public repos remain an open question no lab has answered in public.
The authors list four vectors. Distillation: recover frontier reasoning cheaply and train against it; even short Opus fragments nudged Kimi-K3 toward Opus-style outputs in their tests. Hazardous content: visible refusals can hide intermediate steps the extraction surfaces. Invisible prompt injection: a malicious instruction buried entirely inside an encrypted block can redirect another session's agent without appearing in readable logs. Cross-user extraction: the public-repo scan, already counted.
Alignment researchers have spent years arguing that monitorable chain-of-thought is a fragile safety asset. Labs encrypted that window partly to block distillation and leakage. The paper shows the encryption gave the appearance of a sealed audit trail while shipping a portable escrow to every client. Related Sonar work on forged scratch notes sits next door: once CoT is an object on the wire, attackers treat it as infrastructure. See Researchers Tricked Models With Forged Scratch Notes.
Call the design what it was. Providers needed a way to hide thoughts from users and still bill thinking tokens across stateless calls. They chose a global key and unbound envelopes. Developers followed the docs and published "harmless" ciphertext. Researchers proved the ciphertext was a shared diary.
Strip reasoning fields from any shared transcript. Audit pre-August logs that still carry those blobs. Treat "encrypted thought" as a product claim until the envelope names its session. The labs can patch tomorrow's API. They cannot unsay what already shipped to GitHub.
Lab rankings, model preference, API volume, coding-agent heat, open-source stars, and prediction markets.
Letters
0
No letters yet.