Detokenization Cache Attack Can Reconstruct Local LLM Outputs, Researchers Warn

A new arXiv preprint says text generated by locally run large language models can be reconstructed from CPU cache activity during detokenization, a routine step in turning token IDs back into readable text. The finding challenges a core assumption behind on-device and self-hosted AI: that keeping a model local necessarily keeps its outputs private from other software on the same machine.

That matters because local and self-hosted LLMs are often chosen for privacy, latency or cost, including in agent systems that can access local files, credentials and persistent state. If generated text can leak across processes, sensitive prompts and responses may not stay confined to the application a user intended.

The 18-page paper, “Detokenization Leaks: Reconstructing Local LLM Outputs From Cache Traces,” was posted to arXiv as arXiv:2609.06674v1 after being submitted Sept. 6, 2026. Its authors are Roy Weiss and Benyamin Konstantinov of Ben-Gurion University, Eitam Sheetrit and Tomer Simon of Microsoft Security, and Yisroel Mirsky of Ben-Gurion University, the corresponding author.

The paper’s central claim is straightforward: “We present a new attack that reconstructs the text generated by locally hosted LLMs by observing CPU cache activity during detokenization.” The authors argue the focus on the detokenizer is what makes the method notable. Earlier LLM side-channel work often depended on narrower conditions, such as shared model memory, CPU offloading or particular model designs. By contrast, detokenization is a standard component in common inference pipelines.

In plain terms, the attacker described in the paper is not remote and does not need administrator access. It is an unprivileged user-space process running on the same machine as the victim LLM, scheduled on the simultaneous multithreading, or SMT, sibling of the victim core, with access to clflush and rdtsc instructions. The paper says the attack does not require elevated privileges or changes to the operating system.

Flush+Reload and Prime+Probe are established CPU cache side-channel techniques. What is new here, the authors say, is applying them to the detokenization step shared across common LLM stacks. According to the abstract, “To obtain clean signals, we use Flush+Reload on shared tokenizer code to detect when decoding occurs, which lets us perform Prime+Probe at the right moment and isolate token-dependent cache activity.”

The paper says commonly used tokenizer code is often loaded as shared libraries, enabling that trigger mechanism. From there, the attack uses a clustering and language-model pipeline to reconstruct text from noisy observations.

The researchers evaluated the method against Llama.cpp and Hugging Face Transformers, two widely used inference frameworks. Testing covered UltraChat, ChatDoctor and Code-Alpaca; Phi-3-mini and Llama-3; and multiple hardware platforms, including a 13th Gen Intel Core i9-13950HX.

For the trigger stage, the paper reports a 97.66% true-positive rate with zero false positives across 1.2 million simulated decode invocations. On reconstruction quality, it reports first-segment recovery reached as high as 62.92% high semantic similarity on ChatDoctor and 44.62% on UltraChat. For full responses, up to 58.48% of UltraChat reconstructions achieved semantic similarity of at least 0.9.

The authors also included an end-to-end test against OpenClaw, which they describe as an open-source local LLM agent system. In that setup, using a local Phi-3-mini model, the attack achieved a 30.12% attack success rate and mean semantic similarity of 42.23%, plus or minus 20.28%.

The paper discusses several mitigations, including randomizing decode tables, using short process lifetimes or periodic restarts, disabling SMT or Hyper-Threading, and applying cache partitioning, page-coloring or hardware cache defenses. The authors note tradeoffs in performance and usability.

As of Sept. 9, no public CVE, vendor advisory or official vendor statement tied to the paper had been located in the research provided.