Carnegie Mellon Preprint Finds Reinforcement Learning on Benign Facts Can Amplify LLM Leakage of Memorized Emails

A new Carnegie Mellon University preprint reports a counterintuitive privacy risk in large language models: reinforcement learning on harmless factual question-answer data, with no personal information in the training set, made several models significantly better at revealing private data they had already memorized. The strongest result came from DeepSeek-V3.1, where targeted verbatim recall of Enron employees’ email addresses rose from 0.155 to 0.370 after post-training, according to the paper posted Friday on arXiv.

The finding matters because the training in question was not aimed at private data at all. It used reinforcement learning with verifiable rewards, or RLVR, a post-training method used to improve model reasoning on tasks with checkable answers. In plain terms, the paper suggests that routine work to make models better at factual reasoning may also make latent personal information easier to extract — without obvious warning signs in standard release checks. The authors write: “Memorized private data can be made markedly more extractable by training that never touches it.”

The paper, “Reinforcement Learning on Benign Facts Amplifies Leakage of Memorized Private Data,” was posted to arXiv as arXiv:2608.21727 v1 on Aug. 22. Its authors, Renfei Zhang and Niloofar Mireshghallah of Carnegie Mellon University, studied three instruction-tuned models ranging from roughly 8 billion to 671 billion parameters: Qwen3-8B, Qwen3.5-397B-A17B and DeepSeek-V3.1. To test private-data recall, they used the public Enron email corpus to build prompts that asked for named employees’ email addresses, as well as open-ended prompts asking models to list any Enron email addresses they knew. The reinforcement-learning data came from FACTS-Parametric, a public set of 1,052 short-answer factual questions that the paper says contained no email addresses or other personally identifiable information.

Across all three models, targeted verbatim recall@10 increased after reinforcement learning on the benign facts. Qwen3-8B rose from 0.005 to 0.050. Qwen3.5-397B-A17B increased from 0.050 to 0.135. DeepSeek-V3.1 climbed from 0.155 to 0.370, which the authors describe as a 2.4-fold increase. In untargeted testing, the number of real Enron addresses produced also increased: DeepSeek-V3.1 went from 50 to 83, Qwen3.5-397B-A17B from 5 to 65, and Qwen3-8B from 0 to 16.

The researchers included checks aimed at ruling out a simpler explanation that the models were merely guessing or becoming less cautious. Decoy recall stayed at zero for synthetic email addresses that had never appeared in training data, according to the paper. In DeepSeek-V3.1’s untargeted outputs, precision stayed essentially flat, moving from 82.0% to 82.2%, suggesting the model was not simply fabricating more addresses. The paper also says the effect was not explained by reduced refusals. In one example, DeepSeek-V3.1’s union refusal rate on the targeted probe remained 8.78% before and after reinforcement learning. The authors say reasoning abilities and refusal rates were retained, arguing that the training changed which memorized information was accessible rather than broadly changing the model’s behavior.

Researchers have already shown that large language models can memorize and sometimes regurgitate pieces of their training data, including personal information. What is new here is the claim that unrelated post-training can make already-memorized private data easier to extract even when the fine-tuning set contains no personal data. That is notable because reinforcement-learning-based post-training is now a standard part of many modern LLM pipelines used to improve reasoning and answer quality.

The paper is a preprint, not a peer-reviewed journal publication, and its scope is narrow. The experiments covered one type of personally identifiable information — English-language Enron email addresses — from one public corpus, so the authors do not claim the result generalizes to all forms of private data or all populations. In an ethical note, they say they used only the public Enron dataset and will not release the target pool or reinforcement-learning checkpoints, reporting aggregate counts only to avoid making the underlying associations easier to obtain.