Oxford Preprint: Frontier AI Out‑persuades Expert Humans in Text Chats, Study Finds

An Oxford-led research team says frontier artificial intelligence systems outperformed expert human persuaders in large text-chat experiments, including a real-money charity test in which the AI was more than three times as effective as professional canvassers.

The finding comes from an arXiv paper titled “AI out-persuades expert humans in text-based dialogue” that is labeled “Preprint. Under review.” The paper, first submitted June 15 and revised Nov. 6, reports four preregistered experiments covering 18,978 conversations with 6,923 persuadees.

The authors are affiliated with the University of Oxford, the UK AI Security Institute, Stanford University and the London School of Economics and Political Science. Their studies used single-blind, between-subjects online experiments, meaning participants had a text-based conversation with either an AI or a human persuader but did not know which one they were assigned. The chats lasted from two to 10 turns.

The tests are notable because they measured direct head-to-head persuasion, not just whether a model could generate persuasive writing on its own. Human comparison groups included random laypeople, top-performing laypeople from a preregistered persuasion tournament, professional canvassers and elite competitive debaters, including world and continental champions.

The paper says the expert humans were allowed to choose their own issues, research in advance, practice for hours and compete for 1,000-pound cash bonuses. The AI systems tested were frontier large language models including Claude Opus, ChatGPT-4o/GPT-5.4, Grok 4.20 and Gemini 2.5 Pro.

In the attitude-change studies, participants discussed specified U.K. policy issues and were scored on a 0-100 scale before and after the conversation. The paper reports that AI outperformed random laypeople by 8.2 percentage points, selected laypeople by 5.6 points and elite debaters by 4.6 points. All of those results were reported as statistically significant, with p-values below .001.

The abstract sums up the headline claim this way: “Together, these results establish that frontier AI systems out-persuade expert humans in text-based conversation, with significant implications for political communication.”

One of the paper’s more useful nuances is that better human preparation helped, but did not erase the gap. In Study 2, coaching improved the performance of elite human debaters, according to the paper. But the AI advantage disappeared when researchers constrained the AI to human writing speeds and human-length messages, suggesting that a major edge came from information throughput — how quickly the systems could produce longer, denser arguments in a text chat.

The strongest practical result came in Study 4, a behavioral test involving real money. In that experiment, 2,888 participants could donate part of a 1-pound study bonus to Save the Children after a text conversation. The AI increased donations by 21.3 percentage points compared with a control group, while professional canvassers increased donations by 6.6 points. By that measure, the AI exceeded canvassers by 14.7 points and was a little more than 3.2 times as effective.

The paper also points to an important limit: the effects appear to fade. In a follow-up survey conducted in September 2026, which was not preregistered, the authors found that attitude changes had decayed substantially over time. One example in the paper says AI’s effect fell from 12.9 points immediately after the conversation to 1.9 points at follow-up; the comparable human effect fell from 7.3 points to 0.4 points.

The researchers say data sufficient to reproduce the results and the analysis code are public, though full transcripts were withheld for privacy and dual-use reasons. The work received ethics approval from the Oxford Internet Institute’s Departmental Research Ethics Committee and the Research Assurance Board at the UK AI Security Institute.

The paper’s scope is narrower than the headline may suggest. These were text-only online conversations, many focused on U.K. policy issues, and the paper remains under review rather than peer-reviewed. Still, the scale of the experiments and the head-to-head comparisons with highly incentivized experts make the preprint a notable early test of how persuasive frontier AI can be in direct conversation.