Study finds structured-output decoding can enable 'control-plane' jailbreaks in LLMs

A research team says a standard feature meant to make large language models more reliable — structured-output decoding — can itself become a jailbreak path, with very high reported success rates across major systems from OpenAI, Google and others.

The finding matters because structured outputs are widely used in production AI systems, not just consumer chatbots. Many application programming interfaces, or APIs, use constrained decoding to force a model’s answer to match a JSON schema, tool definition or other machine-readable format. That helps software call tools, fill fields reliably and automate tasks. The new paper argues that same mechanism can create a separate security problem.

The paper, “When Grammar Guides the Attack: Uncovering Control-Plane Vulnerabilities in LLMs with Structured Output,” describes what the authors call “Constrained Decoding Attack (CDA), a new jailbreak class that targets the LLM control plane.” In plain terms, the attack does not primarily rely on a hostile-looking user prompt. Instead, it targets the grammar or schema that steers how the model is allowed to generate its response.

That distinction is central to the paper’s claim. Most jailbreak research has focused on the “data plane” — the visible text a user enters, or prompt injection embedded in retrieved content. This work argues there is also a “control-plane” risk in the hidden rules that constrain output formatting. The authors say that can leave the visible prompt looking benign while the decoding rules push generation in unsafe directions, “exposing a ‘semantic gap’ that demands cross-plane defenses bridging the data and control planes.”

The researchers describe two variants, EnumAttack and DictAttack. The paper says simple grammar auditing can blunt EnumAttack. But it presents DictAttack as the more evasive method and harder to detect, making it the more important result for defenders.

According to the paper, the team evaluated 13 proprietary and open-weight models across five standard jailbreak or harm benchmarks. It reports that DictAttack reached attack success rates of 94.3% to 99.5% on flagship models including gpt-5, gemini-2.5-pro, deepseek-r1 and gpt-oss-120b. One result the paper highlights is gpt-4o: DictAttack achieved a 99.8% attack success rate without defenses and 75.8% even when the model was wrapped in a stronger guarded setup.

That guarded setup is what the authors call a Dual-Plane Guard, adapted from prior work. The paper says the approach is designed to check both ordinary prompts and constrained outputs. Even so, DictAttack still posted a 75.8% attack success rate against it, suggesting that stronger prompt-side defenses alone may not be enough if schemas, grammars or tool definitions can also be influenced at inference time.

The authors are Shuoming Zhang, Jiacheng Zhao and co-authors. Their listed affiliations include the Institute of Computing Technology at the Chinese Academy of Sciences and the University of Chinese Academy of Sciences, along with collaborators at the University of Aberdeen, the University of Leeds and XCORESIGMA CO., LTD.

The paper is posted on arXiv as arXiv:2503.24191. The current PDF is marked version 4, dated Aug. 10, 2026, and the arXiv history shows earlier versions in 2025 and 2026. The manuscript says it is to appear at ACM CCS 2026, a major academic security conference.

The team also published a public, sanitized proof-of-concept repository on GitHub and a project page. The authors say the full evaluation harness and logs are gated for academic access, which limits full third-party reproduction from the public materials alone. On the project page, the researchers say, “We disclosed the underlying vulnerability to OpenAI and Google (Gemini) in early 2025; the embargo has since passed.” The assignment also says the team notified the maintainers of xgrammar.

As of Aug. 11, 2026, no public vendor statement specifically naming the paper was found. That means the takeaway should be framed narrowly: the research argues that defenses for production AI systems need to inspect not just prompts and user inputs, but also the schemas and grammars that shape what a model is allowed to say.