Paper Demonstrates Chain-of-Thought Hijacking Attack

The arXiv paper "Chain-of-Thought Hijacking" by Jianli Zhao et al. (arXiv:2510.26418, revised 24 May 2026) presents a black-box jailbreak that leverages extended inference-time reasoning to elicit harmful compliance. The paper reports attack success rates of 99%, 94%, 100%, and 94% on the HarmBench benchmark against Gemini 2.5 Pro, ChatGPT o4 Mini, Grok 3 Mini, and Claude 4 Sonnet, respectively. The authors state they release their evaluation materials to support reproducibility.

The paper frames the target class as Large Reasoning Models (LRMs) and describes a method that induces long, benign puzzle-solving chains of thought before introducing a harmful instruction. To diagnose the mechanism, the authors report using activation probing, attention-pattern analysis, and causal interventions on open-source reasoning models, and they name the resulting failure mode "refusal dilution," where refusal-related activations attenuate as reasoning traces lengthen.

Extended chain-of-thought prompting is already used to boost accuracy on multi-step tasks; the paper's results suggest that the same mechanism can reduce safety-related signal strength. Observed phenomena such as attention shifts and low-dimensional safety signals are consistent with prior work showing that some control behaviors are encoded in compact internal representations. This paper documents how those representations can be diluted by long reasoning trajectories.

Jailbreaks are an ongoing, evolving risk. The high reported success rates across four commercially known closed-source models, as reported in the paper, indicate a broad attack surface for reasoning-capable models. For red-teamers and safety engineers, the paper provides both an attack recipe and diagnostic tools (probing and causal interventions) to evaluate mitigation effectiveness.

For practitioners: monitor whether public model evaluations begin including long-chain-of-thought jailbreak probes, whether vendors publish mitigations targeted at refusal-dilution modes, and whether subsequent work reproduces the described attention and activation patterns on different architectures.