Reasoning Structure Matters for Safety Alignment of Reasoning Models

Computer Science > Artificial Intelligence [Submitted on 21 Apr 2026] Reasoning Structure Matters for Safety Alignment of Reasoning Models Yeonjun In, Wonjoong Kim, Sangwu Park, Chanyoung Park Large reasoning models (LRMs) achieve strong performance on complex reasoning tasks but often generate harmful responses to malicious user queries. This paper investigates the underlying cause of these safety risks and shows that the issue lies in the reasoning structure itself. Based on this insight, we claim that effective safety alignment can be achieved by altering the reasoning structure. We propose AltTrain, a simple yet effective post training method that explicitly alters the reasoning structure of LRMs. AltTrain is both practical and generalizable, requiring no complex reinforcement learning (RL) training or reward design, only supervised finetuning (SFT) with a lightweight 1K training examples. Experiments across LRM backbones and model sizes demonstrate strong safety alignment, along with robust generalization across reasoning, QA, summarization, and multilingual setting. Comments: ACL 2026 Subjects: Artificial Intelligence (cs.AI) Cite as: arXiv:2604.18946 [cs.AI]   (or arXiv:2604.18946v1 [cs.AI] for this version)   https://doi.org/10.48550/arXiv.2604.18946 Focus to learn more Submission history From: Yeonjun In [view email] [v1] Tue, 21 Apr 2026 00:50:13 UTC (770 KB) Access Paper: HTML (experimental) view license Current browse context: cs.AI < prev   |   next > new | recent | 2026-04 Change to browse by: cs References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)