Breaking Customized LLMs for Coding: Automated Red Teaming for Instruction Backdoor Attacks

Computer Science > Cryptography and Security [Submitted on 6 Aug 2026] Breaking Customized LLMs for Coding: Automated Red Teaming for Instruction Backdoor Attacks Yuchen Chen, Wei Cheng, Yuan Xiao, Wising Sun, Chunrong Fang, Yang Liu, Zhenyu Chen, Baowen Xu LLM customization platforms allow users to build task-specific models for code intelligence tasks by embedding instructions into system prompts, without modifying the underlying model parameters. While these platforms lower the barrier to developing customized LLMs, they also introduce a new attack surface: instruction backdoor attacks, in which adversaries implant hidden malicious behaviors into customized instructions. However, existing attacks suffer from two key limitations. First, they often rely on explicit trigger patterns readily detected by platform-side or user-side inspection. Second, they require substantial manual effort to craft task-specific backdoored instructions, limiting their scalability. In this paper, we propose ARIA, an automated red-teaming framework for crafting covert and effective backdoored instructions against customized LLMs. ARIA leverages an attacker LLM to iteratively generate and refine backdoored instructions, guided by structured feedback from the target LLM along three dimensions: stealthiness, clean-task utility, and backdoor effectiveness. We evaluate ARIA on three code intelligence tasks, using four representative LLMs, and compare it with three baseline attacks. Experimental results show that ARIA achieves the highest attack success rate of 0.945, while maintaining the best clean-task utility across all tasks. ARIA also generalizes well across programming languages and remains robust to generation temperature. Furthermore, ARIA significantly outperforms existing attacks in evading platform-side and user-side detection, achieving a false negative rate of up to 1.000, and stays effective against existing defense methods, demonstrating its strong generalizability and robustness. Comments: Accepted to the 41st IEEE/ACM International Conference on Automated Software Engineering (ASE 2026 Subjects: Cryptography and Security (cs.CR) Cite as: arXiv:2608.05659 [cs.CR]   (or arXiv:2608.05659v1 [cs.CR] for this version)   https://doi.org/10.48550/arXiv.2608.05659 Focus to learn more Submission history From: Yuchen Chen [view email] [v1] Thu, 6 Aug 2026 06:57:10 UTC (434 KB) Access Paper: HTML (experimental) view license Current browse context: cs.CR < prev   |   next > new | recent | 2026-08 Change to browse by: cs References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)