Which Defense Closes Which Threat? Attributing OWASP-LLM-Top-10 Coverage and Its Brittleness Under Paraphrasing

Computer Science > Cryptography and Security [Submitted on 1 Jun 2026] Which Defense Closes Which Threat? Attributing OWASP-LLM-Top-10 Coverage and Its Brittleness Under Paraphrasing Alexandre Cristovão Maiorano Production LLM applications stack several defense families -- refusal-phrase filters, token-budget controls, model allowlists, rate limits, tool-registry authentication -- yet existing breach-and-attack-simulation (BAS) benchmarks report a single aggregate coverage number, hiding which family closes which threat. We measure attribution. We add four OWASP-LLM-Top-10-aware agents to a 21-agent baseline scanner and target a lattice of four synthetic LLM endpoints: L_0 (no defenses), L_1 (refusal-only), L_2 (budget-only), and L_3 (full stack). L_1 and L_2 are sibling single-axis ablations, not subsets of each other; L_3 is their union plus tool-registry authentication and credential scrubbing. Across N=10 replications, the per-OWASP finding count is clean: refusal alone removes all LLM01 (jailbreak) and LLM07 (system-prompt leakage) findings; budget alone removes all LLM02 (sensitive-info disclosure) and LLM10 (unbounded consumption) findings by terminating multi-step sequences; LLM06 (excessive agency) requires the full stack. We probe brittleness under paraphrasing: with 300 Gemini-generated paraphrases (K=5 over a 60-template brittleness corpus), L_1 refusal block rate falls 15 pp on LLM01 and 25 pp on LLM07. A fifth target, L_4-real, swaps the stub backend for Gemini-2.5-flash behind the same L_3 regex and matches L_1 exactly, indicating no measurable alignment contribution beyond the regex (not a general claim about alignment). Budget controls show no drop (0 pp once the rate-limit floor is factored out). A refusal whitelist that clears a static benchmark can be defeated by an LLM-driven paraphraser without changing attack intent; a budget control resists the same mutation. Comments: 17 pages, 4 figures, 7 tables Subjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI) Cite as: arXiv:2606.02822 [cs.CR]   (or arXiv:2606.02822v1 [cs.CR] for this version)   https://doi.org/10.48550/arXiv.2606.02822 Focus to learn more Submission history From: Alexandre Maiorano PhD [view email] [v1] Mon, 1 Jun 2026 19:39:25 UTC (33 KB) Access Paper: HTML (experimental) view license Current browse context: cs.CR < prev   |   next > new | recent | 2026-06 Change to browse by: cs cs.AI References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)