What the Detector Can See: Evaluating CPS Anomaly Detectors Independently of the Decision Rule

Computer Science > Cryptography and Security [Submitted on 3 Aug 2026] What the Detector Can See: Evaluating CPS Anomaly Detectors Independently of the Decision Rule Peiran Shi, Jian Xiang, Xiang Zhang, Chenglong Fu Anomaly detectors are often the last line of defense for cyber-physical systems (CPS). But detectors built in very different ways, from deep neural networks to invariant templates, are usually compared using precision, recall, or F1 at a single operating point. These scores mix two separate things: how well the detector represents the physical process, and how well its alarm threshold is set. We therefore treat a CPS anomaly detector as a two-stage pipeline: Stage 1 maps observations to residuals, and Stage 2 maps residuals to alarms. Instead of scoring only the final alarms, we evaluate Stage 1 directly using normalized residual energy, which has an exact connection to the Kullback-Leibler divergence from the trained-normal reference distribution. Because it does not depend on a specific alarm rule, it can separately measure attack separation, stability across the train-test gap, and the compactness with which a detector encodes the plant. Without any per-detector tuning, we apply this evaluation to five detectors -- GDN, FuSAGNet, TranAD, NSIBF, and GeCo -- across three CPS benchmarks: SWaT, WADI, and HAI. Although the detectors have similar ROC-AUC values on SWaT, their performance differs by more than an order of magnitude at a common false-alarm rate. Rankings also change across testbeds: TranAD ranks first on HAI but last on SWaT, while NSIBF ranks first on WADI but last on HAI. On WADI, localized attacks can evade detectors that pool evidence across all channels, helping explain why NSIBF outperforms methods that do well on other benchmarks. These results show that detection failure can come from different sources: a weak representation, poor threshold calibration, or an attack with little physical effect. A decision-rule-free analysis helps separate these causes. Comments: 17 pages, 5 figures, 8 tables. Code and data: this https URL Subjects: Cryptography and Security (cs.CR) Cite as: arXiv:2608.02821 [cs.CR]   (or arXiv:2608.02821v1 [cs.CR] for this version)   https://doi.org/10.48550/arXiv.2608.02821 Focus to learn more Submission history From: Peiran Shi [view email] [v1] Mon, 3 Aug 2026 19:35:00 UTC (2,032 KB) Access Paper: HTML (experimental) view license Current browse context: cs.CR < prev   |   next > new | recent | 2026-08 Change to browse by: cs References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)