Adaptive Intrusion Detection System using Transformer-Based Neural Networks and Continual Learning Approach with Adversarial Investigation

Computer Science > Cryptography and Security [Submitted on 5 Aug 2026] Adaptive Intrusion Detection System using Transformer-Based Neural Networks and Continual Learning Approach with Adversarial Investigation Azizi Ariffin, Afif Haris, Faiz Zaki, Hazim Hanif, Nor Badrul Anuar Network intrusion detection systems (IDS) trained on fixed traffic snapshots decay silently after deployment as threat distributions shift. Fine-tuning models on new attacks triggers catastrophic forgetting, while retraining from scratch is computationally infeasible. Replay-based continual learning counters this, but existing methods unrealistically confine benign traffic to a single early task and ignore the replay buffer as a potential attack surface. To address this, we present an adaptive IDS framework coupling a tabular transformer encoder with a class balanced experience replay buffer that replays benign traffic at every update to stabilize decision boundaries. We introduce the class-instance incremental (CII) scenario where benign flows reappear alongside new attacks as a more faithful stress test, and probe the buffer with overt label flipping and stealthy backdoor poisoning attacks. On the CICIDS2017 benchmark, our framework achieved 0.9994 accuracy under the traditional class incremental setup and 0.9989 under CII, with negligible forgetting, drastically outperforming sequential fine-tuning (0.0052), EWC (0.0324), LwF (0.0699), and iCaRL (0.8770) baselines. While injecting benign traffic into every experience proves essential for preventing forgetting, the replay buffer introduces critical vulnerabilities. Label-flipping collapses the model entirely (0.0053 accuracy at a 1% budget), and the backdoor maintains 0.97 overall accuracy while driving the attack success rate on trigger flows to 95%, evading standard monitoring. Ultimately, while a modest replay budget recovers near-joint-training performance, ensuring buffer integrity emerges as a strict operational requirement. Subjects: Cryptography and Security (cs.CR) Cite as: arXiv:2608.04602 [cs.CR]   (or arXiv:2608.04602v1 [cs.CR] for this version)   https://doi.org/10.48550/arXiv.2608.04602 Focus to learn more Submission history From: Azizi Ariffin [view email] [v1] Wed, 5 Aug 2026 09:06:45 UTC (1,145 KB) Access Paper: view license Current browse context: cs.CR < prev   |   next > new | recent | 2026-08 Change to browse by: cs References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)