Improving DNS Exfiltration Detection via Transformer Pretraining

Computer Science > Cryptography and Security [Submitted on 9 Apr 2026] Improving DNS Exfiltration Detection via Transformer Pretraining Miloš Tomić, Aleksa Cvetanović, Predrag Tadić We study whether in-domain pretraining of Bidirectional Encoder Representations from Transformer (BERT) model improves subdomain-level detection of exfiltration at low false positive rates. While previous work mostly examines fine-tuned generic Transformers, it does not aim to isolate the effect of pretraining on the downstream task of classification. To address this gap, we develop a controlled pipeline where we freeze operating points on validation and transfer them to the test set, thus enabling clean ablations across different label and pretraining budgets. Our results show significant improvements in the left tail of the Receiver Operating Characteristic (ROC) curve, especially against randomly initialized baseline. Additionally, within pretrained model variants, increasing the number of pretraining steps helps the most when more labeled data are available for fine-tuning. Comments: This is the preprint version of the paper. The final version of the paper has been presented at the TELFOR 2025 conference. The paper has 4 pages, 1 figure and 3 tables Subjects: Cryptography and Security (cs.CR); Machine Learning (cs.LG) Cite as: arXiv:2604.09849 [cs.CR]   (or arXiv:2604.09849v1 [cs.CR] for this version)   https://doi.org/10.48550/arXiv.2604.09849 Focus to learn more Journal reference: 2025 33rd Telecommunications Forum (TELFOR), Belgrade, Serbia, 2025, pp. 1-4 Related DOI: https://doi.org/10.1109/TELFOR67910.2025.11314363 Focus to learn more Submission history From: Miloš Tomić [view email] [v1] Thu, 9 Apr 2026 15:58:34 UTC (108 KB) Access Paper: HTML (experimental) view license Current browse context: cs.CR < prev   |   next > new | recent | 2026-04 Change to browse by: cs cs.LG References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)