Enhancing Malware Detection with Generative AI: Using Variational Autoencoders to Boost Machine Learning Classifiers' Performance

Computer Science > Cryptography and Security [Submitted on 17 May 2026] Enhancing Malware Detection with Generative AI: Using Variational Autoencoders to Boost Machine Learning Classifiers' Performance Mohammad Alharbi, Jeremy Straub The advancement of malware poses obstacles for cybersecurity, necessitating the development of advanced detection techniques. This paper proposes an approach to enhance malware detection through the use of a generative artificial intelligence model. Specifically, variational autoencoders (VAEs) are used with the random forest, XGBoost and sequential model machine learning classifiers. Generated synthetic malware samples are used to address the critical issue of data scarcity for new or less common malware types. This approach can be used to augment datasets to improve classifier robustness. The proposed methodology uses VAEs to create high-quality diverse synthetic datasets that closely mimic real-world malware data. The effectiveness of these augmented datasets is evaluated by comparing the performance of the machine learning classifiers when they are trained with the original data and when they are trained with the synthetic data-augmented datasets. The results demonstrate a notable improvement in the accuracy, precision, recall and F1-scores of the classifiers, when they are trained using the augmented datasets. The enhanced performance for detecting various malware classes shows the potential of this approach to facilitate adaptation to evolving malware threats effectively. This work demonstrates the utility of generative AI for cybersecurity. It also provides a foundation for future research aimed at developing more resilient and adaptive malware detection systems. Subjects: Cryptography and Security (cs.CR) Cite as: arXiv:2606.06501 [cs.CR]   (or arXiv:2606.06501v1 [cs.CR] for this version)   https://doi.org/10.48550/arXiv.2606.06501 Focus to learn more Submission history From: Jeremy Straub [view email] [v1] Sun, 17 May 2026 05:14:31 UTC (439 KB) Access Paper: view license Current browse context: cs.CR < prev   |   next > new | recent | 2026-06 Change to browse by: cs References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)