An Embarrassingly Simple Detector for Model Extraction Attacks in Large Language Model API Traffic

Computer Science > Cryptography and Security [Submitted on 4 Jun 2026] An Embarrassingly Simple Detector for Model Extraction Attacks in Large Language Model API Traffic Shuze Liu, Qianwen Guo, Yushun Dong Large language models (LLMs) are increasingly deployed through hosted APIs, making model extraction a practical threat to model ownership and service security. However, individual extraction queries often resemble benign requests, and existing evaluations often focus on single-query anomaly scoring or pure benign-versus-attacker user settings. We formulate model extraction monitoring as benign-calibrated traffic-window distribution testing and show that an embarrassingly simple detector is effective: embed incoming queries into a semantic space and test whether their aggregate distribution deviates from historical benign traffic. We instantiate the detector with maximum mean discrepancy (MMD), using only benign-vs-benign comparisons to set the decision threshold. We evaluate on fourteen attacker-normal query pairs from four extraction scenarios and compare with adapted PRADA, SEAT, CAP, DATE, and marginal Mahalanobis baselines. Across three random seeds, MMD achieves 0.3% benign FPR, 100.0% pure-attacker TPR, 90.5% average TPR over attacker fractions, and 95.1% balanced accuracy. These results show that benign-calibrated distribution testing is a strong empirical baseline for model extraction detection in both user-level and mixed multi-user LLM API traffic. Code is released at: this https URL. Comments: Preprint. Code available at this https URL Subjects: Cryptography and Security (cs.CR); Computation and Language (cs.CL) Cite as: arXiv:2606.05725 [cs.CR]   (or arXiv:2606.05725v1 [cs.CR] for this version)   https://doi.org/10.48550/arXiv.2606.05725 Focus to learn more Submission history From: Shuze Liu [view email] [v1] Thu, 4 Jun 2026 05:33:49 UTC (233 KB) Access Paper: HTML (experimental) view license Current browse context: cs.CR < prev   |   next > new | recent | 2026-06 Change to browse by: cs cs.CL References & Citations NASA ADS Google Scholar Semantic Scholar Export BibTeX Citation Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Demos Related Papers About arXivLabs Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)