AI Decline? Confidence in Autonomous Penetration Testing Falls

CYBERSECURITY OPERATIONS APPLICATION SECURITY CYBER RISK ENDPOINT SECURITY NEWS AI Decline? Confidence in Autonomous Penetration Testing Falls Companies are still experimenting with automated AI systems to find security weaknesses, but fewer are relying on the technology. Robert Lemos,Contributing Writer June 26, 2026 5 Min Read SOURCE: ASMAULNA VIA SHUTTERSTOCK In 2025, nearly 3 in 10 security professionals thought that fully autonomous AI systems could satisfy their companies' security-testing needs. But after a year of testing and experimentation, that optimism has largely gone away. Instead, chief information security officers (CISOs) and other security practitioners have more realistic expectations of the AI-based systems, which often have significant blind spots, are prone to false positives, and can blow through AI budgets, according to a June 25 report released by Cobalt, a penetration-testing-as-a-service firm. The number of organizations willing to rely on AI-powered penetration testing for their security needs fell to 9% in 2026, down from 29% a year earlier. The vast majority of companies preferred a hybrid, human-in-the-loop approach or relegating only non-critical tasks to automation. Security practitioners are experimenting to find the sweet spot of what can be automated reliably and responsibly, says Gunter Ollmann, chief technology officer for Cobalt. Related:AI Won't Wipe-Out Entry-Level Cybersecurity Jobs "CISOs in particular have been, for at least the last two years, under immense pressure by their leadership team, by their board, to use more AI, and autonomous pentesting fits that bill," he says. "Many of them now have a year under their belt of rolling out AI systems, as well as experimenting with AI pen testing tools, and generally ... their confidence in the security and the efficacy of these tools has dropped." Whether LLMs and AI systems will solve security problems or just present more challenges in the near and medium term is still a major question mark for security practitioners. Vulnerabilities are being reported at a 46% higher rate than forecasted from last year's data, according to an analysis from the Forum of Incident Response and Security Teams (FIRST). In another example of the challenges, Microsoft patched 206 unique CVEs in its June 2026 Patch Tuesday updates, a record driven by AI discovery of flaws. Human verification of AI-discovered flaws will be the bottleneck in the future, FIRST analysts Jerry Gamblin and Eireann Leverett wrote in their analysis. "In an era where AI can find significantly more flaws than human analysts, the constraint is no longer discovery; it is the human capacity to verify, coordinate, and patch," they wrote. "We also believe a crucial bottleneck will be in writing detection signatures for exploitation. The issue often comes down to the difference between identification and true risk detection." CISOs Face Massive Increase in Vulnerabilities A fundamental problem for organizations is that AI-augmented programmers are producing more code, and that translates to more vulnerabilities, even if the code is somewhat higher in quality. Meanwhile, security practitioners are focused on increasing the number of security assessments, with 77% committed to regular security assessments and pen testing, according to Cobalt's report. Related:Meeting Trump's 2030 Quantum Deadline Will be Expensive, Complex While that will require more automation, AI systems and large language models (LLMs) have shown weaknesses. Even with finding more vulnerabilities, AI systems are still missing high- and critical-severity issues, for example. Three-quarters of companies (78%) have had automated systems miss significant vulnerabilities — also known as "false negatives," dampening security professionals' enthusiasm for full automation, according to Cobalt's report.   The amount of data produced by AI security assessment also makes it difficult for humans to keep up, requiring more human oversight and better systems, says Derek Rush, managing senior consultant at Bishop Fox, an offensive security services firm. "These systems generate an enormous volume of data, and it takes an experienced mind to shape the context the LLM produces," he says. "A human expert is needed to decide whether a lead is worth pursuing, and if it is, to work out what the full, validated attack chain looks like. That judgment is exactly what gets missed when you take the human out, which is why buyers are running into the gaps." Related:Thanks for Crushing the Submissions Inbox. We're Trying to Keep Up HackerOne paused its Internet Bug Bounty program because of the growing volume of submissions that needed validation. AI systems could close the gap, but false negatives and false positives continue to be problems, says Sandeep Singh, vice president of product strategy at HackerOne. "False positives are the more familiar problem — they could be noisy and expensive in triage and validation time," he says. "This is where the need for stronger validation becomes more important. However, AI approaches could be used for validation as well so that it doesn’t become a bottleneck for humans." AI Security-Assessment Tools Will Get Better The lesson for CISOs: Don't expect AI penetration-testing and vulnerability-finding tools will replace human penetration testers anytime soon. In the short term, experts are required to get the most out of these systems, but AI systems will become more capable, says HackerOne's Singh. "We'd call it a bump [in the road] because the market briefly conflated 'AI can assist and amplify pentesting' with 'AI can replace the pentester,' and is now correcting," he says. "The durable model is agent-augmented human testing, meaning the agent does the relentless breadth continuously and first pass, and the human does the depth and judgment periodically." However, improvements in AI models will likely continue to emerge quickly, and "long term the trajectory is toward more autonomy," he says. For CISOs, that means that whether to use automation will boil down to the return on investment. Unfortunately, the costs of AI-powered penetration testing services are difficult to predict, and with examples of other business processes running up AI-service fees, security practitioners have become a bit gun-shy, says Cobalt's Ollmann. "The cost piece is a major concern," Ollmann says, but he sees a light — albeit, distant — at the end of the tunnel. "As a vendor in this space, we all expect ... AI is going to become cheaper and overall the net is we win." About the Author Robert Lemos Contributing Writer Rob is an award-winning, veteran technology journalist of more than 30 years, reporting on global cybersecurity issues, the latest offensive and defensive technologies, malware incidents, cyber conflict, and AI's impact on software and cybersecurity.  A former research engineer, Rob has written for more than two dozen publications, including CNET News.com, Dark Reading, MIT's Technology Review, Popular Science, and Wired News. He has received five awards for journalism, including Best Deadline Journalism (Online) in 2003 for his coverage of the Blaster worm. Rob also analyzes data on various trends using Python and R for both his reporting and his clients. Recent reports include analyses of the shortage in cybersecurity workers, annual vulnerability trends, and annual threat reports. Rob holds degrees from Cornell University in Electrical Engineering and Computer Science (double major). Want more Dark Reading stories in your Google search results? ADD US NOW More Insights Industry Reports The total economic impact™ of Snyk How Organizations Are Managing Incident Response How Enterprises Are Developing Secure Applications Inside RSAC 2026: security leaders reveal the risks redefining your defense strategy Essential News & Insights from Black Hat USA 2025 Access More Research Webinars Building a Risk Based Vulnerability Management Program Threat Hunting That Gets Big Results Despite Small Budgets Say Yes to AI: Securing Innovation Without Compromise Zero Trust Identity: Beyond Traditional Authentication Advanced Persistent Threats: A Practical Guide to Detection and Response More Webinars You May Also Like CYBERSECURITY OPERATIONS Hand CVE Over to the Private Sector by Brian Martin JAN 27, 2026 CYBERSECURITY OPERATIONS China Imposes One-Hour Reporting Rule for Major Cyber Incidents by Robert Lemos, Contributing Writer OCT 01, 2025 CYBERSECURITY OPERATIONS CISA, FBI, NSA Warn of Chinese 'Global Espionage System' by Alexander Culafi AUG 28, 2025 CYBERSECURITY OPERATIONS Women Who 'Hacked the Status Quo' Aim to Inspire Security Careers by Elizabeth Montalbano, Contributing Writer JUL 16, 2025 Editor's Choice CYBERSECURITY OPERATIONS Do CISOs Need a Code of Ethics? byDark Reading Editorial Team JUN 24, 2026 CYBERSECURITY OPERATIONS 2026 FIFA World Cup Faces Surge in Cyber Threats byAlexander Culafi JUN 24, 2026 3 MIN READ CYBERSECURITY OPERATIONS EU Gets a Head Start in Developing 6G Network Security byNate Nelson JUN 18, 2026 4 MIN READ Want more Dark Reading stories in your Google search results? Keep up with the latest cybersecurity threats, newly discovered vulnerabilities, data breach information, and emerging trends. Delivered daily or weekly right to your email inbox. SUBSCRIBE AUG 1-6 | MANDALAY BAY, LAS VEGAS USE CODE: DARKREADING & SAVE $200 ON A BRIEFINGS PASS OR $100 ON A BUSINESS PASS The premier cybersecurity event returns. GET YOUR PASS