Securing AI Agents Before They Go Rogue Is Next to Impossible

CYBER RISK APPLICATION SECURITY CYBERSECURITY OPERATIONS THREAT INTELLIGENCE NEWS Securing AI Agents Before They Go Rogue Is Next to Impossible High-autonomy agents with broad permissions and unfettered access are a recipe for disaster, and enterprises need to act now before they become the next horror story. Rob Wright,Senior News Director,Dark Reading June 2, 2026 4 Min Read SOURCE: SUCHAT LONGTHARA VIA GETTY IMAGES Agentic AI adoption is in full swing, but unfortunately for enterprises, completely securing these agents might not be feasible. That's according to Dennis Xu, research vice president at Gartner, who spoke about the dangers of rogue AI agents during the Gartner Security & Risk Management Summit on Monday. "There's a lot of them coming at us — whether we like it or not, whether we know it or not," he said during his presentation.  Xu discussed some of the recent "horror stories" about AI agents going rogue, including the recent PocketOS incident in which an AI coding agent deleted the company's production database and volume-level backups in just nine seconds. The agent, he emphasized, performed these tasks to be helpful, but it gained access to an API for PocketOS's infrastructure provider, which led to catastrophic results. "The industry, by the way, does not have a complete answer for all this yet," he said, which puts the burden on enterprise security teams to build an effective defense on their own. Related:Beyond Assume-Breach: How AI-Native Security Will Reshape Enterprise Defense High Autonomy Agents Create Big Risk Xu's session focused on custom-built agents that organizations deploy within their environments. Most agentic AI offerings today, approximately 90%, are low-autonomy agents, he said. But the remaining 10% are high-autonomy agents that have broad access to tools and data, as well as the freedom to apply reason at runtime to determine the best ways to complete assigned tasks. While those agents may be valuable to organizations because they can perform complex tasks, Xu said securing them is "an open challenge" today, for a variety of reasons, starting with jailbreaks. "The large language model as it is today will always be susceptible to jailbreak and prompt injection attacks — always," Xu said, adding that organizations can't prevent such attacks 100% of the time, regardless of how much money is spent on guardrails. "Jailbreaks are the number one reason why AI agents are risky." But they're not the only reason, Xu said. Today's AI — even the new frontier models — aren't totally reliable when they apply reason, he said. And when unreliable reasoning is combined with privileged system access, high autonomy, and access to sensitive data, that's a recipe for a rogue agent. How Security Teams Can Police Agents Xu outlined several steps that organizations need to take to prevent their AI agents from going rogue. The first step is simple: you can't secure what you can't see, so an effective security program requires agent discovery within the network. This can be performed in a variety of ways, from scanning code repositories to Extended Berkeley Packet Filter (eBPF) monitoring. Related:Anthropic to Open Mythos AI to EU's ENISA The second step is AI security posture management, which Xu said is "super, super complicated" because it must account for the agent itself, the agent's access in the environment, the agent's skills, and the underlying infrastructure, such as the MCP servers. Additionally, the posture management must be continuous, Xu said, because it must be continuous, as AI agents need to be monitored at runtime.  "Once an agent is put into runtime, a lot of components here can be updated," he said. "That same agent that you did the initial assessment, risk analysis, and posture of when it's going to launch into production will no longer be the same agent once it's in production." The third step is penetration testing and red teaming. This is an important step to determine if, for example, agents are over-permissioned and accessing data and systems outside of their designed purpose. According to a recent study by security vendor Akeyeless that surveyed more than 400 IT and security leaders, 84% of respondents said their AI agents can access sensitive data — and 67% believe agents have already accessed data they shouldn't have. Related:As Global Powers Explore Humanoid Robots, Cyber-Risk Looms For the final step, Xu stressed the importance of strong agent defenses. This includes protecting against prompt injections (which he again noted will never be 100% effective), preventing specific threat actor techniques such as memory poisoning, and monitoring for high-risk actions (such as deleting production databases) and "toxic combos" of tools and data usage, he said. For example, an agent may have access to a CRM database and Web fetch tools, and a threat actor potentially could manipulate the agent in pulling sensitive customer data and publishing it on a malicious website. Most importantly, Xu said, is behavior-based detection for the agents themselves that continuously analyzes their activity at runtime to see if it deviates from their baselines. But once again, Xu noted "there's no easy way to do that," as this is an emerging area with unproven solutions. Instead, organizations must closely watch the agent's intent and compare it against the organization's and developer's intents for the agent. For example, a security team could analyze the runtime behavior of an agent over 30 days and see that it only used one of the tools and two of the permissions for which it was initially granted access.  "That could be a good posture finding to say, 'Hey, you know what? Your developer may be over-permissioned their agent,'" he said. "Permission right-sizing, tooling right-sizing — these similar concepts for we've been doing in security for a long time also apply here." About the Author Rob Wright Senior News Director, Dark Reading Rob Wright is a longtime reporter with more than 25 years of experience as a technology journalist. Prior to joining Dark Reading as senior news director, he spent more than a decade at TechTarget's SearchSecurity in various roles, including senior news director, executive editor and editorial director. Before that, he worked for several years at CRN, Tom's Hardware Guide, and VARBusiness Magazine covering a variety of technology beats and trends.  Prior to becoming a technology journalist in 2000, he worked as a weekly and daily newspaper reporter in Virginia, where he won three Virginia Press Association awards in 1998 and 1999. At TechTarget and Dark Reading, he has won several Azbee awards, including the 2026 National Silver Award for a series on vibe coding.  At Dark Reading, Rob currently covers security operations, cloud security, and Internet infrastructure. He has a keen interest in malvertising activity and the certificate authority industry, and has written extensively on both topics. He graduated from the University of Richmond in 1997 with a degree in journalism and English. A native of Massachusetts, he lives in the Boston area.  Want more Dark Reading stories in your Google search results? ADD US NOW More Insights Industry Reports How Organizations Are Managing Incident Response How Enterprises Are Developing Secure Applications Inside RSAC 2026: security leaders reveal the risks redefining your defense strategy Essential News & Insights from Black Hat USA 2025 How Enterprises Are Harnessing Emerging Technologies in Cybersecurity Access More Research Webinars The Frontier AI Era: Why Cybersecurity Must Move at Machine Speed Build vs. Buy: The Hidden Cost of Building Your Own AI Security Stack Defending in the Shadow Era: When the CVE Feed Goes Dark Building SecOps That Make the Most of Every Dollar AI-Powered Credential Security: Intelligence Without Exposure More Webinars You May Also Like CYBER RISK How Can CISOs Respond to Ransomware Getting More Violent? by James Doggett JAN 28, 2026 CYBER RISK US Cyber Pros Plead Guilty Over BlackCat Ransomware Activity by Alexander Culafi JAN 05, 2026 CYBER RISK Switching to Offense: US Makes Cyber Strategy Changes by Robert Lemos, Contributing Writer NOV 21, 2025 CYBER RISK Microsoft Exchange 'Under Imminent Threat,' Act Now by Arielle Waldman NOV 12, 2025 Editor's Choice CYBERSECURITY OPERATIONS 20 Leaders Who Built the CISO Era: 2 Decades of Change byDark Reading Editorial Team MAY 12, 2026 41 MIN READ APPLICATION SECURITY It's Patch Tuesday for Microsoft & Not a Zero-Day In Sight byJai Vijayan MAY 12, 2026 5 MIN READ CYBERATTACKS & DATA BREACHES Instructure Breach Exposes Schools' Vendor Dependence byAlexander Culafi MAY 6, 2026 4 MIN READ Want more Dark Reading stories in your Google search results? Keep up with the latest cybersecurity threats, newly discovered vulnerabilities, data breach information, and emerging trends. Delivered daily or weekly right to your email inbox. SUBSCRIBE Webinars The Frontier AI Era: Why Cybersecurity Must Move at Machine Speed TUESDAY, JUNE 23, 2026 1:00 PM EDT Build vs. Buy: The Hidden Cost of Building Your Own AI Security Stack THURS, JUNE 25, 2026, AT 1PM EST Defending in the Shadow Era: When the CVE Feed Goes Dark TUES, JUNE 16, 2026 AT 1PM EST Building SecOps That Make the Most of Every Dollar THURS, JULY 9, 2026 AT 1PM EST AI-Powered Credential Security: Intelligence Without Exposure WED, JUNE 17, 2026, AT 1PM EST More Webinars BLACK HAT USA | MANDALAY BAY, LAS VEGAS The premier cybersecurity event of the year returns to Mandalay Bay with a re‑engineered, six‑day program built to ignite innovation, push boundaries, and bring the global security community together like never before. Use code: DARKREADING to save $200 on a Briefings pass or $100 on a Business pass. GET YOUR PASS