Security of 100 AI Agents Tested and Ranked – What You Need to Know

AI is our new leader. We just accept and do what it tells us. Maybe we should be a bit more circumspect. Concern over the performance of AI agents has been constant, ranging from ‘leaky’ to just plain wrong decision-making. Since the pressure to use more agents more autonomously because of supercharged AI-assisted attacks is now constant, Adversa AI’s decision to measure and compare the performance and security of 100 agents across ten categories is welcome. But the results are not. Of the 100 agents tested, and positioned within a new AI Risk Quadrant, only 11 are categorized as ‘capable well-defended’.  The root problem is the AI agent ‘lethal trifecta’, which Adversa describes as ‘private data access + exposure to untrusted content + ability for outbound actions’. This translates directly into the standard lethal trifecta of too much power + too much trust + too little control’. Since all three parts of this trifecta are necessary for an AI agent to achieve its goal, capability and security will always be a big ask. Ninety-eight percent of the agents have this trifecta, so it is no surprise to learn – but still shocking to hear – that so few are both capable (useful) and defendable (secure). Capability and security verge on mutual exclusion. “The same vendors shipping the most capable agents ship the widest attack surface – a structural feature of the market, not a handful of outliers,” states Adversa’s analysis in its AI Risk Quadrant for Agent Security report. It calls this a ‘power-protection inversion’ and adds that it appears in all ten agent categories. The agent categories with the greatest power protection inversion, however, are ‘computer agents’ followed by ‘coding agents’. Computer agents are designed to perform a specific task, such as make a decision or perform an action for a user. Since agents can only operate with what they know (the context problem, where poor context leads to bad decisions in all agents), computer agents are given wide access rights, effectively the complete operating system. “A compromise hands the attacker the user’s entire machine, not just one application or tab,” warns Adversa. Such agents also suffer from an issue that affects all agents: the user has little, if any, visibility into or control over what the agent actually does. It is given an input (the task), and it generates an output (the completed task). But with computer agents, the user doesn’t know the route it takes between input and output, nor what specific actions within the operating system it takes along that route. “The deeper issue is that the desktop confirmation step looks like a control while being unreliable in practice,” warns the analysis. ‘The human and the model reason over different abstractions (windows and labels vs. screenshots and accessibility trees). That gap produces confirmation mismatch: the human approves the appearance of the action, not what the agent is about to do, because nothing in the interface surfaces the difference.” The second-worst offender in the exposed giants quadrant is coding agents. This is concerning since ‘vibe-coding’ applications are becoming the future of software, and ‘vibe-coded’ in-house applications may live with us for many years. The analysis sub-divides coding agents into three types: “coding copilots (human reviews each suggestion), autonomous coding agents (goal-in, repo-out), and app builders (prompt-to-deployed-app). The first might appear to be the least dangerous, but the user still doesn’t know what the agent does between input and output. “Coding agents don’t just write code – they touch shell, dependencies, and tokens long before a diff lands in review,” comments Adversa. “This is the class where compromise most directly becomes production compromise. The danger is not bad code suggestions; it is high-trust operation inside the software supply chain. Non-determinism makes code review an incomplete defense: even if a human reviews the final diff, the agent may already have traversed secrets, run tests against production-like services, modified configs, or selected risky dependencies. Review catches outputs; it does not catch the full action trail.” Coding agents figure so highly among the exposed giants because they have a wide attack surface, an extensive blast radius, and poor defense controls. The attack surface is wide because they run shell commands, load MCP servers, and auto-load rules files. The blast radius comes from sitting inside the software supply chain with access to secrets, signing keys, and deployment pipelines. And their primary defense is a code review of the output, which doesn’t consider either the attack surface or the blast radius. We’ve glanced at just two of the ten agent types included in Adversa’s agent analysis and AI Risk Quadrant. The other eight categories are general assistant, work copilot, browser, conversational, custom workflow, business process, platform operations, and data engineering. None come out squeaky clean. Ninety-eight percent of the tested agents are subject to the lethal trifecta, with only one agent in each of the general assistant and data engineering agents being the exceptions. Learn More at the AI Risk Summit | Ritz-Carlton, Half Moon Bay General comments from Adversa include: agent defaults favor velocity over safety; agents with the most power have the least protection, while the agents with the most protection have the least power; only 11% qualify for the capable and defended quadrant; tool execution accounts for 76% of blast radius; 37% of the market is audited more than defended; and 83% of claimed AI agent defenses are not publicly verifiable. Agents are effectively black boxes – it’s a take it or leave it scenario. Business economics is forcing us to take it. Since we cannot control what the agent does while it is running, our only option is to be careful over what we input, and control, where possible, the output.  Here, Adversa recommends concentration on controlling the output since there is little that can be done on the input prompts. “Defend the legs you can own, not the one you can’t,” it suggests. “Prompt injection has no deterministic fix – no classifier reliably separates the agent’s data from its instructions, and vendors concede it. Concede the input boundary and spend the defensive budget on the trifecta legs the operator does control: egress, identity, and irreversible actions.” This is where we are today. The headlong rush into agentic AI solutions is irreversible but concerning. We will only match adversarial AI-assisted attacks by using AI-assisted defense. All businesses will only remain competitive if they are faster, and more efficient than the competition. In business, all roads lead to AI. We must hope, and can probably expect, that AI will improve in all areas in the future. To what extent and when that may happen is another unknown.  But in the meantime, the ultimate message from Adversa’s massive and detailed analysis is clear: “Let’s be careful out there.” Related: Can We Trust AI? No – But Eventually We Must Related: The Wild West of Agentic AI – An Attack Surface CISOs Can’t Afford to Ignore Related: Sweet Security Launches Agentic AI Red Teaming to Counter ‘Mythos Moment’ Related: Raising the Cybersecurity Stakes: Ante up for the Agentic Era WRITTEN BY Kevin Townsend Kevin Townsend is a Senior Contributor at SecurityWeek. He has been writing about high tech issues since before the birth of Microsoft. For the last 15 years he has specialized in information security; and has had many thousands of articles published in dozens of different magazines – from The Times and the Financial Times to current and long-gone computer magazines. More from Kevin Townsend Exclusive: How One Line of Code Put Billions of Microsoft Android App Downloads at Risk Russia-Linked ‘GreyVibe’ Attackers Use AI to Supercharge Cyberattacks New Edamame Platform Aims to Catch AI Coding Agents Going Off the Rails The Credential Crisis: How Stolen Credentials Defeat Modern Security ‘SymJack’ Attack Turns AI Coding Agents Into Supply Chain Attack Delivery Systems AppOmni’s Marlin AI Brings Autonomous Investigation to SaaS Security Open Source DockSec Uses AI to Cut Through Vulnerability Noise in Docker Images Supply Chain Security Crisis: Too Many Vulnerabilities, Too Little Visibility Latest News Kirki, Burst Statistics WordPress Plugin Flaws in Attackers’ Crosshairs Hackers Target Global Stock Exchange in Espionage Operation IMA Diligence Services Data Breach Impacts 525,000 People Organizations Warned of Exploited Linux Kernel Vulnerability ‘HTTP/2 Bomb’ Exploit Knocks Web Servers Offline in Seconds Microsoft Tries to Calm Legal Threat Fears After Zero-Day Disclosure Backlash Trump Signs Executive Order That Invites Vetting of Top AI Models for National Security Risks Two New Reports Offer Competing Explanations for Cybersecurity’s Growing Crisis Trending Webinar: Third-Party Risk In Practice June 4, 2026 Organizations are investing heavily in third-party risk management, but breaches, delays, and blind spots continue to persist. Join this live webinar as we examine the gap between how organizations think their third-party risk programs are performing and what’s actually happening in practice. Register Virtual Roundtable: CISO Forum 2026 Mid-Year Review June 10, 2026 Explore how attackers are using AI to scale threats and how security teams can respond with AI-driven defenses. Protecting against unmonitored use of generative AI (Shadow AI) in business units and building and enforcing AI governance frameworks. Register People on the Move Rapid7 announced that Wael Mohamed will assume the role of Chief Executive Officer, replacing current Chief Executive Officer Corey Thomas, who will become Executive Chairman of the Board. Anurag Jain has been appointed Senior Vice President of Engineering at CodeHunter. CTERA has appointed Tal Sarfaty as Senior Vice President of Cybersecurity. More People On The Move Expert Insights The Zero-Knowledge Threat Actor And The End Of Responsible Disclosure AI can help attackers generate malware, create malicious payloads, bypass simple security checks, and convert vague malicious intent into functional code. (Etay Maor) Raising The Cybersecurity Stakes: Ante Up For The Agentic Era CISOs are now facing machine-speed attacks and asking, “How do I agent?” The industry must provide remediation at scale. (Nadir Izrael) Caught Off Guard: Securing AI After It Hits Production As enterprises rush AI projects into production, security teams are increasingly being forced into reactive mode. (Joshua Goldfarb) Cyber Resilience Is The New Business Continuity Plan The organizations best prepared to face disruption are those that align security, continuity and risk management around what the business cannot afford to lose. (Steve Durbin) Enhancing Data Center Security Without Sacrificing Performance For AI data centers, where the stakes are the highest and performance constraints are the tightest, security and performance are no longer a zero-sum game. (Nadir Izrael) Flipboard Reddit Whatsapp Email