AI's Double Agent: Balancing Cybersecurity Gains and Risks

Strategic Insights -

As AI continues to revolutionize cybersecurity by enhancing defenders' abilities to detect anomalies and respond to attacks, it also poses a significant threat by making it easier for adversaries to scale and automate their attacks. The National Institute of Standards and Technology (NIST) is addressing this dual-use nature of AI in a series of virtual working sessions, aiming to untangle the complexities of AI in cybersecurity and find ways to balance gains and risks. The upcoming Thwarting AI Enabled Cyber Attacks workshop, scheduled for September 2, will focus on how AI-empowered attacks can be used to bypass traditional defenses and how agencies and organizations can build resilience in the face of these threats.

Key Takeaways:

  • AI is making it easier for adversaries to scale and automate their attacks, while also enhancing defenders' abilities to detect anomalies and respond to attacks.
  • The dual-use nature of AI is a concern for both government and the private sector, with NIST addressing this issue in a series of virtual working sessions.
  • Agentic AI, with its ability to autonomously adapt and execute multi-step operations, is a particularly dangerous development in the near future, and is already being used for offensive operations.
  • According to a recent Palo Alto Network's Unit 42 report, agentic AI can increase the speed, scale, and sophistication of attacks, with the average time it takes to break in and start exfiltrating data reduced to just 25 minutes.
  • NIST and other experts emphasize the need for proactive defense, including automated red teaming, zero-trust principles, and better identity controls, as well as the importance of human oversight in the development of AI-driven code.
  • The Open Worldwide Application Security Project warns about the growing presence of agentic AI variations of existing threats, including memory poisoning, misuse of integrated tools, and privilege escalations.

Statistics:

  • 44% of attacks in a recent Palo Alto Network's Unit 42 simulation study were successful, with the agentic AI able to thwart or trick most traditional defenses.
  • The average time it took an agentic AI to break in and start exfiltrating data was 25 minutes, 100 times faster than with a non-AI enhanced attack.
  • 37.6% increase in critical vulnerabilities in a University of San Francisco study after just five rounds of AI changes, with the problems getting worse with more iterations.

Sources:

  • National Institute of Standards and Technology (NIST) (https://www.nccoe.nist.gov/projects/cyber-ai-profile)
  • Palo Alto Network's Unit 42 (https://www.paloaltonetworks.com/blog/2025/05/unit-42-develops-agentic-ai-attack-framework/)
  • University of San Francisco study (https://arxiv.org/html/2506.11022v1)
  • Open Worldwide Application Security Project (https://genai.owasp.org/resource/agentic-ai-threats-and-mitigations/)
  • NIST's upcoming Thwarting AI Enabled Cyber Attacks workshop (https://www.nccoe.nist.gov/get-involved/attend-events/nist-nccoe-cyber-ai-profile-virtual-working-session-series-thwarting-ai)