Should you let an AI agent attack your production network?
Someone’s AI is going to test your production network. Make sure it’s yours.
The Situation
Anthropic, one of the leading AI model developers, disclosed last November the first reported AI-orchestrated cyber espionage campaign, in which AI performed 80-90% of the work against roughly 30 targets, including financial institutions. This month, Microsoft’s Digital Defense Report concluded that AI has given attackers the near-term advantage. Attackers are already aiming AI agents at networks like yours, so AI will find your weaknesses either way; your only decision is whether your team finds them first.
The Exposure
Penetration testing typically runs once a year, the minimum many regulators set (for example, New York’s Department of Financial Services). Boards know cyber risk well, but only the largest firms with a dedicated and mature cyber risk function have evaluated how AI changes it. The major AI providers now ship more capable models every few months, and Anthropic has released new versions almost monthly this year, so any annual test falls behind what hackers are already using. Keeping pace with AI threats is now a fiduciary question for the board.
The Judgment Call
Today your CISO’s team, or the third-party testing firm it hires, runs that annual test and keeps AI to automated scanning, because an autonomous agent applied to a production system risks taking it down. That natural caution means the only live AI testing your production network is experiencing every day belongs to someone else. In a Stanford-led study, the AI agent outperformed 9 of 10 professional penetration testers on a live network of about 8,000 hosts, and one version of the AI only cost $18 an hour versus $60 for a person. It has a higher false-positive rate, but that’s why a person has to direct it. Let a person-directed AI agent attack your network, not just scan it, and run it at least quarterly under rules of engagement the board has seen. Coordinate with your cloud providers and essential vendors, as they’ll need notice and agreed terms before an agent touches anything they host.
This Week’s Action
Ask your CISO when a person-directed AI agent last attacked your network, and if the answer is never, what rules of engagement the board would need to see before one runs this quarter.
Working through a specific AI decision right now? Reply to this email and tell me what it is. I read every response and answer directly.
Blackbox Zero helps regulated mid-market firms govern AI and put it to work, as a Fractional Chief AI Officer, through a 12-Week Governance Sprint, or starting with a 2-hour AI Exposure Diagnostic. Reply with "Fractional," "Sprint," or "Diagnostic" to start a conversation.
Chris Cook writes Judgment Call every other week. He’s a former IBM Vice President and Deputy Chief Auditor, published in Corporate Board Member and AI Journal.
Chris Cook
Managing Partner & Founder, Blackbox Zero
Forwarded this by a colleague? Subscribe to Judgment Call
