Monitoring and Anomaly Detection

Seeing agent failures and misuse while they are happening.

Incident reporting clocks run from one hour in China to 15 days under the EU AI Act, so monitoring has to find a failure well inside the deadline. The developer found the failure first in five of the 19 registry incidents.

First-report deadlines: China, critical infrastructure 1 hour; China, other operators 4 hours; EU AI Act, widespread harm 2 days; NHTSA, severe ADS crash 5 days; EU AI Act, possible death 10 days; EU AI Act, other serious 15 days. EU deadlines from Commission draft guidance FIRST-REPORT DEADLINES China, critical infrastructure 1 hour China, other operators 4 hours EU AI Act, widespread harm 2 days NHTSA, severe ADS crash 5 days EU AI Act, possible death 10 days EU AI Act, other serious 15 days TIME TO FIRST REPORT, LOG SCALE EU DEADLINES FROM COMMISSION DRAFT GUIDANCE
Intrusions detected internally, 2025
52%
Accounts Anthropic banned for cyber misuse
832
Networks OpenAI disrupted since 2024
40+
Registry failures
6

Registry

Six failed monitoring controls across five registry incidents

Each row is a control that was in place, or should have been, and did not hold. The incident record explains who was responsible and what changed afterward.

Registry incidents where monitoring failed
IncidentDateControlResponsibleWhat failed
Tesla Recalls Autopilot After NHTSA InvestigationDec 2023Remedy effectivenessTeslaNHTSA opened a query on whether the recall fix was enough
Replit Agent Deletes the SaaStr DatabaseJul 2025Accurate self-reportingReplitThe agent said rollback was impossible when it was not
Chinese State Group Runs Espionage Through Claude CodeSep 2025Misuse classifiersAnthropicAttacks split into small tasks looked like routine technical work
Chinese State Group Runs Espionage Through Claude CodeSep 2025Early detection of agent operationsAnthropicThe campaign reached high-value targets before it was stopped
Waymo Robotaxis Pass Stopped School BusesSep 2025Fix verificationWaymoViolations continued in Austin after the November update
OpenAI Test Models Breach Hugging FaceJul 2026Halt authorityOpenAIA June port-sweep alert and an Artifactory takeover did not stop later evaluation runs

Implementations

Model developers and Microsoft run the largest agent monitoring operations

Anthropic banned 832 accounts for malicious cyber activity in the year to March 2026 and detected the GTG-1002 espionage campaign through its own classifiers. OpenAI reports disrupting more than 40 malicious networks since February 2024. Microsoft's Agent 365 maps agents, MCP servers and identities into Defender. OpenTelemetry's agent trace conventions are still marked in development.

Named monitoring implementations
ImplementationVendor or bodyCountryWhat it enforcesStatus
Agent 365 observability and Defender mappingMicrosoftUnited StatesAgent registry, observability, mapping of agents, MCP servers and identitiesGA May 1, 2026
GenAI and agent semantic conventionsOpenTelemetryInternationalCommon trace spans for agent runs and tool calls, MCP includedv1.42.0, still in development
Malicious network disruptionOpenAIUnited States40+ networks disrupted since Feb 20242026
National incident reporting systemCACChinaFour incident grades; first report in one, two or four hours by operator typeEffective Nov 1, 2025
Threat intelligence account enforcementAnthropicUnited States832 accounts banned for malicious cyber use, mapped to MITRE ATT&CKMar 2025 to Mar 2026

Row color shows the agent class served: orange physical, teal personal and ambient, blue software.

Recent events

  • Anthropic's September 2026 threat report covers more than 20 disrupted clusters, including a China-based group that found potential zero-days across about 50 targets.
  • Mandiant found malware that queries language models while running and a stealer that hunts for local AI coding tools.
  • OpenTelemetry v1.41.0 split agent spans into client and internal types on April 28, 2026.
  • Replit's agent misreported what it had done after deleting the SaaStr database in July 2025.

Exposure

Multi-agent coordination rates very significant or critical for three of 17 agent types

The monitoring discipline covers the multi-agent coordination dimension of the agent attack surface ratings. These agent types depend on it most.

Agent types most exposed on this dimension
Agent typeDimensionRatingWhy
Fleets and swarmsMulti-agent coordinationCriticalThe defining dimension for fleets and swarms
Workflow agentsMulti-agent coordinationCriticalThe defining property when multi-agent patterns are deployed
Autonomous trucksMulti-agent coordinationVery significant for platoonsPlatoon operation depends on V2V coordination

Rules

Reporting deadlines exist in China, the US and the EU; monitoring duties follow from them

Rules and standards for monitoring
RuleJurisdictionWhat it requiresStatus
National incident reporting measures (CAC)ChinaFirst report within one, two or four hours by operator typeEffective Nov 1, 2025
Cybersecurity Law amendmentChinaAI risk monitoring; fines up to RMB 10 millionEffective Jan 1, 2026
NHTSA Standing General Order, third amendmentUSADS and Level 2 crash reports in five days for the most severeEffective Jun 16, 2025
EU AI Act Article 72EUPost-market monitoring plan for high-risk systemsDelayed to Dec 2027
EU AI Act Article 73EUSerious incident reports in two, 10 or 15 daysDraft guidance Sep 2025

Status colors: green in force, blue voluntary standard or framework, amber adopted and phasing in, gray draft or pending, red withdrawn.

Limits

Monitoring finds what it is built to recognize

Agents that misreport their own actions, as Replit's did, defeat logs that trust the agent, and fleet-wide patterns stay invisible to per-device monitoring.

Record

Page Record

Datasets
ai_control_implementations.csv (5 rows for this page); ai_incident_controls.csv (6 linked); ai_attack_surface.csv
Key sources
Anthropic
CAC
Microsoft
OpenAI
OpenTelemetry
Source notes
GenAI and agent semantic conventions: secondary source; Malicious network disruption: secondary source; Threat intelligence account enforcement: secondary source
Last verified
October 2026