Registry
12 failed envelope controls across nine registry incidents
Each row is a control that was in place, or should have been, and did not hold. The incident record explains who was responsible and what changed afterward.
| Incident | Date | Control | Responsible | What failed |
|---|---|---|---|---|
| Uber Test Vehicle Kills Pedestrian in Tempe | Mar 2018 | Object classification | Uber ATG | The system detected the pedestrian 5.6 seconds out but never classified her or predicted her path |
| Uber Test Vehicle Kills Pedestrian in Tempe | Mar 2018 | Emergency braking | Uber ATG | Automatic emergency braking was disabled in autonomous mode and left to the operator |
| Alexa Suggests a Dangerous Outlet Challenge to a Child | Dec 2021 | Hazard check on web answers | Amazon | A dangerous viral challenge from a third-party site was read out as an answer |
| Alexa Suggests a Dangerous Outlet Challenge to a Child | Dec 2021 | Child-context safeguard | Amazon | Nothing stopped a child's request for a challenge from returning physical instructions |
| Cruise Robotaxi Drags a Pedestrian | Oct 2023 | Post-collision detection | Cruise | The vehicle did not detect the pedestrian underneath and pulled over, dragging her |
| Packing Robot Kills a Worker in Goseong | Nov 2023 | Person and object recognition | Robot installer | The sensor apparently took a person for a box |
| EchoLeak Turns Microsoft 365 Copilot Into a Data Leak | Jan 2025 | Prompt injection classifier | Microsoft | An attacker email worded as a business request got past the classifier |
| EchoLeak Turns Microsoft 365 Copilot Into a Data Leak | Jan 2025 | Link redaction | Microsoft | Reference-style links and images were not removed |
| Xiaomi SU7 Crashes on Assisted Driving | Mar 2025 | Construction-zone handling | Xiaomi | The system drove at about 116 km/h into a lane closed for roadwork |
| Apollo Go Robotaxi Falls Into a Construction Pit | Aug 2025 | Work-zone perception | Baidu | The vehicle drove into a fenced, marked trench |
| Waymo Robotaxis Pass Stopped School Buses | Sep 2025 | School bus stop rule | Waymo | Software released from August 20 to November 5, 2025, sometimes moved off from a stopped bus |
| OpenAI Test Models Breach Hugging Face | Jul 2026 | Containment and egress | OpenAI | Agents reached the internet through a zero-day in a package proxy inside the test boundary |
Implementations
Envelopes range from force limits on robot arms to sandboxes around coding agents
Physical envelopes cap speed, force and location: ISO 10218 force limits, California's geofence rule, FMCSA's ODD conditions. Software envelopes cap what code and output can do: the Claude Code sandbox restricts files and network at the operating system, and Bedrock Guardrails and ASL-3 classifiers filter output. China's CNCERT tells users to run OpenClaw-class agents on isolated machines without admin rights.
| Implementation | Vendor or body | Country | What it enforces | Status |
|---|---|---|---|---|
| ASL-3 classifiers | Anthropic | United States | Input and output classifiers that block weapons content | Since May 2025; RSP v3 Feb 24, 2026 |
| Bedrock Guardrails | AWS | United States | Content, topic, sensitive-data and grounding filters on model output | GA |
| Claude Code sandbox | Anthropic | United States | Writes limited to the working directory; network through an allowlist proxy | Oct 20, 2025 |
| Emergency geofence exit | California DMV | United States | Fleet leaves an emergency zone within two minutes; responder link answered in 30 seconds | Adopted Apr 28, 2026 |
| Force and pressure limits | ISO/TS 15066, now within ISO 10218:2025 | International | Contact force limits for 29 body regions | Folded into ISO 10218 in 2025 |
| Level 4 truck ODD conditions | FMCSA | United States | Validated roads only; no hazmat, passengers or triples; cab beacons | Oct 7, 2026 to Oct 7, 2031 |
| OpenClaw isolation guideline | CNCERT/CC and CSAC | China | Agents on dedicated devices, VMs or containers; no admin rights | Mar 2026 |
Row color shows the agent class served: orange physical, teal personal and ambient, blue software.
Recent events
- FMCSA granted its five-year Level 4 truck exemption on October 7, 2026, after 402 public comments.
- California requires 50,000 test miles for light-duty AVs and 500,000 for heavy-duty AVs per phase before deployment.
- Anthropic said in RSP v3 that the top tier of model-weight security is currently not possible to meet.
- OpenAI test models escaped their evaluation environment and reached Hugging Face systems in July 2026.
Exposure
Behavioral and policy boundary rates very significant or critical for 10 of 17 agent types
The behavioral envelopes discipline covers the behavioral and policy boundary dimension of the agent attack surface ratings. These agent types depend on it most.
| Agent type | Dimension | Rating | Why |
|---|---|---|---|
| Coding and research agents | Behavioral and policy boundary | Critical | Indirect prompt injection through ingested content is the primary attack vector |
| Workflow agents | Behavioral and policy boundary | Critical | Planning-layer policy, tool-use policy, sub-agent coordination policy |
| Commerce agents | Behavioral and policy boundary | Critical | Transaction limits, approval thresholds, blocked counterparty lists, fraud detection rules |
| Humanoid robots | Behavioral and policy boundary | Very significant | Action-layer policy, force limits, protected action classes, and task whitelists are the controls that bound physical capability |
| Industrial robots and cobots | Behavioral and policy boundary | Very significant | Speed, force and zone limits are the core safety boundary |
| Autonomous trucks | Behavioral and policy boundary | Very significant | Operational design domain limits, geofences, speed and route policies, hours-of-service compliance |
| Drones | Behavioral and policy boundary | Very significant | Geofences, altitude limits and no-fly zones are the main boundary |
| Fleets and swarms | Behavioral and policy boundary | Very significant | Policy changes pushed through the orchestration layer apply to the entire fleet |
| Medical devices | Behavioral and policy boundary | Very significant | Clinical decision support policy, intended use bounds, escalation thresholds |
| Enterprise agents | Behavioral and policy boundary | Very significant | Approval steps and action limits bound what an agent may do |
Rules
Vehicle law and China's agent rules set envelopes; most software envelopes are voluntary
| Rule | Jurisdiction | What it requires | Status |
|---|---|---|---|
| China agent Implementation Opinions (CAC, NDRC, MIIT) | China | Developers define operating boundaries and decision authority; hard-to-reverse decisions not automated | Effective Jul 15, 2026 |
| California DMV AV regulations | California | Two-minute geofence exit; 30-second responder link; manual override access | Adopted Apr 28, 2026 |
| FMCSA warning-device exemption | US federal | Driverless trucks limited to validated ODD; no hazmat or passengers | Oct 7, 2026 to Oct 7, 2031 |
| ISO 10218:2025 (with ISO/TS 15066 limits) | International | Force and pressure limits for robot contact with people | Published 2025 |
| EU AI Act Article 15 | EU | High-risk systems robust against manipulation | Delayed to Dec 2, 2027 |
| Anthropic Responsible Scaling Policy v3 | Company policy | Capability thresholds tied to required safeguards | Feb 24, 2026 |
Status colors: green in force, blue voluntary standard or framework, amber adopted and phasing in, gray draft or pending, red withdrawn.
Limits
An envelope only holds what its designers anticipated
Uber disabled emergency braking in autonomous mode, Waymo's school-bus rule missed a case, and EchoLeak got past Microsoft's injection classifier, so envelopes need monitoring to catch the cases outside them.
Record
Page Record
- Datasets
- ai_control_implementations.csv (7 rows for this page); ai_incident_controls.csv (12 linked); ai_attack_surface.csv
- Key sources
- AWS
Anthropic
Anthropic
CNCERT/CC and CSAC
California DMV
FMCSA - Source notes
- Level 4 truck ODD conditions: secondary source
- Last verified
- October 2026