Evidence
Courts, recalls and probes all moved against driver assistance in 2025
| Date | Case | Country | What happened | Source |
|---|---|---|---|---|
| Mar 29, 2025 | Xiaomi SU7 NOA crash | China | Car hit a construction barrier at about 97 km/h; three students died | Source |
| Aug 1, 2025 | Benavides v. Tesla | United States | Miami federal jury found Tesla partly liable for a 2019 Autopilot death; about $243 million owed | Source |
| Oct 9, 2025 | NHTSA FSD preliminary evaluation | United States | 2.88 million vehicles after 58 violation reports, 14 crashes and 23 injuries | Source |
| Dec 2025 | Waymo school-bus recall | United States | Software recall of 3,067 robotaxis after school-bus passings in Austin | Source |
| Mar 31, 2026 | Apollo Go Wuhan stall | China | Robotaxis froze across Wuhan, stranding passengers on elevated roads | Source |
Registry
Six registry incidents involve automated driving safety
Registry incidents meet a stricter bar than the cases above: public sources name the developer, the failed control and who found it.
Exposure
All 17 agent types rate very significant or critical on behavioral and policy boundary or perception and sensors
From the attack surface ratings, the agent types where this risk has the widest path in.
| Agent type | Dimension | Rating |
|---|---|---|
| Coding and research agents | Behavioral and policy boundary | Critical |
| Workflow agents | Behavioral and policy boundary | Critical |
| Commerce agents | Behavioral and policy boundary | Critical |
| Robotaxis and autonomous vehicles | Perception and sensors | Very significant |
| Humanoid robots | Perception and sensors | Very significant |
| Industrial robots and cobots | Perception and sensors | Very significant |
| Autonomous trucks | Perception and sensors | Very significant |
| Drones | Perception and sensors | Very significant |
| Fleets and swarms | Behavioral and policy boundary | Very significant |
| Phones and tablets | Perception and sensors | Very significant |
| Wearables | Perception and sensors | Very significant |
| Vehicle cabin AI | Perception and sensors | Very significant |
| Smart home assistants | Perception and sensors | Very significant |
| Medical devices | Perception and sensors | Very significant |
| Public infrastructure | Perception and sensors | Very significant |
| Ambient sensors | Perception and sensors | Very significant |
| Enterprise agents | Behavioral and policy boundary | Very significant |
Controls that bound it
- Behavioral Envelopes: hard limits on action enforced outside the model
- Human Oversight: human authority to approve, intervene and stop
- Monitoring and Anomaly Detection: seeing failures and misuse while they happen
Rules
China set a mandatory Level 2 standard; the US relies on crash reporting and recalls
| Rule | Jurisdiction | What it requires | Status |
|---|---|---|---|
| GB 47955-2026 combined driving assistance | China | Three Level 2 tiers, driver monitoring, data logging | Effective Jan 1, 2027 |
| MIIT driver-assist directives | China | No "autonomous driving" marketing; OTA and beta tests need approval | Apr 16, 2025 |
| NHTSA Standing General Order, third amendment | US | ADS and Level 2 crash reporting | Effective Jun 16, 2025 |
| UN Regulation 171 | UNECE / EU | Level 2 performance and driver monitoring | Adopted 2024 |
Status colors: green in force, blue voluntary standard or framework, amber adopted and phasing in, gray draft or pending, red withdrawn.
Record
Page Record
- Risk group
- Human Risks
- Datasets
- ai_risk_evidence.csv (5 rows); registry incidents (6 linked); ai_attack_surface.csv
- Key sources
- Xiaomi SU7 NOA crash
Benavides v. Tesla
NHTSA FSD preliminary evaluation
Waymo school-bus recall
Apollo Go Wuhan stall - Last verified
- October 2026