137AI > AI Risks & Management > Data Risks from AI

Data Risks from AI

Every deployed AI agent sits inside a closed loop. Sensors capture data from the agent's environment. Telemetry flows over networks to a cloud or fleet backend. The data informs model training and operational decisions. Updated models, policies, and configurations flow back to the deployed agents over the air. The agents act on the new instructions, producing more sensor data that feeds the next training cycle. Each stage of this loop is a target, and corruption introduced at any stage propagates forward. What separates data risks from conventional cybersecurity is that the loop closes: a successful attack on training data does not merely expose information, it shapes the behavior of every agent that subsequently runs the corrupted model.

Telemetry Capture Integrity at the Source

The first link in the loop is the sensor itself. Cameras can be blinded, lidar can be confused with adversarial reflectors, GNSS can be spoofed, microphones can be flooded with ultrasonic injection that humans cannot hear but the microphone records as legitimate audio. A sensor returning falsified data does not look broken to the agent. It looks normal. Downstream systems that trust the sensor's output build their decisions on a foundation that is already corrupted. The risk applies across every agent category that depends on environmental sensing, from the robotaxi reading the road to the smart home assistant reading the room to the ambient sensor network reading the city. Telemetry Capture Integrity covers the spoofing and tampering techniques observed in research and in the field, the sensor redundancy and cross-validation approaches that bound them, and the cryptographic attestation patterns being developed to prove a sensor's output came from the sensor at the time it claims.

Transit Security Between Edge and Cloud

Telemetry leaves the agent and travels to fleet backends, training pipelines, and decision-support systems. The transit path includes edge gateways, cellular and WiFi links, content delivery networks, and the cloud ingress points where the data lands for processing. Attackers who reach any segment of the path can intercept, modify, replay, or delete the data flowing through it. The attacks are not new in form, but they take on new consequence when the data being moved is the input to a model that will be deployed back to a fleet. A man-in-the-middle who modifies a percentage of telemetry samples in transit may not be detected by conventional network monitoring, but the corruption reaches the next model anyway. Transit Security covers the attack surface across edge gateways and cloud ingress, the end-to-end encryption and message authentication that bound interception and modification, and the provenance tracking that detects when telemetry has been altered between source and destination.

Training Data Poisoning

The model trained on poisoned data is itself poisoned. The poisoning need not be obvious. Small perturbations introduced into a small fraction of training samples can produce a model that behaves correctly on most inputs and incorrectly on a narrow set of attacker-chosen inputs. The poisoned behavior is invisible to validation that exercises the model on a representative test set, because the trigger conditions are not in the test set. Poisoning can be introduced upstream of the operator: an open dataset gets a few samples added, a contractor processing labeling adds a few miscategorized examples, a contributor to a pretrained foundation model embeds a small bias that survives fine-tuning. Once the poisoned model is deployed back to the fleet, the attacker's trigger conditions become exploitable across every agent running that model. Training Data Poisoning covers the attack patterns from research and observed practice, the dataset provenance and curation controls that bound them, and the detection mechanisms developed to catch poisoning before deployment.

Model Update Integrity

Models reach deployed agents through update pipelines. A compromise of the pipeline pushes the attacker's model to every agent the operator manages. The update mechanism is the most concentrated attack target in the entire loop because a single successful compromise reaches the entire fleet simultaneously. Adjacent to model updates are policy updates, configuration updates, firmware updates, and the third-party software libraries the agent depends on. Each is a candidate for a supply-chain-of-updates attack in which the attacker reaches the agent not directly but through the pipeline that ships software to it. Model Update Integrity covers the signing, attestation, staged rollout, and rollback mechanisms that bound the risk, and the failures and near-misses that have demonstrated the consequences when those controls are absent or weak.

Fleet-Scale Coordinated Attacks

Most attacks on individual agents are limited in consequence to the individual agent. Attacks that reach the orchestration plane controlling many agents at once produce events with no per-agent equivalent. A coordinated misbehavior across thousands of robotaxis at the same moment, a synchronized payload release across a delivery robot fleet, a fleet-wide humanoid action triggered from a single compromised command channel — each is a category of incident that scales with the deployed fleet size rather than with attacker effort. The orchestration plane includes fleet management APIs, dispatch systems, remote supervision channels, and the OTA infrastructure described above. Fleet-Scale Coordinated Attacks covers the patterns specific to fleet-level compromise, the segmentation and authority partitioning that bound the blast radius, and the detection mechanisms that catch coordinated anomaly before it propagates.

Surveillance Material Harvesting

Agents with cameras, microphones, and behavioral sensors accumulate intimate material as a byproduct of normal operation. Robotaxi cabin recordings, smart home audio, smart glasses video, cabin AI driver monitoring, ambient sensor data from public spaces — each is a corpus that has value beyond the safety and operational purposes the agent was deployed for. An attacker who reaches the storage or transit of this material gains leverage that no conventional data breach produces. Blackmail material does not need to be deployed at scale to be valuable. A handful of intimate recordings is enough to coerce a target individually, and a fleet-scale compromise of ambient capture produces enough material to coerce a population. Surveillance Material Harvesting covers the collection surfaces that produce blackmail-grade material, the storage and retention practices that determine how much accumulates, and the access control and encryption mechanisms that bound who can reach it after collection.

Supply-Chain-of-Updates Attacks

An agent operator may sign and stage their own updates with discipline, and still ship a compromised model if upstream dependencies are corrupted. The pretrained foundation model the operator fine-tunes might be poisoned. The open-source library the agent links might have an injected backdoor. The labeling contractor's pipeline might be compromised. The hardware on which the agent runs might have a chip-level supply chain issue. Each is an upstream vector that traditional perimeter security does not address because the compromise enters the operator's environment through trusted channels. Supply-Chain-of-Updates Attacks covers the upstream vectors, the bill-of-materials and provenance tracking practices that surface them, and the verification and reproducibility mechanisms being developed to give operators confidence in what they are shipping.

OT/ICS Telemetry and Digital Twin Deception

AI is increasingly embedded in operational technology environments: grid forecasting, predictive maintenance, traffic optimization, water treatment, manufacturing process control. The telemetry that feeds these AI systems comes from industrial sensors, and the decisions the AI makes feed back into industrial control. An attacker who tampers with the telemetry produces an operational decision the legitimate operator would never authorize. A digital twin built on corrupted telemetry shows a plant operator a synthetic state that diverges from physical reality, leading the operator to make decisions based on a false picture. The risk is distinct from traditional ICS attacks because the corruption rides the AI layer rather than the control layer, and the existing ICS security discipline does not yet fully address it. OT/ICS Telemetry & Digital Twin Deception covers the attack surface specific to AI-mediated industrial environments, the cross-validation and physical-model-consistency checks that bound it, and the governance vacuum at the intersection of AI regulation and industrial control regulation.

Related Coverage

Risks & Management | Human Risks | Risk Management | Agents