When a robot makes a mistake

Imagine a robot placing an object in the wrong location. It might be a minor inconvenience. In another setting, the same movement could block someone's path or damage a delicate component. Context determines what an error means.

“The robot made a mistake” is therefore the beginning of an investigation. What task had it received? What had it perceived? Which software version was running? How was human assistance defined? A sound responsibility framework considers these questions before an event. The examples here are illustrative scenarios, not reports of an actual accident.

First, return to a safe state

When something goes wrong, the immediate need is to protect people and prevent escalation. The response should be defined in advance for the system and its environment. Who can stop the task, how support is reached and who authorises a restart belong in ordinary operating arrangements.

Human oversight requires more than having someone nearby. That person needs to understand the system's state, access intervention controls and have enough time to make a decision. If the assigned role and actual working conditions do not support one another, oversight that exists on paper may be weak in practice.

Three questions of responsibility

The first is operational: who does what when a problem occurs? The second is technical: which team investigates, using which records? The third is legal: what obligations do the parties have in relation to harm? These questions are connected, but they are different assessments.

Manufacturers, software suppliers, integrators and operators may have distinct roles. Contractual allocation of work matters; it should not be assumed to remove every legal responsibility. A specific case depends on product characteristics, modifications, use and applicable law.

The EU AI Act framework addresses AI systems through use context and risk. Its requirements for high-risk systems include areas such as records, documentation, human oversight and robustness. A robot's use of AI does not automatically put every application in the same risk category. ^[1]

Why does the law also discuss software?

A robot's behaviour may change after delivery. Software updates, new tasks and hardware modifications can affect the system. It should be possible to trace who made a change and under which conditions it was validated.

The EU's 2024 Product Liability Directive expressly covers software and AI systems and addresses matters including access to evidence for people harmed by defective products. Transition to the new regime needs separate consideration of national implementation and the date a product was placed on the market. At this issue's September 2026 editorial cutoff, that transition is still under way. The new provisions should not be read as applying identically to every older product. ^[2]

The engineering lesson is the importance of being able to explain the past. Without a record of the configuration and task at the time, understanding an event becomes harder. Unlimited recording is unnecessary; meaningful traceability requires defined purposes, access and retention periods for the data that are needed.

Records should support learning

If an employee intervened, an investigation focused only on the last person to press a button would be incomplete. Was the display understandable? Was training adequate? Did the system communicate its expected behaviour? Did workload affect the intervention? Error analysis should identify connections that can reduce recurrence.

EU-OSHA recommends considering organisational and psychosocial effects alongside technical risks in advanced robotics. Its approach provides a useful framework for understanding safety as more than a property of a device. ^[3]

Employees who report faults or near misses can offer valuable experience. Whether they see what happens after a report also affects trust. If a problem recurs and nobody receives feedback, the recording process is failing to support organisational learning.

An answer society can understand

A hotel guest or a visitor to a public building need not understand the technical architecture. When a problem occurs, they want a person they can reach, an understandable explanation and a clear route forward. Robots entering shared spaces bring this communication responsibility with them.

Trust cannot rest on a claim of perfection. Honest operating limits, the ability to investigate errors and the authority to stop use when needed provide a stronger foundation. When a robot makes a mistake, responsibility must not disappear into a gap between organisations.