Machine Learning Failure Mode and Effects Analysis

Description

Machine Learning Failure Mode and Effects Analysis (ML FMEA) applies the process form of failure mode and effects analysis, a long-established worksheet method in which a team lists what could go wrong at each step of a process, rates each failure and acts on the worst first, to the pipeline that produces a machine-learning model rather than to the model alone. The team works through the pipeline one step at a time (in the 2025 reference version: collect data requests, collect data, ingest data, validate data, preprocess data, train, tune, analyse, deploy and validate the model, and analyse feedback from the field) and, prompted by guide words such as missing, incorrect or too late, records for each step how the resulting model could fail, the effect on the wider system, the pipeline causes, and the current practices that would prevent or catch the failure. Each row is rated for severity, occurrence and detection, and the product of the three, the risk priority number, ranks the rows so that recommended actions and owners go to the highest risks first; severity is taken from the system-level hazard analysis, because the harm a model failure can do depends on the system it sits in. The method's authors publish a pre-populated template listing typical failure modes, causes and mitigations for each step, which a team tailors to its own model and application. The result is a worksheet of ranked pipeline failure modes, their controls and assigned actions, kept up to date as the pipeline changes; it looks at failures one at a time in how the model is built and does not by itself analyse how the deployed system behaves.

Example Use Cases

Safety

An autonomous-trucking company's perception team fills in the template for its lane-detection model before each major retraining. One row under 'collect data' records that data gathered in a single season could leave the model unable to see faded or snow-covered markings; its severity comes from the vehicle-level hazard analysis, and it leads to an action, with an owner and a date, to constrain data requests by season and weather before release.

A medical-imaging company preparing a chest X-ray triage model for regulatory review uses the worksheet to show its assessors where the pipeline's controls sit. High-ranked rows include training labels taken from a single radiologist's reading, which could teach the model to miss small nodules, mitigated by double reading of a labelled sample, and online learning in deployment, mitigated by switching it off.

After a warehouse robot's object detector starts misclassifying a new kind of shrink-wrapped pallet, the vendor's safety engineer returns to the worksheet, finds that the 'analyse model feedback' step had no control for new packaging reaching customer sites, adds drift monitoring and a trigger for new data requests, and re-rates the affected rows.

Limitations

It considers one failure mode at a time, step by step, so it does not show how failures in different steps combine, or how a model that meets its specification can still contribute to an accident through its interaction with people and other parts of the system. Teams that need those combinations and interactions have to pair it with a system-level analysis.
Severity, occurrence and detection are judgements on ordinal scales, and there is little data on how often an ML pipeline failure occurs or how reliably a control catches it. The method's authors introduced ML-specific rating criteria in 2026 because traditional FMEA criteria proved ambiguous for ML development; the public template, last updated in April 2025, has no rating criteria, so teams must set and record their own.
It analyses how the model is built, not how it behaves in use: a complete worksheet shows that known pipeline failure modes have controls, not that the deployed model performs safely. Severity has to come from a system-level hazard analysis, so the method cannot stand alone, and its authors present it as one part of a safety management system and safety case.
The pre-populated template makes a quick start but invites teams to tick through the listed failure modes instead of looking for ones specific to their system. Its examples come mostly from autonomous-vehicle development, so teams in other domains, or with other kinds of model such as language models, must add their own failure modes; the 2026 revision added steps for problem definition and for building ground truth or reward signals.
Published evidence that the method finds failures other methods miss is limited. The 2025 paper that introduced it gave no applied example, and the 2026 follow-up reporting applications to autonomous vehicles and humanoid robots is led by the method's originators: four of its twelve authors, including the first author, wrote the 2025 paper, so no independent group has yet reported using it.

Resources

Research Papers

Introducing the ML FMEA

Paul Schmitt et al.•Apr 1, 2025

Software Packages

Machine Learning FMEA (ML FMEA) Template

Repository for the Machine Learning Failure Mode and Effects Analysis (ML FMEA) Template. The ML FMEA is detailed within the SAE World Congress 2025 publication, "Introducing the ML FMEA&quot...

Documentations

The ML FMEA: An Introduction (Torc Robotics publication page)

Related Techniques

Name	Description	Assurance Goals
System-Theoretic Process Analysis	System-Theoretic Process Analysis (STPA) is a hazard analysis method developed by Nancy Leveson at MIT that treats safety as a control problem: a loss can occur when parts of a system that have not failed interact unsafely, for example when a controller acts on a wrong belief about the process it controls. The analysis has four steps: define the losses to prevent and the system-level hazards that lead to them; model the system as a control structure of controllers, the control actions they issue and the feedback they receive; identify unsafe control actions; and write loss scenarios explaining how each could arise. An unsafe control action is one of four types (not providing an action causes a hazard, providing it causes a hazard, providing it too early, too late or in the wrong order, or stopping it too soon or applying it too long), and loss scenarios cover causes such as missing or wrong feedback, a flawed control algorithm, or a controller's process model (its internal picture of what it controls) that no longer matches reality. In a system with machine-learning components, a learned model appears in the control structure as a controller whose control algorithm and process model were learned from data, or as a feedback source, such as a perception model, whose errors mislead a controller; loss scenarios include training data that missed an operating condition, a shift from training to deployment conditions, and operators who over-rely on the model, and published adaptations also model data collection, training and evaluation as control loops, so that a scenario can trace an unsafe control action back to a development decision such as an incomplete reward function. The output is a traceable set of unsafe control actions, loss scenarios and the safety constraints and requirements derived from them, which the team uses to change the design and to decide what to test; it analyses a model of the system before or alongside testing, and it gives no probabilities.	Safety
Fault Tree Analysis	Fault tree analysis (FTA) is a deductive, top-down method that starts from one undesired system-level event, such as an automated braking function failing to brake for an obstacle, and works backwards to the combinations of lower-level events that could cause it. The analyst builds a tree of logic gates beneath that top event: an OR gate means any one of its inputs is enough to cause the event above it, an AND gate means all of its inputs must occur together, and the branches end in basic events such as a hardware failure, a human error or a software error. Solving the tree gives its minimal cut sets, the smallest combinations of basic events that produce the top event (a set with a single event is a single point of failure), and, when each basic event has a probability, the probability of the top event, which can be checked against a target or used to set a probability budget for each part. In a system with a machine-learning component, the model's errors enter the tree as basic events of their own, which unlike random hardware failures happen during normal operation at a rate that has to be measured on test or operational data, often depend on conditions such as weather or occlusion that can be added beneath it as conditioning events, and can be correlated from one input to the next. Working down from an acceptable top-event probability turns the model's share into a budget for its error rate, which becomes a performance requirement on the model; the tree shows where model errors matter in the system and how much error it can tolerate, but not why the model makes them.	Safety
STRIDE-AI Threat Modelling	STRIDE-AI is a threat-modelling method for machine-learning (ML) systems, proposed by Mauri and Damiani (2021, extended 2022), that lists, before any testing, the ways an attacker could compromise each asset of an ML pipeline and how serious each would be. It adapts Microsoft's STRIDE scheme, in which each of six threat types (spoofing, tampering, repudiation, information disclosure, denial of service and elevation of privilege) breaks one security property (authenticity, integrity, non-repudiation, confidentiality, availability and authorisation), and gives each property an ML-specific definition: confidentiality, for example, means that querying a model reveals nothing beyond its inputs and outputs. The analyst surveys the assets produced across the ML life cycle, such as raw, labelled and validation data, hyperparameters, trained and deployed models and the model architecture, and with each asset's owners runs a failure mode and effects analysis (FMEA, a structured review of how a component could fail and with what effect), answering set questions about how the asset could fail, be misused or be abused. Each failure mode is mapped to the properties it would break and so to STRIDE threats, using the method's published tables of asset types, properties and known attacks; the analyst then works with developers to draw threat trees showing the conditions under which each threat could be realised, and rates each threat's priority (the worked example uses DREAD, which scores five factors from 1 to 10). The output is a threat model, a table of assets, threatened properties, threats and priorities with their threat trees: the method attacks and measures nothing, and choosing and testing controls for the threats it finds is left to other techniques.	Security