STRIDE-AI Threat Modelling
Description
STRIDE-AI is a threat-modelling method for machine-learning (ML) systems, proposed by Mauri and Damiani (2021, extended 2022), that lists, before any testing, the ways an attacker could compromise each asset of an ML pipeline and how serious each would be. It adapts Microsoft's STRIDE scheme, in which each of six threat types (spoofing, tampering, repudiation, information disclosure, denial of service and elevation of privilege) breaks one security property (authenticity, integrity, non-repudiation, confidentiality, availability and authorisation), and gives each property an ML-specific definition: confidentiality, for example, means that querying a model reveals nothing beyond its inputs and outputs. The analyst surveys the assets produced across the ML life cycle, such as raw, labelled and validation data, hyperparameters, trained and deployed models and the model architecture, and with each asset's owners runs a failure mode and effects analysis (FMEA, a structured review of how a component could fail and with what effect), answering set questions about how the asset could fail, be misused or be abused. Each failure mode is mapped to the properties it would break and so to STRIDE threats, using the method's published tables of asset types, properties and known attacks; the analyst then works with developers to draw threat trees showing the conditions under which each threat could be realised, and rates each threat's priority (the worked example uses DREAD, which scores five factors from 1 to 10). The output is a threat model, a table of assets, threatened properties, threats and priorities with their threat trees: the method attacks and measures nothing, and choosing and testing controls for the threats it finds is left to other techniques.
Example Use Cases
Security
The security architect for a solar-farm operator's fault-prediction service, which trains on sensor readings streamed from plant data loggers to an external ML platform, maps each asset to its threatened properties. The threat trees show that the loggers' signatures stop injected readings but not label flipping, because labels are added later on the operator's own platform, so that threat is ranked and passed to the platform team.
Before buying a third-party chest X-ray triage model, a hospital's information-security team runs STRIDE-AI over the vendor's pipeline documentation: training data, the supplied model file, the inference service and the results sent to radiologists. Threats such as tampering with the model file and disclosure of training images through model outputs set the scope of the security tests and contract terms the hospital asks the vendor to meet.
A bank's model-security lead repeats the analysis when a card-fraud model moves from monthly retraining to online learning on live transactions. The transaction stream becomes a training-data asset that anyone making transactions can influence, so tampering with it to drift the model is added as a threat, rated and entered in the bank's risk register.
Limitations
- A finished threat model is not evidence that no other threats exist. Its coverage depends on the asset survey and on what the analysts know; the authors state that ML vulnerabilities cannot be fully enumerated in one pass and that failure modes must be updated as the system and known attacks change, and a 2006 Microsoft account of STRIDE notes there is no well-established way to measure a threat model's quality. Claims should state what was in scope and when the analysis was last revised.
- STRIDE-AI identifies and ranks threats but does not choose or test controls. Both papers' abstracts say it guides control selection, but neither gives a procedure: the 2021 paper puts control selection outside its scope and the 2022 paper leaves it to future work, noting that no security control framework designed for ML was available. A claim that the threats are mitigated needs separate evidence, such as attack tests against each control and a control catalogue taken from another source.
- Priorities are expert judgements. The worked example uses DREAD scores, which the authors note can give inconsistent results between raters and are no longer recommended as the only basis for ranking. Two teams may rank the same threats differently, so claims that rest on the ranking are stronger when several people score independently or a published severity scale such as a bug bar is used.
- The mapping from failure modes to properties and threats is done by hand for every asset, and needs asset owners, developers and a security analyst together. Later work describes it as lacking automation, which limits how far it scales to large pipelines; each significant change, such as moving a model to online learning, means repeating part of the analysis.
- The published catalogue of assets, failure modes and known attacks was built in 2021-22 around the life cycle of a conventionally trained model, from requirements to maintenance. It does not list assets specific to language-model applications and agents, such as system prompts, retrieval corpora or connected tools, so analysts of those systems must add the assets and their threats themselves, without guidance from the method.