data requirements
test scenarios
Requires predefined test cases or scenarios
14 techniques
| Goals | Models | Data Types | Description | |||
|---|---|---|---|---|---|---|
| Safety Envelope Testing | Testing | Architecture/model Agnostic Requirements/black Box | Any | Safety envelope testing checks how an AI system performs at and beyond the edges of the conditions it was designed for,... | ||
| Agent Goal Misalignment Testing | Testing | Architecture/model Agnostic Architecture/neural Networks/transformer/llm +3 | Text | Agent goal misalignment testing checks whether an AI agent takes unsanctioned means to reach a goal it has been given.... | ||
| Multi-Agent System Testing | Testing | Architecture/model Agnostic Requirements/black Box | Any | Multi-agent system testing examines systems in which several AI agents act on shared tasks or resources, to find... | ||
| Reward Hacking Detection | Testing | Architecture/model Agnostic Architecture/neural Networks/transformer/llm +2 | Any | Reward hacking is when an agent trained to maximise a proxy reward (a score that only approximates what its designers... | ||
| Scenario-Based Simulation Testing | Testing | Architecture/model Agnostic Requirements/black Box | Any | Scenario-based simulation testing checks an automated system, most often an automated driving system, by running it... | ||
| Dangerous Capability Evaluation | Testing | Architecture/neural Networks/transformer/llm Paradigm/generative +1 | Text | Dangerous capability evaluation measures whether a frontier AI model can carry out tasks that could enable severe harm,... | ||
| Linear Bound Propagation Verification | Algorithmic | Architecture/neural Networks Requirements/white Box | Image Tabular | Linear bound propagation verification proves that a trained neural network meets a stated property for every input in a... | ||
| SMT-Based Neural Network Verification | Algorithmic | Architecture/neural Networks Architecture/neural Networks/feedforward +1 | Tabular | SMT-based neural network verification decides exactly whether any input in a defined region can make a trained neural... | ||
| Metamorphic Testing | Testing | Architecture/model Agnostic Requirements/black Box | Any | Metamorphic testing checks a model or AI system without needing the correct output for each test input, by checking that... | ||
| Appropriate Reliance Measurement | Metric | Architecture/model Agnostic Requirements/black Box | Any | Appropriate reliance measurement tests whether people who receive an AI system's advice follow it when it is right and... | ||
| Explanation Satisfaction Scale | Metric | Architecture/model Agnostic Requirements/black Box | Any | The Explanation Satisfaction Scale is a short questionnaire on which people who have used an AI system's explanations... | ||
| Forward Simulation Test | Testing | Architecture/model Agnostic Requirements/black Box | Any | A forward simulation test checks whether people who have studied a model's explanations can predict the model's output... | ||
| Think-Aloud Study of Explanation Use | Testing | Architecture/model Agnostic Requirements/black Box | Any | A think-aloud study of explanation use asks a small number of participants to say what they are thinking while they use... | ||
| Bias Benchmark for Question Answering | Testing | Architecture/neural Networks/transformer/llm Requirements/black Box | Text | The Bias Benchmark for Question Answering (BBQ) is a fixed set of about 58,500 multiple-choice reading questions that... |
Rows per page
Page 1 of 1