Explainability
Human Evaluation
Techniques that test explanations with the people who use them, through studies, questionnaires or measures of their decisions
3 subcategories • 4 techniques
Decision Study
Measures whether explanations improve people's decisions with the model (e.g., appropriate-reliance measurement)
Rating Scale
A validated questionnaire on how people rate an explanation (e.g., the Explanation Satisfaction Scale)
Understanding Test
Tests whether people can predict or describe the model's behaviour from its explanations (e.g., forward simulation tests, think-aloud studies)
Techniques
4 techniques
| Goals | Models | Data Types | Description | |||
|---|---|---|---|---|---|---|
| Appropriate Reliance Measurement | Metric | Architecture/model Agnostic Requirements/black Box | Any | Appropriate reliance measurement tests whether people who receive an AI system's advice follow it when it is right and... | ||
| Explanation Satisfaction Scale | Metric | Architecture/model Agnostic Requirements/black Box | Any | The Explanation Satisfaction Scale is a short questionnaire on which people who have used an AI system's explanations... | ||
| Forward Simulation Test | Testing | Architecture/model Agnostic Requirements/black Box | Any | A forward simulation test checks whether people who have studied a model's explanations can predict the model's output... | ||
| Think-Aloud Study of Explanation Use | Testing | Architecture/model Agnostic Requirements/black Box | Any | A think-aloud study of explanation use asks a small number of participants to say what they are thinking while they use... |
Rows per page
Page 1 of 1