expertise needed
human factors
Requires knowledge of how people oversee, rely on and interact with AI systems: designing human review workflows, managing reviewer workload and fatigue, and communicating model confidence and limits to users.
8 techniques
| Goals | Models | Data Types | Description | |||
|---|---|---|---|---|---|---|
| Human-in-the-Loop Safeguards | Process | Architecture/model Agnostic Requirements/black Box | Any | Human-in-the-loop safeguards hold an AI system's output at a checkpoint until a trained person has reviewed it.... | ||
| Confidence Thresholding | Algorithmic | Architecture/model Agnostic Requirements/black Box +1 | Any | Confidence thresholding uses a model score to decide whether a prediction should proceed automatically or receive... | ||
| API Usage Pattern Monitoring | Algorithmic | Architecture/model Agnostic Requirements/black Box | Any | API usage pattern monitoring examines how each account uses a deployed model's API, to catch misuse that no single... | ||
| Out-of-Domain Detection | Algorithmic | Architecture/model Agnostic Architecture/neural Networks/transformer/llm +2 | Text | Out-of-domain detection checks whether a user's query falls inside the topics and tasks a conversational AI system is... | ||
| Appropriate Reliance Measurement | Metric | Architecture/model Agnostic Requirements/black Box | Any | Appropriate reliance measurement tests whether people who receive an AI system's advice follow it when it is right and... | ||
| Explanation Satisfaction Scale | Metric | Architecture/model Agnostic Requirements/black Box | Any | The Explanation Satisfaction Scale is a short questionnaire on which people who have used an AI system's explanations... | ||
| Forward Simulation Test | Testing | Architecture/model Agnostic Requirements/black Box | Any | A forward simulation test checks whether people who have studied a model's explanations can predict the model's output... | ||
| Think-Aloud Study of Explanation Use | Testing | Architecture/model Agnostic Requirements/black Box | Any | A think-aloud study of explanation use asks a small number of participants to say what they are thinking while they use... |
Rows per page
Page 1 of 1