applicable models
probabilistic output
Model must provide probability distributions as output
10 techniques
| Goals | Models | Data Types | Description | |||
|---|---|---|---|---|---|---|
| Monte Carlo Dropout | Algorithmic | Architecture/neural Networks Paradigm/probabilistic +4 | Any | Monte Carlo dropout estimates how uncertain a neural network is about each prediction. Dropout randomly switches off... | ||
| Out-of-Distribution Detector for Neural Networks | Algorithmic | Architecture/neural Networks Paradigm/discriminative +3 | Image | ODIN (Out-of-Distribution Detector for Neural Networks) flags inputs that look unlike the data a neural network... | ||
| Temperature Scaling | Algorithmic | Architecture/neural Networks Paradigm/discriminative +4 | Any | Temperature scaling is a correction applied after training to a classifier that is more confident than it is accurate, a... | ||
| Area Under Precision-Recall Curve | Metric | Architecture/model Agnostic Paradigm/supervised +2 | Any | Area Under Precision-Recall Curve (AUPRC) summarises how well a classifier finds the cases that matter when those cases... | ||
| Confidence Thresholding | Algorithmic | Architecture/model Agnostic Requirements/black Box +1 | Any | Confidence thresholding uses a model score to decide whether a prediction should proceed automatically or receive... | ||
| Reject Option Classification | Algorithmic | Architecture/model Agnostic Paradigm/supervised +2 | Any | Reject Option Classification is a post-processing method, proposed by Kamiran and colleagues (2012), that changes some... | ||
| Calibration with Equality of Opportunity | Algorithmic | Architecture/model Agnostic Paradigm/supervised +2 | Any | Calibration with equality of opportunity is a post-processing step that adjusts a classifier's scores so that two groups... | ||
| Canary Exposure Testing | Testing | Architecture/neural Networks Paradigm/generative +1 | Text | Canary exposure testing measures how strongly a training procedure makes a generative sequence model, such as a language... | ||
| Reliability Diagram with Expected Calibration Error | Metric | Architecture/model Agnostic Requirements/black Box +1 | Any | A reliability diagram with expected calibration error (ECE) measures whether a classifier's stated confidence matches... | ||
| Continuous Ranked Probability Score | Metric | Architecture/model Agnostic Requirements/black Box +1 | Time-series Tabular | The continuous ranked probability score (CRPS) measures the quality of a probabilistic forecast of a numerical quantity,... |
Rows per page
Page 1 of 1