Transparency
Confidence and Limits
Techniques that show users how far to trust each output and when a request falls outside what the system can handle.
10 techniques
10 techniques
| Goals | Models | Data Types | Description | |||
|---|---|---|---|---|---|---|
| Conformal Prediction | Algorithmic | Architecture/model Agnostic Requirements/black Box | Any | Conformal prediction wraps any trained model so that, instead of a single answer, it returns a set of plausible answers... | ||
| Platt Scaling | Algorithmic | Architecture/model Agnostic Paradigm/supervised +1 | Any | Platt scaling turns a classifier's raw scores into calibrated probabilities, meaning that among the cases given a... | ||
| Temperature Scaling | Algorithmic | Architecture/neural Networks Paradigm/discriminative +4 | Any | Temperature scaling is a correction applied after training to a classifier that is more confident than it is accurate, a... | ||
| Deep Ensembles | Algorithmic | Architecture/neural Networks Paradigm/parametric +2 | Any | Deep ensembles estimate how uncertain a neural network's predictions are by training several copies of the same network... | ||
| Bootstrap Ensembles | Algorithmic | Architecture/model Agnostic Paradigm/supervised +1 | Any | Bootstrap ensembles estimate how much a model's predictions depend on the particular sample of data it was trained on.... | ||
| Jackknife Resampling | Algorithmic | Architecture/model Agnostic Paradigm/supervised +1 | Any | Jackknife resampling, also called leave-one-out resampling, shows how much a result computed from a dataset depends on... | ||
| Confidence Thresholding | Algorithmic | Architecture/model Agnostic Requirements/black Box +1 | Any | Confidence thresholding uses a model score to decide whether a prediction should proceed automatically or receive... | ||
| Knowledge Boundary Probing | Algorithmic | Architecture/neural Networks/transformer/llm Requirements/black Box | Text | Knowledge boundary probing maps what a language model can and cannot reliably answer. A probe set is built from... | ||
| Out-of-Domain Detection | Algorithmic | Architecture/model Agnostic Architecture/neural Networks/transformer/llm +2 | Text | Out-of-domain detection checks whether a user's query falls inside the topics and tasks a conversational AI system is... | ||
| Isotonic Regression Calibration | Algorithmic | Architecture/model Agnostic Paradigm/supervised +1 | Any | Isotonic regression calibration turns a classifier's scores into calibrated probabilities by fitting a step-shaped... |
Rows per page
Page 1 of 1