Reliability
37 techniques
Building AI systems that perform consistently and predictably.
Browse by subcategory
All techniques
37 techniques
| Goals | Models | Data Types | Description | |||
|---|---|---|---|---|---|---|
| Monte Carlo Dropout | Algorithmic | Architecture/neural Networks Paradigm/probabilistic +4 | Any | Monte Carlo dropout estimates how uncertain a neural network is about each prediction. Dropout randomly switches off... | ||
| Out-of-Distribution Detector for Neural Networks | Algorithmic | Architecture/neural Networks Paradigm/discriminative +3 | Image | ODIN (Out-of-Distribution Detector for Neural Networks) flags inputs that look unlike the data a neural network... | ||
| Permutation Tests | Algorithmic | Architecture/model Agnostic Requirements/black Box | Any | A permutation test asks how likely an observed result would be if there were no real relationship in the data. The... | ||
| Quantile Regression | Algorithmic | Architecture/linear Models/regression Architecture/neural Networks +4 | Tabular | Quantile regression predicts chosen percentiles of an outcome, such as the 10th, 50th and 90th, rather than only its... | ||
| Conformal Prediction | Algorithmic | Architecture/model Agnostic Requirements/black Box | Any | Conformal prediction wraps any trained model so that, instead of a single answer, it returns a set of plausible answers... | ||
| Platt Scaling | Algorithmic | Architecture/model Agnostic Paradigm/supervised +1 | Any | Platt scaling turns a classifier's raw scores into calibrated probabilities, meaning that among the cases given a... | ||
| Temperature Scaling | Algorithmic | Architecture/neural Networks Paradigm/discriminative +4 | Any | Temperature scaling is a correction applied after training to a classifier that is more confident than it is accurate, a... | ||
| Deep Ensembles | Algorithmic | Architecture/neural Networks Paradigm/parametric +2 | Any | Deep ensembles estimate how uncertain a neural network's predictions are by training several copies of the same network... | ||
| Bootstrap Ensembles | Algorithmic | Architecture/model Agnostic Paradigm/supervised +1 | Any | Bootstrap ensembles estimate how much a model's predictions depend on the particular sample of data it was trained on.... | ||
| Jackknife Resampling | Algorithmic | Architecture/model Agnostic Paradigm/supervised +1 | Any | Jackknife resampling, also called leave-one-out resampling, shows how much a result computed from a dataset depends on... | ||
| Cross-validation | Testing | Architecture/model Agnostic Paradigm/supervised +1 | Any | Cross-validation estimates how well a model will perform on data it has not seen, and how much that estimate depends on... | ||
| Area Under Precision-Recall Curve | Metric | Architecture/model Agnostic Paradigm/supervised +2 | Any | Area Under Precision-Recall Curve (AUPRC) summarises how well a classifier finds the cases that matter when those cases... | ||
| Safety Envelope Testing | Testing | Architecture/model Agnostic Requirements/black Box | Any | Safety envelope testing checks how an AI system performs at and beyond the edges of the conditions it was designed for,... | ||
| Runtime Monitoring and Circuit Breakers | Algorithmic | Architecture/model Agnostic Requirements/black Box | Any | Runtime monitoring and circuit breakers observe a deployed AI service and restrict model calls when configured signals... | ||
| Monotonicity Constraints | Algorithmic | Architecture/neural Networks Architecture/probabilistic/gaussian Processes +3 | Tabular | Monotonicity constraints require a model's prediction to move only one way as a chosen input rises, with the other... | ||
| Prompt Robustness Testing | Testing | Architecture/neural Networks/transformer/llm Paradigm/generative +1 | Text | Prompt robustness testing measures how much a language model's answers, or its performance on a task, change when the... | ||
| Calibration with Equality of Opportunity | Algorithmic | Architecture/model Agnostic Paradigm/supervised +2 | Any | Calibration with equality of opportunity is a post-processing step that adjusts a classifier's scores so that two groups... | ||
| Agent Goal Misalignment Testing | Testing | Architecture/model Agnostic Architecture/neural Networks/transformer/llm +3 | Text | Agent goal misalignment testing checks whether an AI agent takes unsanctioned means to reach a goal it has been given.... | ||
| Constitutional AI Evaluation | Testing | Architecture/neural Networks/transformer/llm Requirements/white Box +1 | Text | Constitutional AI evaluation checks whether a language model follows the written principles, or 'constitution', that it... | ||
| Continual Learning Stability Testing | Testing | Architecture/model Agnostic Architecture/neural Networks +1 | Any | Continual learning stability testing checks whether a model that is updated over time with new data or tasks keeps what... |
Rows per page
Page 1 of 2