Reliability
Performance Assessment
Techniques that measure how well a model performs and whether the measured result is real and repeatable.
10 techniques
10 techniques
| Goals | Models | Data Types | Description | |||
|---|---|---|---|---|---|---|
| Permutation Tests | Algorithmic | Architecture/model Agnostic Requirements/black Box | Any | A permutation test asks how likely an observed result would be if there were no real relationship in the data. The... | ||
| Cross-validation | Testing | Architecture/model Agnostic Paradigm/supervised +1 | Any | Cross-validation estimates how well a model will perform on data it has not seen, and how much that estimate depends on... | ||
| Area Under Precision-Recall Curve | Metric | Architecture/model Agnostic Paradigm/supervised +2 | Any | Area Under Precision-Recall Curve (AUPRC) summarises how well a classifier finds the cases that matter when those cases... | ||
| Cross-Modal Consistency Scoring | Metric | Architecture/model Agnostic Requirements/black Box | Image Text | Cross-modal consistency scoring checks whether content in one modality matches content in another, such as an image and... | ||
| Synthetic Data Evaluation | Testing | Architecture/model Agnostic Requirements/black Box | Tabular | Synthetic data evaluation checks whether an artificial dataset made to stand in for sensitive real records is safe to... | ||
| Retrieval-Augmented Generation Evaluation | Testing | Architecture/neural Networks/transformer/llm Paradigm/generative +1 | Text | A retrieval-augmented generation (RAG) system answers a question by retrieving passages from a document collection and... | ||
| Bootstrap Confidence Intervals for Evaluation Metrics | Algorithmic | Architecture/model Agnostic Requirements/black Box | Any | Bootstrap confidence intervals put a range around a model's measured performance, showing how much the score could... | ||
| Operating-Point ROC Analysis | Metric | Architecture/model Agnostic Paradigm/supervised +1 | Any | Operating-point ROC analysis measures how often a classifier makes each kind of mistake at the decision threshold it... | ||
| Continuous Ranked Probability Score | Metric | Architecture/model Agnostic Requirements/black Box +1 | Time-series Tabular | The continuous ranked probability score (CRPS) measures the quality of a probabilistic forecast of a numerical quantity,... | ||
| Rolling-Origin Evaluation | Testing | Architecture/model Agnostic Requirements/black Box | Time-series | Rolling-origin evaluation, also called time-series cross-validation or backtesting, estimates how accurately a... |
Rows per page
Page 1 of 1