expertise needed
domain expertise
Requires deep understanding of the problem domain
31 techniques
| Goals | Models | Data Types | Description | |||
|---|---|---|---|---|---|---|
| Safety Envelope Testing | Testing | Architecture/model Agnostic Requirements/black Box | Any | Safety envelope testing systematically evaluates AI system performance at the boundaries of its intended operational... | ||
| Internal Review Boards | Process | Architecture/model Agnostic Requirements/black Box | Any | Internal Review Boards (IRBs) provide independent, systematic evaluation of AI/ML projects throughout their lifecycle to... | ||
| Human-in-the-Loop Safeguards | Process | Architecture/model Agnostic Requirements/black Box | Any | Human-in-the-loop safeguards establish systematic checkpoints where human experts review, validate, or override AI/ML... | ||
| Datasheets for Datasets | Documentation | Architecture/model Agnostic Requirements/black Box | Any | Datasheets for datasets establish comprehensive documentation standards for datasets, systematically recording creation... | ||
| Monotonicity Constraints | Algorithmic | Architecture/probabilistic/gaussian Processes Architecture/tree Based +2 | Tabular | Monotonicity constraints enforce consistent directional relationships between input features and model predictions,... | ||
| Concept Activation Vectors | Algorithmic | Architecture/neural Networks Requirements/gradient Access +2 | Any | Concept Activation Vectors (CAVs), also known as Testing with Concept Activation Vectors (TCAV), identify mathematical... | ||
| Relabelling | Procedural | Architecture/model Agnostic Paradigm/supervised +2 | Any | A preprocessing fairness technique that modifies class labels in training data to achieve equal positive outcome rates... | ||
| Adversarial Robustness Testing | Testing | Architecture/model Agnostic Architecture/neural Networks +2 | Image Text Tabular | Adversarial robustness testing evaluates model resilience against intentionally crafted inputs designed to cause... | ||
| Adversarial Training Evaluation | Testing | Architecture/model Agnostic Architecture/neural Networks +1 | Any | Adversarial training evaluation assesses whether models trained with adversarial examples have genuinely improved... | ||
| Agent Goal Misalignment Testing | Testing | Architecture/model Agnostic Architecture/neural Networks/transformer/llm +3 | Any | Agent goal misalignment testing identifies scenarios where AI agents pursue objectives in unintended ways or develop... | ||
| API Usage Pattern Monitoring | Algorithmic | Architecture/model Agnostic Requirements/black Box | Any | API usage pattern monitoring analyses AI model API usage to detect anomalies and generate evidence of secure operation.... | ||
| Chain-of-Thought Faithfulness Evaluation | Testing | Architecture/neural Networks/transformer/llm Paradigm/generative +1 | Text | Chain-of-thought faithfulness evaluation assesses the quality and faithfulness of step-by-step reasoning produced by... | ||
| Constitutional AI Evaluation | Testing | Architecture/neural Networks/transformer/llm Requirements/white Box +1 | Text | Constitutional AI evaluation assesses models trained to follow explicit behavioural principles or 'constitutions' that... | ||
| Continual Learning Stability Testing | Testing | Architecture/model Agnostic Architecture/neural Networks +1 | Any | Continual learning stability testing evaluates whether models that learn from streaming data maintain performance on... | ||
| Data Poisoning Detection | Algorithmic | Architecture/model Agnostic Requirements/white Box +1 | Any | Data poisoning detection identifies malicious training data designed to compromise model behaviour. This technique... | ||
| Few-Shot Fairness Evaluation | Testing | Architecture/neural Networks/transformer/llm Paradigm/generative +1 | Text | Few-shot fairness evaluation assesses whether in-context learning with few-shot examples introduces or amplifies biases... | ||
| Hallucination Detection | Testing | Architecture/neural Networks/transformer Architecture/neural Networks/transformer/llm +2 | Text | Hallucination detection identifies when generative models produce factually incorrect, fabricated, or ungrounded... | ||
| Safety Guardrails | Algorithmic | Architecture/model Agnostic Requirements/black Box | Text | Safety guardrails apply real-time content moderation and safety constraints during deployment, generating evidence of... | ||
| Jailbreak Resistance Testing | Testing | Architecture/neural Networks/transformer/llm Requirements/black Box | Text | Jailbreak resistance testing evaluates LLM defences against techniques that bypass safety constraints. This involves... | ||
| Epistemic Uncertainty Quantification | Algorithmic | Architecture/model Agnostic Requirements/black Box | Any | Epistemic uncertainty quantification systematically measures model uncertainty about what it knows, partially knows, and... |
Rows per page
Page 1 of 2