Safety
30 techniques
Ensuring AI systems operate safely and do not cause harm.
Browse by subcategory
All techniques
30 techniques
| Goals | Models | Data Types | Description | |||
|---|---|---|---|---|---|---|
| Out-of-Distribution Detector for Neural Networks | Algorithmic | Architecture/neural Networks Paradigm/discriminative +3 | Image | ODIN (Out-of-Distribution Detector for Neural Networks) flags inputs that look unlike the data a neural network... | ||
| Safety Envelope Testing | Testing | Architecture/model Agnostic Requirements/black Box | Any | Safety envelope testing checks how an AI system performs at and beyond the edges of the conditions it was designed for,... | ||
| Red Teaming | Process | Architecture/model Agnostic Requirements/black Box | Any | Red teaming is a structured exercise in which a team deliberately tries to make an AI system fail, so that weaknesses... | ||
| Human-in-the-Loop Safeguards | Process | Architecture/model Agnostic Requirements/black Box | Any | Human-in-the-loop safeguards hold an AI system's output at a checkpoint until a trained person has reviewed it.... | ||
| Confidence Thresholding | Algorithmic | Architecture/model Agnostic Requirements/black Box +1 | Any | Confidence thresholding uses a model score to decide whether a prediction should proceed automatically or receive... | ||
| Runtime Monitoring and Circuit Breakers | Algorithmic | Architecture/model Agnostic Requirements/black Box | Any | Runtime monitoring and circuit breakers observe a deployed AI service and restrict model calls when configured signals... | ||
| Model Cards | Documentation | Architecture/model Agnostic Requirements/black Box | Any | A model card is a short structured document published with a trained model, stating what the model is for and what it... | ||
| Datasheets for Datasets | Documentation | Architecture/model Agnostic Requirements/black Box | Any | A datasheet for a dataset is a short document that accompanies the dataset and answers a set of questions about it, in... | ||
| Causal Mediation Analysis in Language Models | Algorithmic | Architecture/neural Networks/transformer Architecture/neural Networks/transformer/llm +3 | Text | Causal mediation analysis traces which internal components of a language model carry the effect of a change in the input... | ||
| Agent Goal Misalignment Testing | Testing | Architecture/model Agnostic Architecture/neural Networks/transformer/llm +3 | Text | Agent goal misalignment testing checks whether an AI agent takes unsanctioned means to reach a goal it has been given.... | ||
| API Usage Pattern Monitoring | Algorithmic | Architecture/model Agnostic Requirements/black Box | Any | API usage pattern monitoring examines how each account uses a deployed model's API, to catch misuse that no single... | ||
| Constitutional AI Evaluation | Testing | Architecture/neural Networks/transformer/llm Requirements/white Box +1 | Text | Constitutional AI evaluation checks whether a language model follows the written principles, or 'constitution', that it... | ||
| Continual Learning Stability Testing | Testing | Architecture/model Agnostic Architecture/neural Networks +1 | Any | Continual learning stability testing checks whether a model that is updated over time with new data or tasks keeps what... | ||
| Data Poisoning Detection | Algorithmic | Architecture/model Agnostic Requirements/white Box +1 | Any | Data poisoning detection looks for training examples that an attacker has planted to change how a model behaves. The... | ||
| Jailbreak Resistance Testing | Testing | Architecture/neural Networks/transformer/llm Requirements/black Box | Text | Jailbreak resistance testing checks whether a language model or chatbot can be talked out of its safety rules. Testers... | ||
| Multi-Agent System Testing | Testing | Architecture/model Agnostic Requirements/black Box | Any | Multi-agent system testing examines systems in which several AI agents act on shared tasks or resources, to find... | ||
| Out-of-Domain Detection | Algorithmic | Architecture/model Agnostic Architecture/neural Networks/transformer/llm +2 | Text | Out-of-domain detection checks whether a user's query falls inside the topics and tasks a conversational AI system is... | ||
| Prompt Injection Testing | Testing | Architecture/neural Networks/transformer/llm Paradigm/generative +1 | Text | Prompt injection testing checks whether an AI system can be made to follow instructions its builders did not intend. In... | ||
| Reward Hacking Detection | Testing | Architecture/model Agnostic Architecture/neural Networks/transformer/llm +2 | Any | Reward hacking is when an agent trained to maximise a proxy reward (a score that only approximates what its designers... | ||
| Toxicity Detection | Testing | Architecture/model Agnostic Architecture/neural Networks/transformer/llm +2 | Text | Toxicity detection scores a model's outputs with a classifier trained to recognise abusive language, such as insults,... |
Rows per page
Page 1 of 2