Safety
Hazard Identification
Techniques that find how a system could fail or cause harm, by probing it for unsafe behaviours, failure modes and rare scenarios.
5 techniques
5 techniques
| Goals | Models | Data Types | Description | |||
|---|---|---|---|---|---|---|
| Red Teaming | Process | Architecture/model Agnostic Requirements/black Box | Any | Red teaming is a structured exercise in which a team deliberately tries to make an AI system fail, so that weaknesses... | ||
| Agent Goal Misalignment Testing | Testing | Architecture/model Agnostic Architecture/neural Networks/transformer/llm +3 | Text | Agent goal misalignment testing checks whether an AI agent takes unsanctioned means to reach a goal it has been given.... | ||
| Multi-Agent System Testing | Testing | Architecture/model Agnostic Requirements/black Box | Any | Multi-agent system testing examines systems in which several AI agents act on shared tasks or resources, to find... | ||
| Reward Hacking Detection | Testing | Architecture/model Agnostic Architecture/neural Networks/transformer/llm +2 | Any | Reward hacking is when an agent trained to maximise a proxy reward (a score that only approximates what its designers... | ||
| Dangerous Capability Evaluation | Testing | Architecture/neural Networks/transformer/llm Paradigm/generative +1 | Text | Dangerous capability evaluation measures whether a frontier AI model can carry out tasks that could enable severe harm,... |
Rows per page
Page 1 of 1