Security
Vulnerability Testing
Techniques that attack a model or its defences to measure how well they hold, such as jailbreak, injection or extraction tests.
7 techniques
7 techniques
| Goals | Models | Data Types | Description | |||
|---|---|---|---|---|---|---|
| Red Teaming | Process | Architecture/model Agnostic Requirements/black Box | Any | Red teaming is a structured exercise in which a team deliberately tries to make an AI system fail, so that weaknesses... | ||
| Adversarial Training Evaluation | Testing | Architecture/model Agnostic Architecture/neural Networks +1 | Any | Adversarial training evaluation tests whether a model trained on adversarial examples (inputs altered slightly to cause... | ||
| Jailbreak Resistance Testing | Testing | Architecture/neural Networks/transformer/llm Requirements/black Box | Text | Jailbreak resistance testing checks whether a language model or chatbot can be talked out of its safety rules. Testers... | ||
| Model Extraction Defence Testing | Testing | Architecture/model Agnostic Requirements/black Box | Any | Model extraction defence testing checks whether a model offered through a query interface, such as a prediction API, can... | ||
| Membership Inference Attack Testing | Testing | Architecture/model Agnostic Requirements/black Box | Any | Membership inference attack testing checks whether someone with access to a trained model can tell if a particular... | ||
| Prompt Injection Testing | Testing | Architecture/neural Networks/transformer/llm Paradigm/generative +1 | Text | Prompt injection testing checks whether an AI system can be made to follow instructions its builders did not intend. In... | ||
| AutoAttack | Testing | Architecture/neural Networks Paradigm/supervised +2 | Image | AutoAttack is a fixed suite of four automated adversarial attacks, introduced by Croce and Hein (ICML 2020), that... |
Rows per page
Page 1 of 1