applicable models
llm
Techniques for Large Language Models (GPT, BERT, etc.)
19 techniques
| Goals | Models | Data Types | Description | |||
|---|---|---|---|---|---|---|
| Prompt Robustness Testing | Testing | Architecture/neural Networks/transformer/llm Paradigm/generative +1 | Text | Prompt robustness testing measures how much a language model's answers, or its performance on a task, change when the... | ||
| Causal Mediation Analysis in Language Models | Algorithmic | Architecture/neural Networks/transformer Architecture/neural Networks/transformer/llm +3 | Text | Causal mediation analysis traces which internal components of a language model carry the effect of a change in the input... | ||
| Agent Goal Misalignment Testing | Testing | Architecture/model Agnostic Architecture/neural Networks/transformer/llm +3 | Text | Agent goal misalignment testing checks whether an AI agent takes unsanctioned means to reach a goal it has been given.... | ||
| Chain-of-Thought Faithfulness Evaluation | Testing | Architecture/neural Networks/transformer/llm Paradigm/generative +1 | Text | Chain-of-thought faithfulness evaluation tests whether the step-by-step reasoning a language model writes out is what... | ||
| Constitutional AI Evaluation | Testing | Architecture/neural Networks/transformer/llm Requirements/white Box +1 | Text | Constitutional AI evaluation checks whether a language model follows the written principles, or 'constitution', that it... | ||
| Embedding Bias Analysis | Testing | Architecture/neural Networks Architecture/neural Networks/transformer +3 | Text Image | Embedding bias analysis tests whether the numerical representations a model uses for words, images or documents link... | ||
| Few-Shot Fairness Evaluation | Testing | Architecture/neural Networks/transformer/llm Paradigm/generative +1 | Text | Few-shot fairness evaluation tests whether the worked examples placed in a prompt change how fairly a language model... | ||
| Jailbreak Resistance Testing | Testing | Architecture/neural Networks/transformer/llm Requirements/black Box | Text | Jailbreak resistance testing checks whether a language model or chatbot can be talked out of its safety rules. Testers... | ||
| Knowledge Boundary Probing | Algorithmic | Architecture/neural Networks/transformer/llm Requirements/black Box | Text | Knowledge boundary probing maps what a language model can and cannot reliably answer. A probe set is built from... | ||
| Out-of-Domain Detection | Algorithmic | Architecture/model Agnostic Architecture/neural Networks/transformer/llm +2 | Text | Out-of-domain detection checks whether a user's query falls inside the topics and tasks a conversational AI system is... | ||
| Prompt Injection Testing | Testing | Architecture/neural Networks/transformer/llm Paradigm/generative +1 | Text | Prompt injection testing checks whether an AI system can be made to follow instructions its builders did not intend. In... | ||
| Retrieval-Augmented Generation Evaluation | Testing | Architecture/neural Networks/transformer/llm Paradigm/generative +1 | Text | A retrieval-augmented generation (RAG) system answers a question by retrieving passages from a document collection and... | ||
| Reward Hacking Detection | Testing | Architecture/model Agnostic Architecture/neural Networks/transformer/llm +2 | Any | Reward hacking is when an agent trained to maximise a proxy reward (a score that only approximates what its designers... | ||
| Toxicity Detection | Testing | Architecture/model Agnostic Architecture/neural Networks/transformer/llm +2 | Text | Toxicity detection scores a model's outputs with a classifier trained to recognise abusive language, such as insults,... | ||
| Training Data Extraction Testing | Testing | Architecture/neural Networks/transformer/llm Paradigm/generative +1 | Text | Training data extraction testing measures how much of its training text a generative language model reproduces word for... | ||
| PII Detection and Redaction in Training Text | Algorithmic | Architecture/model Agnostic Architecture/neural Networks/transformer/llm +1 | Text | PII detection and redaction finds personally identifiable information (PII), such as names, email addresses, phone... | ||
| Dangerous Capability Evaluation | Testing | Architecture/neural Networks/transformer/llm Paradigm/generative +1 | Text | Dangerous capability evaluation measures whether a frontier AI model can carry out tasks that could enable severe harm,... | ||
| Sparse Autoencoder Feature Analysis | Algorithmic | Architecture/neural Networks/transformer Architecture/neural Networks/transformer/llm +2 | Text | Sparse autoencoder feature analysis looks for human-interpretable units inside a neural network, usually a language... | ||
| Bias Benchmark for Question Answering | Testing | Architecture/neural Networks/transformer/llm Requirements/black Box | Text | The Bias Benchmark for Question Answering (BBQ) is a fixed set of about 58,500 multiple-choice reading questions that... |
Rows per page
Page 1 of 1