Bias Benchmark for Question Answering

Description

The Bias Benchmark for Question Answering (BBQ) is a fixed set of about 58,500 multiple-choice reading questions that tests whether a language model's answers rely on social stereotypes, across nine social dimensions (age, disability status, gender identity, nationality, physical appearance, race and ethnicity, religion, socio-economic status and sexual orientation) and two combinations of them, using stereotypes documented as harmful in US English-speaking contexts (Parrish et al., 2022). Each item describes two people from different groups and asks a question that points at a stereotype, such as who is bad at maths, with three answers to choose from: either person, or an answer such as 'cannot be determined'; in the ambiguous version the passage does not say which person the question is about, so 'cannot be determined' is correct, and in the disambiguated version an added sentence settles it. Items come in balanced sets that swap the two groups and their order and pair each negative question with a non-negative one, so that a model which simply prefers one group whatever the question is not counted as stereotyping. From the model's chosen answers (generated, or picked by comparing the probabilities it gives the three options) the test reports accuracy in each version and a bias score per category from -100% to 100%: the share of answers naming a person that follow the stereotype, rescaled so that 0 means no consistent direction and a negative score leans against the stereotype, and in the ambiguous version multiplied by the error rate, so that stereotyped guesses weigh more when the model guesses often; it also reports how far accuracy falls when the correct answer goes against the stereotype. Unlike substituting identity terms into a deployer's own inputs, or varying the people shown in few-shot examples, BBQ is a fixed external question set that tests what a model concludes about people when the text does, and does not, support a conclusion.

Example Use Cases

Fairness

A government department choosing between three language models for a service that answers citizens' questions about benefits runs the full BBQ set on each, with the same prompt, answer extraction and scoring for all three. It compares accuracy and both bias scores category by category, rules out any model whose ambiguous-context bias score in any category is further from zero, in either direction, than a limit set before testing, and records that the result covers US stereotypes only.

A recruitment software company fine-tuning an open-weight model to answer HR helpdesk questions runs BBQ on the base model and on every fine-tuned version through the same evaluation harness, logging each answer. A release is blocked if any category's ambiguous-context bias score moves further from zero than the base model's by more than an agreed margin, or if the accuracy gap between items whose correct answer matches the stereotype and items whose correct answer goes against it widens.

An NHS trust's digital team preparing a health-information assistant needs evidence about UK stereotypes, which BBQ was not built for. Following the procedure used to build a Korean version, it keeps templates that transfer, changes target groups where UK stereotypes differ, drops those that do not apply, writes UK-specific ones, surveys members of the UK public to confirm each stereotype and its target group, and reports results on the original and adapted sets side by side.

Limitations

BBQ's stereotypes, and most of its categories, come from US English-speaking contexts, and its authors say results should not be used as evidence about other cultures. Adapting it to Korean meant changing target groups, dropping templates and adding four categories, and a machine-translated BBQ gave different bias scores from the adapted version (Jin et al., 2024). For a UK deployment the original set says nothing about UK-specific stereotypes.
A low bias score is weak evidence that a model is unbiased. Each category rests on about 25 templates with very short passages and three fixed answers; the authors warn that a low score is not proof of an unbiased model, and that for a model used on very different text BBQ is unlikely to be a valid measure. It counts only stereotypes that change the chosen answer, so it misses bias that shifts the model's probabilities without changing its choice.
The bias score leaves out 'cannot be determined' answers and has a sign, so it must be read with accuracy and by its distance from zero. A model that always gives that answer has full accuracy and a zero bias score on ambiguous items, yet fails every disambiguated one. A negative score means answers lean against the stereotype, as HELM found for most of the models it tested (Liang et al., 2023); a one-sided limit such as 'below 5%' passes such a model.
Scores depend on the prompt format and on how answers are extracted or scored, so they compare only across models tested the same way. In the original study one UnifiedQA model, given the same items in two input formats, scored 2.4 and 18.6 on ambiguous gender-identity items. Results for a single template rest on few items (one reported example has 72), so they are noisy and should be combined across templates for the same stereotype.
It is weak evidence about bias against people with two combined identities. In the original study, results on the two intersectional categories (race by gender, race by socio-economic status) were too inconsistent for the authors to conclude that models responded to more than one aspect of identity, and the lm-evaluation-harness task notes that these categories are not really taken into account in its bias scores.

Resources

Research Papers

BBQ: A hand-built bias benchmark for question answering

Alicia Parrish et al.•Jan 1, 2022

Software Packages

Language Model Evaluation Harness (EleutherAI/lm-evaluation-harness)

Oct 12, 2026

A framework for few-shot evaluation of language models.

Documentations

BBQ: Repository for the Bias Benchmark for QA dataset (nyu-mll/BBQ)

Oct 8, 2026

Repository for the Bias Benchmark for QA dataset.

Related Techniques

Name	Description	Assurance Goals
Few-Shot Fairness Evaluation	Few-shot fairness evaluation tests whether the worked examples placed in a prompt change how fairly a language model treats different demographic groups. The evaluator builds many versions of the same prompt, varying which examples appear and how each demographic group is represented among them. Each version is run on the same test inputs, and group-level results are compared across versions. Typical checks include the gap in outcomes between groups, whether outputs echo stereotypes present in the examples, and whether an input gets the same treatment when only a demographic term in it is swapped. The result shows how sensitive the model's fairness is to the choice of examples, and which compositions make outcomes more or less even.	Fairness
Embedding Bias Analysis	Embedding bias analysis tests whether the numerical representations a model uses for words, images or documents link social groups to stereotyped attributes. An embedding places each item at a point in a many-dimensional space so that similar items sit close together. The best-known test, the Word Embedding Association Test (Caliskan et al., 2017), compares how close two sets of target words, such as engineering and nursing occupations, sit to two sets of attribute words, such as male and female terms, and reports an effect size and a significance level. Other analyses project embeddings onto a gender or ethnicity direction, or cluster them to see whether groups separate. The scores describe the representation itself, and any link to bias in an application built on it has to be tested separately.	Fairness
Causal Discrimination Testing	Causal discrimination testing checks whether a system gives different outputs to otherwise identical inputs because of a protected attribute, by changing only that attribute, or a name, term or other proxy that signals it, and measuring how often or how much the output changes. Galhotra, Brun and Meliou (2017), who called the approach fairness testing, generate inputs from a description of the valid input fields, make copies of each that differ only in the protected attribute (for example, recorded sex), run them all through the system as a black box, and report the causal discrimination score: the share of inputs whose decision changes, sampled until it is known within a chosen margin of error and confidence level. For text, the same test substitutes identity terms or names in otherwise unchanged sentences, such as one religion or ethnicity for another, and reports the average change in the model's score across the substitutions, which Garg et al. (2019) call the counterfactual token fairness gap. Because every other part of the input is held fixed, the test needs no model of how the attribute influences other variables: it measures whether the system itself responds to the attribute, or to the chosen proxy. The output is a score per attribute and, if the tester logs them, the input pairs whose outputs differed.	Fairness