Statistical Oversampling Methods

Description

Statistical oversampling methods rebalance a training set by adding synthetic examples of an under-represented category instead of copying existing ones. The foundational method, SMOTE (Synthetic Minority Over-sampling Technique), takes a minority-class example, picks one of its nearest minority-class neighbours, and creates a new example at a random point on the line between the two in feature space. Variants differ in which examples seed the new points, such as Borderline-SMOTE (examples near the class boundary), ADASYN (examples with many neighbours from the other class) and SVMSMOTE (the support vectors of a support vector machine). Standard SMOTE balances the outcome labels and ignores group membership, so for fairness work the same generation is applied within each combination of protected group and label, making every combination equally large in the training data. Only the training data change, and plain SMOTE needs numeric features, with SMOTE-NC covering a mix of numeric and categorical ones.

Example Use Cases

Fairness

A bank's credit team finds that approved applicants from one ethnic group make up a small share of its loan training data. It adds synthetic approved applicants for that group with SMOTE until every group-and-outcome combination is the same size, then compares the approval-rate gap between groups, and the gap in approval of applicants who went on to repay, on held-out applications against a model trained on the original data.

A hospital analytics team building a readmission model finds that patients from one small ethnic group rarely appear among the readmitted cases. It applies Borderline-SMOTE within each group's readmitted cases, then reports recall for each group on held-out patients, with and without oversampling, to its clinical safety committee before deployment.

A university's student-support team builds a dropout-risk model from past cohorts in which students from low-income areas who left are few. It uses ADASYN to add synthetic examples for that combination, then checks on held-out cohorts that the model flags those students at a rate no further below the rate for other students than the model trained on the original data.

Limitations

Plain SMOTE and its variants balance the outcome labels, not the groups, and can widen group gaps. In one study (logistic regression on the Adult income data), SMOTE raised the statistical parity difference between sexes from 0.21 to 0.33 (Chakraborty et al., 2021). Balancing protected-group-and-label combinations is needed for fairness, and the gaps must be measured on held-out data.
Synthetic points are interpolated from the few real examples already present, so they add no new information about a small group and do not repair why it is under-represented, such as how the data were collected. Counts on the oversampled set look larger than the real evidence behind them.
Interpolation assumes the space between two minority examples also belongs to the minority class. Where examples are scattered, overlap the majority class or include outliers, new points land in the wrong region and add label noise. ADASYN, which favours hard cases, can concentrate on outliers.
In high-dimensional data SMOTE mostly fails to correct the lean towards the majority class and is less effective than random undersampling. It lowers variability and makes synthetic points correlated, and with k-nearest-neighbour classifiers and no feature selection it tips predictions towards the minority class (Blagus and Lusa, 2013).
Oversampling before splitting the data, or before cross-validation, leaks information into the test set and inflates results. Resampling must touch only the training part, for example inside an imbalanced-learn Pipeline, and tests should use the natural class mix.

Resources

Research Papers

SMOTE: Synthetic Minority Over-sampling Technique

N. V. Chawla et al.•Jun 1, 2002

An approach to the construction of classifiers from imbalanced datasets is described. A dataset is imbalanced if the classification categories are not approximately equally represented. Often real-world data sets are predominately composed of ``normal'' examples with only a small percentage of ``abnormal'' or ``interesting'' examples. It is also the case that the cost of misclassifying an abnormal (interesting) example as a normal example is often much higher than the cost of the reverse error. Under-sampling of the majority (normal) class has been proposed as a good means of increasing the sensitivity of a classifier to the minority class. This paper shows that a combination of our method of over-sampling the minority (abnormal) class and under-sampling the majority (normal) class can achieve better classifier performance (in ROC space) than only under-sampling the majority class. This paper also shows that a combination of our method of over-sampling the minority class and under-sampling the majority class can achieve better classifier performance (in ROC space) than varying the loss ratios in Ripper or class priors in Naive Bayes. Our method of over-sampling the minority class involves creating synthetic minority class examples. Experiments are performed using C4.5, Ripper and a Naive Bayes classifier. The method is evaluated using the area under the Receiver Operating Characteristic curve (AUC) and the ROC convex hull strategy.

ADASYN: Adaptive synthetic sampling approach for imbalanced learning

Haibo He et al.•Jun 1, 2008

This paper presents a novel adaptive synthetic (ADASYN) sampling approach for learning from imbalanced data sets. The essential idea of ADASYN is to use a weighted distribution for different minority class examples according to their level of difficulty in learning, where more synthetic data is generated for minority class examples that are harder to learn compared to those minority examples that are easier to learn. As a result, the ADASYN approach improves learning with respect to the data distributions in two ways: (1) reducing the bias introduced by the class imbalance, and (2) adaptively shifting the classification decision boundary toward the difficult examples. Simulation analyses on several machine learning data sets show the effectiveness of this method across five evaluation metrics.

Software Packages

scikit-learn-contrib/imbalanced-learn

Feb 16, 2026

A Python Package to Tackle the Curse of Imbalanced Datasets in Machine Learning

Tutorials

Compare over-sampling samplers - imbalanced-learn

Documentations

Over-sampling - imbalanced-learn user guide

Related Techniques

Name	Description	Assurance Goals
Preferential Sampling	Preferential Sampling is a pre-processing method, proposed by Kamiran and Calders, that rebalances a training set so that a protected group and everyone else have the same rate of positive outcomes, such as being hired or approved. A ranking model first scores every example by how likely it is to have a positive label. The method then compares the size of each group-and-outcome combination with the size it would have if group and outcome were unrelated. It adds copies of positive examples from the disadvantaged group and negative examples from the favoured group, and removes negative examples from the disadvantaged group and positive examples from the favoured group, always starting with those closest to the decision boundary. Labels are never changed, so a standard classifier can learn from the rebalanced set.	Fairness
Disparate Impact Remover	Disparate Impact Remover is a pre-processing method that edits the numerical features of a dataset so that they reveal less about a protected attribute such as race or sex. For each feature it compares how the values are distributed within each protected group, then moves every value towards a shared distribution while keeping its rank within its own group, so people near the top of their group stay near the top. The user sets a repair level from 0 (no change) to 1 (each feature is distributed identically in every group). The aim is to avoid disparate impact, where one group's selection rate falls below 80 per cent of the most favoured group's rate. Stronger repair narrows that gap but costs predictive accuracy.	Fairness
Fair Adversarial Networks	Fair Adversarial Networks is a pre-processing method, proposed by Cevora (2020), that rewrites a dataset so that a protected characteristic such as race, sex or age is hard to predict from it, including through proxies such as postcode or occupation. It trains two neural networks against each other. An autoencoder (a network that learns to reproduce its input) rewrites each record while staying as close to the original as it can, and an adversary network tries to predict the protected characteristic from the rewritten records. The autoencoder is penalised both for changing the data and for the adversary's success, so over alternating rounds of training it strips out the information the adversary relies on and changes little else. The output is the rewritten dataset, which any downstream analysis or model can use unchanged, and in the paper success means a freshly trained adversary does no better than always guessing the most common category.	Fairness