Machine Unlearning
Description
Machine unlearning removes the influence of specific training records from a trained model, so that the model behaves as if it had never seen them. It is used when someone withdraws consent or asks for deletion, for example under the right to erasure (the 'right to be forgotten') in data protection law, and retraining from scratch would be too slow or costly. Exact methods, such as sharded training, split the data into parts that each train their own sub-model, so a deletion means retraining only the sub-model that used those records, which gives a result statistically the same as retraining without them. Approximate methods instead edit the trained model directly to reduce the records' influence, which is faster but gives weaker assurance, and certified methods add a formal bound on how much influence can remain.
Example Use Cases
Privacy
A social media platform responds to a user's deletion request by removing that user's interaction history from its recommendation model, retraining only the affected part rather than the whole system. The data protection team uses this to decide whether it can meet deletion deadlines without a full retrain each time.
A hospital withdraws a patient's records from a readmission-risk model after the patient withdraws consent for research use, retraining only the shard that contained them. The research governance board uses the retraining log to decide whether the model can stay in use.
A lender removes a former customer's transaction history from its credit-risk model after the customer withdraws consent for analytics use. The model risk team decides whether a faster approximate update is acceptable or whether the affected shard must be retrained exactly.
Transparency
A financial institution records the withdrawn transaction-record identifiers, affected constituent models and retraining checkpoints in its SISA workflow. Reviewers use this update history to inspect the scope of the completed unlearning operation.
Limitations
- Exact unlearning for deep neural networks gives limited savings over full retraining: sharded training sped unlearning up by only 1.36 times on ImageNet-scale classification in the SISA paper. For very large foundation models, even storing the checkpoints needed for partial retraining can be infeasible, which limits exact methods to smaller models.
- Approximate methods can leave residual influence from the removed data in the model, and their guarantees are weaker or only probabilistic. That matters when a regulator or the data subject expects genuine erasure, because the result cannot be shown to match a model retrained without the data.
- Unlearning is hard to verify from outside. There is no direct way to measure one record's influence on a model's weights, so the data subject must largely trust the provider's procedure, and membership inference tests can detect leakage but cannot prove removal.
- Removing many records, or records the model relies on, can lower accuracy, and sharded exact methods lose accuracy because each sub-model sees only part of the data, most on complex tasks. Teams with frequent deletion requests need to monitor accuracy after each round.
- Exact approaches such as sharded training must be built into the training pipeline before the model is first trained, and most methods need white-box access to weights and training data. A model already trained without them, or a third-party model, can only be retrained or approximately edited.
Resources
Research Papers
Digital forgetting in large language models: A survey of unlearning methods
Large language models (LLMs) have become the state of the art in natural language processing. The massive adoption of generative LLMs and the capabilities they have shown have prompted public concerns regarding their impact on the labor market, privacy, the use of copyrighted work, and how these models align with human ethics and the rule of law. As a response, new regulations are being pushed, which require developers and service providers to evaluate, monitor, and forestall or at least mitigate the risks posed by their models. One mitigation strategy is digital forgetting: given a model with undesirable knowledge or behavior, the goal is to obtain a new model where the detected issues are no longer present. Digital forgetting is usually enforced via machine unlearning techniques, which modify trained machine learning models for them to behave as models trained on a subset of the original training data. In this work, we describe the motivations and desirable properties of digital forgetting when applied to LLMs, and we survey recent works on machine unlearning. Specifically, we propose a taxonomy of unlearning methods based on the reach and depth of the modifications done on the models, we discuss and compare the effectiveness of machine unlearning methods for LLMs proposed so far, and we survey their evaluation. Finally, we describe open problems of machine unlearning applied to LLMs and we put forward recommendations for developers and practitioners.
Towards Making Systems Forget with Machine Unlearning
Today's systems produce a rapidly exploding amount of data, and the data further derives more data, forming a complex data propagation network that we call the data's lineage. There are many reasons that users want systems to forget certain data including its lineage. From a privacy perspective, users who become concerned with new privacy risks of a system often want the system to forget their data and lineage. From a security perspective, if an attacker pollutes an anomaly detector by injecting manually crafted data into the training data set, the detector must forget the injected data to regain security. From a usability perspective, a user can remove noise and incorrect entries so that a recommendation engine gives useful recommendations. Therefore, we envision forgetting systems, capable of forgetting certain data and their lineages, completely and quickly. This paper focuses on making learning systems forget, the process of which we call machine unlearning, or simply unlearning. We present a general, efficient unlearning approach by transforming learning algorithms used by a system into a summation form. To forget a training data sample, our approach simply updates a small number of summations -- asymptotically faster than retraining from scratch. Our approach is general, because the summation form is from the statistical query learning in which many machine learning algorithms can be implemented. Our approach also applies to all stages of machine learning, including feature selection and modeling. Our evaluation, on four diverse learning systems and real-world workloads, shows that our approach is general, effective, fast, and easy to use.
Machine Unlearning
Once users have shared their data online, it is generally difficult for them to revoke access and ask for the data to be deleted. Machine learning (ML) exacerbates this problem because any model trained with said data may have memorized it, putting users at risk of a successful privacy attack exposing their information. Yet, having models unlearn is notoriously difficult.We introduce SISA training, a framework that expedites the unlearning process by strategically limiting the influence of a data point in the training procedure. While our framework is applicable to any learning algorithm, it is designed to achieve the largest improvements for stateful algorithms like stochastic gradient descent for deep neural networks. SISA training reduces the computational overhead associated with unlearning, even in the worst-case setting where unlearning requests are made uniformly across the training set. In some cases, the service provider may have a prior on the distribution of unlearning requests that will be issued by users. We may take this prior into account to partition and order data accordingly, and further decrease overhead from unlearning.Our evaluation spans several datasets from different domains, with corresponding motivations for unlearning. Under no distributional assumptions, for simple learning tasks, we observe that SISA training improves time to unlearn points from the Purchase dataset by 4.63×, and 2.45× for the SVHN dataset, over retraining from scratch. SISA training also provides a speed-up of 1.36× in retraining for complex learning tasks such as ImageNet classification; aided by transfer learning, this results in a small degradation in accuracy. Our work contributes to practical data governance in machine unlearning.