Federated Learning

Description

Federated learning trains one shared model across several parties, such as phones or hospitals, without collecting their data in one place. Each party trains the current model on its own data and sends back only an update, such as the changed weights (the model's internal numbers) or the gradients (the direction of change). A central coordinator combines the updates, typically by averaging, into an improved shared model and sends it out again, and the cycle repeats over many rounds. Raw records stay with their owners, which limits how much sensitive data is stored or moved and can support data protection rules that favour collecting less. It is not a privacy guarantee on its own, because updates can still reveal information about the data behind them and the coordinator has to be trusted.

Example Use Cases

Privacy

A mobile keyboard team trains a next-word prediction model on users' phones and receives only model updates, so typed text never reaches its servers. The team uses this to decide whether personalised suggestions can ship without collecting message content centrally.

Hospitals in several countries train a model that predicts COVID-19 outcomes on their own patient records and share only model updates with a coordinating centre, so no patient record leaves a hospital. Each hospital's data governance board uses this to decide whether it can join the collaboration.

Several banks jointly train a fraud-detection model on their own transaction data and share only model updates, so no bank's customer records reach a competitor. Each bank's compliance team uses the setup to decide whether joint training fits its data-sharing restrictions.

Limitations

Communication overhead can be substantial, especially with frequent model updates and large models, potentially limiting scalability and increasing training time compared to centralised approaches.
Statistical heterogeneity across participants (non-IID data distributions) can lead to training instability, slower convergence, and reduced model performance compared to centralised training on pooled data.
System heterogeneity in computational capabilities, network connectivity, and availability of participating devices can create bottlenecks and introduce bias towards more capable participants.
Privacy vulnerabilities remain through gradient leakage attacks, model inversion, and membership inference attacks that can potentially reconstruct sensitive information from shared model updates.
A faulty or malicious participant can submit harmful updates that the coordinator cannot check against the participant's data, for example to plant a hidden backdoor in the shared model. Defences such as robust aggregation add cost, and the coordinator must also be trusted with the updates it receives.

Resources

Research Papers

Communication-Efficient Learning of Deep Networks from Decentralized Data

H. Brendan McMahan et al.•Jan 26, 2023

Modern mobile devices have access to a wealth of data suitable for learning models, which in turn can greatly improve the user experience on the device. For example, language models can improve speech recognition and text entry, and image models can automatically select good photos. However, this rich data is often privacy sensitive, large in quantity, or both, which may preclude logging to the data center and training there using conventional approaches. We advocate an alternative that leaves the training data distributed on the mobile devices, and learns a shared model by aggregating locally-computed updates. We term this decentralized approach Federated Learning. We present a practical method for the federated learning of deep networks based on iterative model averaging, and conduct an extensive empirical evaluation, considering five different model architectures and four datasets. These experiments demonstrate the approach is robust to the unbalanced and non-IID data distributions that are a defining characteristic of this setting. Communication costs are the principal constraint, and we show a reduction in required communication rounds by 10-100x as compared to synchronized stochastic gradient descent.

Software Packages

flwrlabs/flower: Flower: A Friendly Federated AI Framework

Oct 8, 2026

Flower: A Friendly Federated AI Framework

Tutorials

Federated Learning - DeepLearning.AI

Daniel J. Beutel and Nicholas Lane•Jan 1, 2025

Documentations

Open Federated Learning (OpenFL) Documentation

Openfl Developers•Jan 1, 2020

Related Techniques

Name	Description	Assurance Goals
Differentially Private Stochastic Gradient Descent	Differentially private stochastic gradient descent (DP-SGD) trains a neural network, or another model fitted by gradient descent, so that the finished model carries a formal differential privacy guarantee about the individual examples it was trained on. Differential privacy is more often applied by adding calibrated noise to the answer of a query or to a published statistic; DP-SGD instead changes the training algorithm itself. At each step it draws a random batch, computes the gradient (the direction in which an example pushes the model's parameters) separately for every example, scales down any gradient longer than a fixed clipping bound, adds Gaussian noise proportional to that bound to the sum, and updates the model with the noisy average, so that no single example can move the model by much and what influence remains is masked. A privacy accountant adds up the privacy cost of every step from the noise level, the sampling rate and the number of steps, and reports the total as epsilon and delta: epsilon limits how much more likely any trained model is with a given example in the training data than without it, and delta is a small probability that this limit fails. The output is a trained model with a stated (epsilon, delta) guarantee, which also covers anything later computed from the model, including its predictions.	Privacy
Machine Unlearning	Machine unlearning removes the influence of specific training records from a trained model, so that the model behaves as if it had never seen them. It is used when someone withdraws consent or asks for deletion, for example under the right to erasure (the 'right to be forgotten') in data protection law, and retraining from scratch would be too slow or costly. Exact methods, such as sharded training, split the data into parts that each train their own sub-model, so a deletion means retraining only the sub-model that used those records, which gives a result statistically the same as retraining without them. Approximate methods instead edit the trained model directly to reduce the records' influence, which is faster but gives weaker assurance, and certified methods add a formal bound on how much influence can remain.	Privacy Transparency
Learning Adversarially Fair and Transferable Representations	Learning Adversarially Fair and Transferable Representations (LAFTR) trains an encoder that turns each record into a representation, a vector of learned features, from which a protected attribute is hard to recover, so that models later built on that representation by other parties, including for prediction tasks not known in advance, also treat groups fairly. It was designed for a data owner who shares encoded data with downstream model builders and cannot control how they train. During training, the encoder, a decoder that reconstructs the input and a classifier for the original task play against an adversary network that tries to predict the protected attribute from the representation; the encoder is updated to defeat the adversary while keeping the information the other two need. The adversary's objective is matched to demographic parity, equalised odds or equal opportunity, and the authors prove that an optimal adversary's objective bounds that unfairness measure for any classifier trained on the representation (for the two error-rate measures the adversary also sees the labels used in training, so the attribute is hidden given the label, and the bound is with respect to those labels). The output is a frozen encoder, delivered with the accuracy and fairness of classifiers trained on top of it.	Fairness