Robust Aggregation for Federated Learning

We present a novel approach to federated learning that endows its aggregation process with greater robustness to potential poisoning of local data or model parameters of participating devices. The proposed approach, Robust Federated Aggregation (RFA), relies on the aggregation of updates using the g...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:IEEE transactions on signal processing Ročník 70; s. 1142 - 1154
Hlavní autori: Pillutla, Krishna, Kakade, Sham M., Harchaoui, Zaid
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: New York IEEE 2022
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Predmet:
Agglomeration
Aggregates
Algorithms
Collaborative work
Computer vision
corrupted updates
Corruption
data privacy
distributed learning
Federated learning
Machine learning
Natural language processing
Optimization
Privacy
robust aggregation
Robustness
Servers
Signal processing algorithms
ISSN:1053-587X, 1941-0476
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:We present a novel approach to federated learning that endows its aggregation process with greater robustness to potential poisoning of local data or model parameters of participating devices. The proposed approach, Robust Federated Aggregation (RFA), relies on the aggregation of updates using the geometric median, which can be computed efficiently using a Weiszfeld-type algorithm. RFA is agnostic to the level of corruption and aggregates model updates without revealing each device's individual contribution. We establish the convergence of the robust federated learning algorithm for the stochastic learning of additive models with least squares. We also offer two variants of RFA: a faster one with one-step robust aggregation, and another one with on-device personalization. We present experimental results with additive models and deep networks for three tasks in computer vision and natural language processing. The experiments show that RFA is competitive with the classical aggregation when the level of corruption is low, while demonstrating greater robustness under high corruption.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1053-587X
1941-0476
DOI:10.1109/TSP.2022.3153135