Adaptive Federated Learning on Non-IID Data With Resource Constraint

Federated learning (FL) has been widely recognized as a promising approach by enabling individual end-devices to cooperatively train a global model without exposing their own data. One of the key challenges in FL is the non-independent and identically distributed (Non-IID) data across the clients, w...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:IEEE transactions on computers Ročník 71; číslo 7; s. 1655 - 1667
Hlavní autori: Zhang, Jie, Guo, Song, Qu, Zhihao, Zeng, Deze, Zhan, Yufeng, Liu, Qifeng, Akerkar, Rajendra
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: New York IEEE 01.07.2022
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Predmet:
Adaptation models
Agglomeration
batch size adaption
Clients
Computational modeling
Data models
Data privacy
Deep learning
deep reinforcement learning
Distributed databases
Federated learning
Model accuracy
non-IID data
Servers
Training
ISSN:0018-9340, 1557-9956
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Federated learning (FL) has been widely recognized as a promising approach by enabling individual end-devices to cooperatively train a global model without exposing their own data. One of the key challenges in FL is the non-independent and identically distributed (Non-IID) data across the clients, which decreases the efficiency of stochastic gradient descent (SGD) based training process. Moreover, clients with different data distributions may cause bias to the global model update, resulting in a degraded model accuracy. To tackle the Non-IID problem in FL, we aim to optimize the local training process and global aggregation simultaneously. For local training, we analyze the effect of hyperparameters (e.g., the batch size, the number of local updates) on the training performance of FL. Guided by the toy example and theoretical analysis, we are motivated to mitigate the negative impacts incurred by Non-IID data via selecting a subset of participants and adaptively adjust their batch size. A deep reinforcement learning based approach has been proposed to adaptively control the training of local models and the phase of global aggregation. Extensive experiments on different datasets show that our method can improve the model accuracy by up to 30 percent, as compared to the state-of-the-art approaches.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0018-9340
1557-9956
DOI:10.1109/TC.2021.3099723