Cooperative Learning VIA Federated Distillation OVER Fading Channels

Cooperative training methods for distributed machine learning are typically based on the exchange of local gradients or local model parameters. The latter approach is known as Federated Learning (FL). An alternative solution with reduced communication overhead, referred to as Federated Distillation...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Proceedings of the ... IEEE International Conference on Acoustics, Speech and Signal Processing (1998) S. 8856 - 8860
Hauptverfasser: Ahn, Jin-Hyun, Simeone, Osvaldo, Kang, Joonhyuk
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: IEEE 01.05.2020
Schlagworte:
Distributed training
Fading channels
federated learning
joint source-channel coding
Machine learning
Protocols
Speech coding
Speech processing
Training
Wireless communication
ISSN:2379-190X
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Cooperative training methods for distributed machine learning are typically based on the exchange of local gradients or local model parameters. The latter approach is known as Federated Learning (FL). An alternative solution with reduced communication overhead, referred to as Federated Distillation (FD), was recently proposed that exchanges only averaged model outputs. While prior work studied implementations of FL over wireless fading channels, here we propose wireless protocols for FD and for an enhanced version thereof that leverages an offline communication phase to communicate "mixed-up" covariate vectors. The proposed implementations consist of different combinations of digital schemes based on separate source-channel coding and of over-the-air computing strategies based on analog joint source-channel coding. It is shown that the enhanced version FD has the potential to significantly outperform FL in the presence of limited spectral resources.
ISSN:2379-190X
DOI:10.1109/ICASSP40776.2020.9053448