Energy-Aware Heterogeneous Federated Learning via Approximate DNN Accelerators
Gespeichert in:
| Titel: | Energy-Aware Heterogeneous Federated Learning via Approximate DNN Accelerators |
|---|---|
| Autoren: | Kilian Pfeiffer, Konstantinos Balaskas, Kostas Siozios, Jörg Henkel |
| Quelle: | IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems. 44:2054-2066 |
| Publication Status: | Preprint |
| Verlagsinformationen: | Institute of Electrical and Electronics Engineers (IEEE), 2025. |
| Publikationsjahr: | 2025 |
| Schlagwörter: | ddc:004, FOS: Computer and information sciences, DATA processing & computer science, Hardware Architecture (cs.AR), Computer Science - Hardware Architecture |
| Beschreibung: | In Federated Learning (FL), devices that participate in the training usually have heterogeneous resources, i.e., energy availability. In current deployments of FL, devices that do not fulfill certain hardware requirements are often dropped from the collaborative training. However, dropping devices in FL can degrade training accuracy and introduce bias or unfairness. Several works have tackled this problem on an algorithm level, e.g., by letting constrained devices train a subset of the server neural network (NN) model. However, it has been observed that these techniques are not effective w.r.t. accuracy. Importantly, they make simplistic assumptions about devices' resources via indirect metrics such as multiply accumulate (MAC) operations or peak memory requirements. We observe that memory access costs (that are currently not considered in simplistic metrics) have a significant impact on the energy consumption. In this work, for the first time, we consider on-device accelerator design for FL with heterogeneous devices. We utilize compressed arithmetic formats and approximate computing, targeting to satisfy limited energy budgets. Using a hardware-aware energy model, we observe that, contrary to the state of the art's moderate energy reduction, our technique allows for lowering the energy requirements (by 4x) while maintaining higher accuracy. accepted at IEEE TCAD |
| Publikationsart: | Article |
| ISSN: | 1937-4151 0278-0070 |
| DOI: | 10.1109/tcad.2024.3509793 |
| DOI: | 10.48550/arxiv.2402.18569 |
| Zugangs-URL: | http://arxiv.org/abs/2402.18569 |
| Rights: | IEEE Copyright arXiv Non-Exclusive Distribution |
| Dokumentencode: | edsair.doi.dedup.....d96d0a3d28363b06657995e5ef573d2f |
| Datenbank: | OpenAIRE |
Full Text Finder 
Nájsť tento článok vo Web of Science | Abstract: | In Federated Learning (FL), devices that participate in the training usually have heterogeneous resources, i.e., energy availability. In current deployments of FL, devices that do not fulfill certain hardware requirements are often dropped from the collaborative training. However, dropping devices in FL can degrade training accuracy and introduce bias or unfairness. Several works have tackled this problem on an algorithm level, e.g., by letting constrained devices train a subset of the server neural network (NN) model. However, it has been observed that these techniques are not effective w.r.t. accuracy. Importantly, they make simplistic assumptions about devices' resources via indirect metrics such as multiply accumulate (MAC) operations or peak memory requirements. We observe that memory access costs (that are currently not considered in simplistic metrics) have a significant impact on the energy consumption. In this work, for the first time, we consider on-device accelerator design for FL with heterogeneous devices. We utilize compressed arithmetic formats and approximate computing, targeting to satisfy limited energy budgets. Using a hardware-aware energy model, we observe that, contrary to the state of the art's moderate energy reduction, our technique allows for lowering the energy requirements (by 4x) while maintaining higher accuracy.<br />accepted at IEEE TCAD |
|---|---|
| ISSN: | 19374151 02780070 |
| DOI: | 10.1109/tcad.2024.3509793 |