Energy-Aware Heterogeneous Federated Learning via Approximate DNN Accelerators
Uložené v:
| Názov: | Energy-Aware Heterogeneous Federated Learning via Approximate DNN Accelerators |
|---|---|
| Autori: | Kilian Pfeiffer, Konstantinos Balaskas, Kostas Siozios, Jörg Henkel |
| Zdroj: | IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems. 44:2054-2066 |
| Publication Status: | Preprint |
| Informácie o vydavateľovi: | Institute of Electrical and Electronics Engineers (IEEE), 2025. |
| Rok vydania: | 2025 |
| Predmety: | ddc:004, FOS: Computer and information sciences, DATA processing & computer science, Hardware Architecture (cs.AR), Computer Science - Hardware Architecture |
| Popis: | In Federated Learning (FL), devices that participate in the training usually have heterogeneous resources, i.e., energy availability. In current deployments of FL, devices that do not fulfill certain hardware requirements are often dropped from the collaborative training. However, dropping devices in FL can degrade training accuracy and introduce bias or unfairness. Several works have tackled this problem on an algorithm level, e.g., by letting constrained devices train a subset of the server neural network (NN) model. However, it has been observed that these techniques are not effective w.r.t. accuracy. Importantly, they make simplistic assumptions about devices' resources via indirect metrics such as multiply accumulate (MAC) operations or peak memory requirements. We observe that memory access costs (that are currently not considered in simplistic metrics) have a significant impact on the energy consumption. In this work, for the first time, we consider on-device accelerator design for FL with heterogeneous devices. We utilize compressed arithmetic formats and approximate computing, targeting to satisfy limited energy budgets. Using a hardware-aware energy model, we observe that, contrary to the state of the art's moderate energy reduction, our technique allows for lowering the energy requirements (by 4x) while maintaining higher accuracy. accepted at IEEE TCAD |
| Druh dokumentu: | Article |
| ISSN: | 1937-4151 0278-0070 |
| DOI: | 10.1109/tcad.2024.3509793 |
| DOI: | 10.48550/arxiv.2402.18569 |
| Prístupová URL adresa: | http://arxiv.org/abs/2402.18569 |
| Rights: | IEEE Copyright arXiv Non-Exclusive Distribution |
| Prístupové číslo: | edsair.doi.dedup.....d96d0a3d28363b06657995e5ef573d2f |
| Databáza: | OpenAIRE |
Full Text Finder 
Nájsť tento článok vo Web of Science | Abstrakt: | In Federated Learning (FL), devices that participate in the training usually have heterogeneous resources, i.e., energy availability. In current deployments of FL, devices that do not fulfill certain hardware requirements are often dropped from the collaborative training. However, dropping devices in FL can degrade training accuracy and introduce bias or unfairness. Several works have tackled this problem on an algorithm level, e.g., by letting constrained devices train a subset of the server neural network (NN) model. However, it has been observed that these techniques are not effective w.r.t. accuracy. Importantly, they make simplistic assumptions about devices' resources via indirect metrics such as multiply accumulate (MAC) operations or peak memory requirements. We observe that memory access costs (that are currently not considered in simplistic metrics) have a significant impact on the energy consumption. In this work, for the first time, we consider on-device accelerator design for FL with heterogeneous devices. We utilize compressed arithmetic formats and approximate computing, targeting to satisfy limited energy budgets. Using a hardware-aware energy model, we observe that, contrary to the state of the art's moderate energy reduction, our technique allows for lowering the energy requirements (by 4x) while maintaining higher accuracy.<br />accepted at IEEE TCAD |
|---|---|
| ISSN: | 19374151 02780070 |
| DOI: | 10.1109/tcad.2024.3509793 |