Device Sampling for Heterogeneous Federated Learning: Theory, Algorithms, and Implementation
The conventional federated learning (FedL) architecture distributes machine learning (ML) across worker devices by having them train local models that are periodically aggregated by a server. FedL ignores two important characteristics of contemporary wireless networks, however: (i) the network may c...
Uloženo v:
| Vydáno v: | Annual Joint Conference of the IEEE Computer and Communications Societies s. 1 - 10 |
|---|---|
| Hlavní autoři: | , , , , , |
| Médium: | Konferenční příspěvek |
| Jazyk: | angličtina |
| Vydáno: |
IEEE
10.05.2021
|
| Témata: | |
| ISSN: | 2641-9874 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | The conventional federated learning (FedL) architecture distributes machine learning (ML) across worker devices by having them train local models that are periodically aggregated by a server. FedL ignores two important characteristics of contemporary wireless networks, however: (i) the network may contain heterogeneous communication/computation resources, while (ii) there may be significant overlaps in devices' local data distributions. In this work, we develop a novel optimization methodology that jointly accounts for these factors via intelligent device sampling complemented by device-to-device (D2D) offloading. Our optimization aims to select the best combination of sampled nodes and data offloading configuration to maximize FedL training accuracy subject to realistic constraints on the network topology and device capabilities. Theoretical analysis of the D2D offloading subproblem leads to new FedL convergence bounds and an efficient sequential convex optimizer. Using this result, we develop a sampling methodology based on graph convolutional networks (GCNs) which learns the relationship between network attributes, sampled nodes, and resulting offloading that maximizes FedL accuracy. Through evaluation on real-world datasets and network measurements from our IoT testbed, we find that our methodology while sampling less than 5% of all devices outperforms conventional FedL substantially both in terms of trained model accuracy and required resource utilization. |
|---|---|
| AbstractList | The conventional federated learning (FedL) architecture distributes machine learning (ML) across worker devices by having them train local models that are periodically aggregated by a server. FedL ignores two important characteristics of contemporary wireless networks, however: (i) the network may contain heterogeneous communication/computation resources, while (ii) there may be significant overlaps in devices' local data distributions. In this work, we develop a novel optimization methodology that jointly accounts for these factors via intelligent device sampling complemented by device-to-device (D2D) offloading. Our optimization aims to select the best combination of sampled nodes and data offloading configuration to maximize FedL training accuracy subject to realistic constraints on the network topology and device capabilities. Theoretical analysis of the D2D offloading subproblem leads to new FedL convergence bounds and an efficient sequential convex optimizer. Using this result, we develop a sampling methodology based on graph convolutional networks (GCNs) which learns the relationship between network attributes, sampled nodes, and resulting offloading that maximizes FedL accuracy. Through evaluation on real-world datasets and network measurements from our IoT testbed, we find that our methodology while sampling less than 5% of all devices outperforms conventional FedL substantially both in terms of trained model accuracy and required resource utilization. |
| Author | Hosseinalipour, Seyyedali Morabito, Roberto Brinton, Christopher G. Wang, Su Lee, Mengyuan Chiang, Mung |
| Author_xml | – sequence: 1 givenname: Su surname: Wang fullname: Wang, Su email: wang2506@purdue.edu organization: Purdue University,School of Electrical and Computer Engineering – sequence: 2 givenname: Mengyuan surname: Lee fullname: Lee, Mengyuan email: mengyuan_lee@zju.edu.cn organization: Zhejiang University,College of Information Science and Electronic Engineering – sequence: 3 givenname: Seyyedali surname: Hosseinalipour fullname: Hosseinalipour, Seyyedali email: hosseina@purdue.edu organization: Purdue University,School of Electrical and Computer Engineering – sequence: 4 givenname: Roberto surname: Morabito fullname: Morabito, Roberto email: roberto.morabito@princeton.edu organization: Princeton University, and Ericsson Research,Department of Electrical Engineering – sequence: 5 givenname: Mung surname: Chiang fullname: Chiang, Mung email: chiang@purdue.edu organization: Purdue University,School of Electrical and Computer Engineering – sequence: 6 givenname: Christopher G. surname: Brinton fullname: Brinton, Christopher G. email: cgb@purdue.edu organization: Purdue University,School of Electrical and Computer Engineering |
| BookMark | eNotkMtOwkAYRkejiYA8gZtxT3FuzMUdASskaBfizoRMO39hDJ0h02rC29tE8i3O5uQsviG6CTEAQo-UTCkl5mn9nheL4k0wo-mUEUanRmhtiLxCY6M0lXImCGczdo0GTAqaGa3EHRq27TchRCsmB-hrCb--Avxhm9PRhz2uY8Ir6CDFPQSIPy3OwUGyHTi8AZtCLz3j7QFiOk_w_LiPyXeHpp1gGxxe9xVoIHS28zHco9vaHlsYXzhCn_nLdrHKNsXrejHfZJ4R3mXcqFo4yw0XTFRlqRivHVhqDa0kLRkRklYCnOJCu8ookFJaZ7XoVwLnfIQe_rseAHan5BubzrvLGfwPPPNXfQ |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IH CBEJK RIE RIO |
| DOI | 10.1109/INFOCOM42981.2021.9488906 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP) 1998-present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISBN | 9781665403252 166540325X |
| EISSN | 2641-9874 |
| EndPage | 10 |
| ExternalDocumentID | 9488906 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IF 6IH 6IK 6IL 6IM 6IN AAJGR AAWTH ABLEC ACGFS ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IJVOP IPLJI M43 OCL RIE RIL RIO |
| ID | FETCH-LOGICAL-i203t-397f4da393424cbb723fdea1a91c61b20461c4ed7348dc97e666ada84848be333 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 83 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000702210400239&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 02:39:50 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i203t-397f4da393424cbb723fdea1a91c61b20461c4ed7348dc97e666ada84848be333 |
| PageCount | 10 |
| ParticipantIDs | ieee_primary_9488906 |
| PublicationCentury | 2000 |
| PublicationDate | 2021-May-10 |
| PublicationDateYYYYMMDD | 2021-05-10 |
| PublicationDate_xml | – month: 05 year: 2021 text: 2021-May-10 day: 10 |
| PublicationDecade | 2020 |
| PublicationTitle | Annual Joint Conference of the IEEE Computer and Communications Societies |
| PublicationTitleAbbrev | INFOCOM |
| PublicationYear | 2021 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0008726 |
| Score | 2.5355215 |
| Snippet | The conventional federated learning (FedL) architecture distributes machine learning (ML) across worker devices by having them train local models that are... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 1 |
| SubjectTerms | Conferences Device-to-device communication Machine learning Network topology Servers Training Wireless networks |
| Title | Device Sampling for Heterogeneous Federated Learning: Theory, Algorithms, and Implementation |
| URI | https://ieeexplore.ieee.org/document/9488906 |
| WOSCitedRecordID | wos000702210400239&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8NAEF5qEdGLj1Z8s4LHbptkt93Em1RDPdgWVOhBKPuYtAWbSJv6-51NQ1XwIrksCUtgZ7LzTXa-bwi5UR5GeTQ248KzzGF0pnhbMCXQm8EmRmlTNJuQ_X44GkXDCmlsuDAAUBSfQdMNi7N8m5mV-1XWitDbIqevvSVlZ83V2uy6oQw6O-S61NBsPfbjQXfwhLtt6LLAwG-Wk391USmCSLz_v9cfkPo3G48ON3HmkFQgPSJ7P4QEa-TtHtwXT5-VKxBPJxShKO25SpcMHQQwu6exU41AYGlpKak6uaVrYn6D3r1PssUsn86XDapSSwvJ4HnJSkrr5DV-eOn2WNk3gc0Cj-cMIUYirOIRF4EwWsuAJxaUryLfdHwdOI11I8A6YRtrIgmYwiirQoGXBs75MammWQonhHLEI55M8DGAUGGodeArLSPtmzYaX56Smlun8cdaGmNcLtHZ37fPya4zBSvETy9INV-s4JJsm898tlxcFfb8ApRsooY |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1bS8MwFA5DxcuLl028G8HHZWuTbG19k2nZcOsGTtiDMHI5nQPXytb5-026MhV8kb6ElkDJOc35TnO-7yB0KxwT5Y2xCeOOJhajE8EanAhuvBl0rIRUebMJL4r80SgYlFB1zYUBgLz4DGp2mJ_l61Qt7a-yemC8LbD62psNzqmzYmut913fo81tdFOoaNY7Udhv9Xtmv_VtHkjdWjH9Vx-VPIyE-_97gQNU-ebj4cE60hyiEiRHaO-HlGAZvT6A_ebxs7Al4skEGzCK27bWJTUuAia_x6HVjTDQUuNCVHVyh1fU_Cq-f5-k82n2NltUsUg0zkWDZwUvKamgl_Bx2GqTonMCmVKHZcSAjJhrwQLGKVdSepTFGoQrAlc1XUmtyrrioK20jVaBByaJEVr43FwSGGPHaCNJEzhBmBlE4nixeQzAhe9LSV0hvUC6qmHM752isl2n8cdKHGNcLNHZ37ev0U572OuOu53o6RztWrOQXAr1Am1k8yVcoi31mU0X86vctl81oaXN |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Annual+Joint+Conference+of+the+IEEE+Computer+and+Communications+Societies&rft.atitle=Device+Sampling+for+Heterogeneous+Federated+Learning%3A+Theory%2C+Algorithms%2C+and+Implementation&rft.au=Wang%2C+Su&rft.au=Lee%2C+Mengyuan&rft.au=Hosseinalipour%2C+Seyyedali&rft.au=Morabito%2C+Roberto&rft.date=2021-05-10&rft.pub=IEEE&rft.eissn=2641-9874&rft.spage=1&rft.epage=10&rft_id=info:doi/10.1109%2FINFOCOM42981.2021.9488906&rft.externalDocID=9488906 |