FedGroup: Efficient Federated Learning via Decomposed Similarity-Based Clustering
Federated Learning (FL) enables the multiple participating devices to collaboratively contribute to a global neural network model while keeping the training data locally. Unlike the centralized training setting, the non-IID and imbalanced (statistical heterogeneity) training data of FL is distribute...
Uložené v:
| Vydané v: | 2021 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom) s. 228 - 237 |
|---|---|
| Hlavní autori: | , , , , , , |
| Médium: | Konferenčný príspevok.. |
| Jazyk: | English |
| Vydavateľské údaje: |
IEEE
01.09.2021
|
| Predmet: | |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | Federated Learning (FL) enables the multiple participating devices to collaboratively contribute to a global neural network model while keeping the training data locally. Unlike the centralized training setting, the non-IID and imbalanced (statistical heterogeneity) training data of FL is distributed in the federated network, which will increase the divergences between the local models and the global model, further degrading performance. In this paper, we propose a novel clustered federated learning (CFL) framework FedGroup, in which we 1) group the training of clients based on the similarities between the clients' optimization directions for high training performance; 2) construct a new data-driven distance measure to improve the efficiency of the client clustering procedure. 3) implement a newcomer device cold start mechanism based on the auxiliary global model for framework scalability and practicality.FedGroup can achieve improvements by dividing joint optimization into groups of sub-optimization and can be combined with FL optimizer FedProx. The convergence and complexity are analyzed to demonstrate the efficiency of our proposed framework. We also evaluate FedGroup and FedGrouProx (combined with FedProx) on several open datasets and made comparisons with related CFL frameworks. The results show that FedGroup can significantly improve absolute test accuracy by +14.1% on FEMNIST compared to FedAvg, +3.4% on Sentiment140 compared to FedProx, +6.9% on MNIST compared to FeSEM. |
|---|---|
| AbstractList | Federated Learning (FL) enables the multiple participating devices to collaboratively contribute to a global neural network model while keeping the training data locally. Unlike the centralized training setting, the non-IID and imbalanced (statistical heterogeneity) training data of FL is distributed in the federated network, which will increase the divergences between the local models and the global model, further degrading performance. In this paper, we propose a novel clustered federated learning (CFL) framework FedGroup, in which we 1) group the training of clients based on the similarities between the clients' optimization directions for high training performance; 2) construct a new data-driven distance measure to improve the efficiency of the client clustering procedure. 3) implement a newcomer device cold start mechanism based on the auxiliary global model for framework scalability and practicality.FedGroup can achieve improvements by dividing joint optimization into groups of sub-optimization and can be combined with FL optimizer FedProx. The convergence and complexity are analyzed to demonstrate the efficiency of our proposed framework. We also evaluate FedGroup and FedGrouProx (combined with FedProx) on several open datasets and made comparisons with related CFL frameworks. The results show that FedGroup can significantly improve absolute test accuracy by +14.1% on FEMNIST compared to FedAvg, +3.4% on Sentiment140 compared to FedProx, +6.9% on MNIST compared to FeSEM. |
| Author | Liang, Liang Liu, Duo Ji, Xinyuan Duan, Moming Chen, Xianzhang Tan, Yujuan Liu, Renping |
| Author_xml | – sequence: 1 givenname: Moming surname: Duan fullname: Duan, Moming organization: Chongqing University,College of Computer Science,Chongqing,China – sequence: 2 givenname: Duo surname: Liu fullname: Liu, Duo organization: Chongqing University,College of Computer Science,Chongqing,China – sequence: 3 givenname: Xinyuan surname: Ji fullname: Ji, Xinyuan organization: Xi'an Jiaotong University,School of Computer Science and Technology,Xi'an,China – sequence: 4 givenname: Renping surname: Liu fullname: Liu, Renping organization: Chongqing University,College of Computer Science,Chongqing,China – sequence: 5 givenname: Liang surname: Liang fullname: Liang, Liang organization: Chongqing University,College of Computer Science,Chongqing,China – sequence: 6 givenname: Xianzhang surname: Chen fullname: Chen, Xianzhang organization: Chongqing University,College of Computer Science,Chongqing,China – sequence: 7 givenname: Yujuan surname: Tan fullname: Tan, Yujuan organization: Chongqing University,College of Computer Science,Chongqing,China |
| BookMark | eNotjctOhDAYhWuiCx3nCdzwAsXeoNTdDHNxEhI16HryF35ME6CkgMm8vZhxdU6-nMsDue19j4QYzmLOmXk-le8but3lrZ9rWvrKQZv7jpbzOIHrF5sIlvFYMMFjxpgSN2RtdMbTNFEy0Yrfk48D1sfg5-El2jeNqxz2U7QwDDBhHRUIoXf9d_TjINph5bvBjwsvXedaCG660C38gbxdTjEs0Udy10A74vpfV-TrsP_MX2nxdjzlm4I6weRElWYcrLGIElOWoKmFrJSSPEPgqBkIENpKYRPUBmsuRSWXjpVaC22UlSvydN11iHgegusgXM4mVUpnQv4CiLFXbw |
| CODEN | IEEPAD |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IL CBEJK RIE RIL |
| DOI | 10.1109/ISPA-BDCloud-SocialCom-SustainCom52081.2021.00042 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan All Online (POP All Online) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP All) 1998-Present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| EISBN | 9781665435741 1665435747 |
| EndPage | 237 |
| ExternalDocumentID | 9644782 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: National Natural Science Foundation of China funderid: 10.13039/501100001809 – fundername: Fundamental Research Funds for the Central Universities funderid: 10.13039/501100012226 |
| GroupedDBID | 6IE 6IL CBEJK RIE RIL |
| ID | FETCH-LOGICAL-i203t-4701ab9bee3e605e9d23c44318ea1e70a2a27b32b5e79ed132c3470b3772794b3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 78 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000766837400027&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 05:06:17 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i203t-4701ab9bee3e605e9d23c44318ea1e70a2a27b32b5e79ed132c3470b3772794b3 |
| PageCount | 10 |
| ParticipantIDs | ieee_primary_9644782 |
| PublicationCentury | 2000 |
| PublicationDate | 2021-Sept. |
| PublicationDateYYYYMMDD | 2021-09-01 |
| PublicationDate_xml | – month: 09 year: 2021 text: 2021-Sept. |
| PublicationDecade | 2020 |
| PublicationTitle | 2021 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom) |
| PublicationTitleAbbrev | ISPA-BDCLOUD-SOCIALCOM-SUSTAINCOM |
| PublicationYear | 2021 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| Score | 2.2002928 |
| Snippet | Federated Learning (FL) enables the multiple participating devices to collaboratively contribute to a global neural network model while keeping the training... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 228 |
| SubjectTerms | Collaborative work Distributed Machine Learning Federated Learning Linear approximation Neural networks Performance evaluation Scalability Training Training data |
| Title | FedGroup: Efficient Federated Learning via Decomposed Similarity-Based Clustering |
| URI | https://ieeexplore.ieee.org/document/9644782 |
| WOSCitedRecordID | wos000766837400027&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA5tEfGk0opv9uDR2N2k22y82ceil1KpQm8lj1lZqF1pu_39TrJLRfDiLQwbCLMJ33zJzDeE3JnQ8CwximYqipGghIoqbvoUpEG8j42JE-WbTYjJJJnP5bRB7ve1MADgk8_gwQ39W74tTOmuyroSwRsRrUmaQvSrWq1DImvZzO7LbPpEB6PhsigtrYpb8UzRWVWIhMOYIf4hJWSR1-hkvxqqeDxJj_-3khPS-SnMC6Z7yDklDVi1yWsK1l8hPQZjLwiBU4PUiURgHGmDWkH1I9jlKhiByyEvNmif5Z850lqMwulAOcNwWTrVBPy0Q97T8dvwmdadEmjOQr6lPRFGSksNwAH5CUjLuOlhbJCAikCEiikmNGc6BiHBIgM1HOdojrE1HkjNz0hrVazgnARJnInMQCQhhp4OIbE6inC7WSUUUidxQdrOHYuvSgxjUXvi8m_zFTly_q6Ssq5Ja7su4YYcmN0236xv_R_8BkkuoMs |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3fS8MwED7mFPVJZRN_2wcfjWuT1ba-uR9lwzkmm7C3kaRXKcxVtnV_v5e2TARffAtHA-Ga8N2X3H0HcKdtLWJfSxZLxyWCYksmhX5kGGjCe1dr15d5swlvOPSn02BUgfttLQwi5sln-GCG-Vt-lOrMXJU1AgJvQrQd2DWds8pqrX0ISuHMRn88ematTnueZhErylvpVLFxUYpEQ5cTAhIp5E6u0sl_tVTJESU8-t9ajqH-U5pnjbagcwIVXNTgLcQov0R6srq5JARNtUIjE0GRZGSVGqof1iaRVgdNFnm6Ivs4-UyI2FIczlrSGNrzzOgm0Kd1eA-7k3aPlb0SWMJtsWZNz3akChSiQGIoGERc6CZFBz5KBz1bcsk9Jbhy0QswIg6qBc1RgqJrOpJKnEJ1kS7wDCzfjb1YoxOgi01lox8px6ENF0lPEnnyzqFm3DH7KuQwZqUnLv4238JBb_I6mA36w5dLODS-L1K0rqC6XmZ4DXt6s05Wy5v8b34D1YukFA |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2021+IEEE+Intl+Conf+on+Parallel+%26+Distributed+Processing+with+Applications%2C+Big+Data+%26+Cloud+Computing%2C+Sustainable+Computing+%26+Communications%2C+Social+Computing+%26+Networking+%28ISPA%2FBDCloud%2FSocialCom%2FSustainCom%29&rft.atitle=FedGroup%3A+Efficient+Federated+Learning+via+Decomposed+Similarity-Based+Clustering&rft.au=Duan%2C+Moming&rft.au=Liu%2C+Duo&rft.au=Ji%2C+Xinyuan&rft.au=Liu%2C+Renping&rft.date=2021-09-01&rft.pub=IEEE&rft.spage=228&rft.epage=237&rft_id=info:doi/10.1109%2FISPA-BDCloud-SocialCom-SustainCom52081.2021.00042&rft.externalDocID=9644782 |