Clustered Federated Multi-Task Learning: A Communication-and-Computation Efficient Sparse Sharing Approach
Federated multi-task learning (FMTL) is a promising technology to tackle one of the most severe non-independent and identically distributed (non-IID) data challenge in federated learning (FL), which treats each client as a single task and learns personalized models by exploiting task correlations. H...
Gespeichert in:
| Veröffentlicht in: | IEEE transactions on wireless communications Jg. 24; H. 6; S. 4824 - 4838 |
|---|---|
| Hauptverfasser: | , , , , , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
New York
IEEE
01.06.2025
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Schlagworte: | |
| ISSN: | 1536-1276, 1558-2248 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | Federated multi-task learning (FMTL) is a promising technology to tackle one of the most severe non-independent and identically distributed (non-IID) data challenge in federated learning (FL), which treats each client as a single task and learns personalized models by exploiting task correlations. However, the transmission of individual task models generally results in a significant amount of communication overhead compared with global model broadcasting. Furthermore, related works mainly focus on FMTLs with default and static relationships among tasks, which obliterates the non-IID data characteristic. To address these issues, we propose a novel Clustered FMTL mechanism via Sparse Sharing (FedSS). Specifically, we introduce an iterative model pruning approach that trains customized client models to deal with the non-IID issue. Thereafter, we divide clients into different tasks according to their model similarities to promote communication efficiency. Based on clustered tasks, we introduce a sparse sharing mechanism that allows clients to share model parameters dynamically among different tasks to further boost the training performance. On the other aspect, the infertile communication resources would degrade the FMTL performance by restricting the personalized model transmissions. Hence, we first theoretically analyze the convergence performance of the proposed FedSS, which quantitatively unveils the relationship between the local model training performance and communication resources. Thereafter, we formulate a communication-and-computation efficient optimization problem via a joint sparsity ratio assignment and bandwidth allocation strategy. Closed-form expressions for the optimal sparsity ratio and bandwidth allocation are derived based on Lyapunov optimization and block coordinate update (BCU) algorithms. Numerical results illustrate that the proposed FedSS outperforms the benchmarks, and achieves an efficient communication and computation performance. |
|---|---|
| AbstractList | Federated multi-task learning (FMTL) is a promising technology to tackle one of the most severe non-independent and identically distributed (non-IID) data challenge in federated learning (FL), which treats each client as a single task and learns personalized models by exploiting task correlations. However, the transmission of individual task models generally results in a significant amount of communication overhead compared with global model broadcasting. Furthermore, related works mainly focus on FMTLs with default and static relationships among tasks, which obliterates the non-IID data characteristic. To address these issues, we propose a novel Clustered FMTL mechanism via Sparse Sharing (FedSS). Specifically, we introduce an iterative model pruning approach that trains customized client models to deal with the non-IID issue. Thereafter, we divide clients into different tasks according to their model similarities to promote communication efficiency. Based on clustered tasks, we introduce a sparse sharing mechanism that allows clients to share model parameters dynamically among different tasks to further boost the training performance. On the other aspect, the infertile communication resources would degrade the FMTL performance by restricting the personalized model transmissions. Hence, we first theoretically analyze the convergence performance of the proposed FedSS, which quantitatively unveils the relationship between the local model training performance and communication resources. Thereafter, we formulate a communication-and-computation efficient optimization problem via a joint sparsity ratio assignment and bandwidth allocation strategy. Closed-form expressions for the optimal sparsity ratio and bandwidth allocation are derived based on Lyapunov optimization and block coordinate update (BCU) algorithms. Numerical results illustrate that the proposed FedSS outperforms the benchmarks, and achieves an efficient communication and computation performance. |
| Author | Wen, Dingzhu Chen, Qimei Li, Ming Ai, Yuhan Zeng, Jun Zhu, Guangxu Jiang, Hao |
| Author_xml | – sequence: 1 givenname: Yuhan surname: Ai fullname: Ai, Yuhan email: aiyuhan@whu.edu.cn organization: School of Electronic Information, Wuhan University, Wuhan, China – sequence: 2 givenname: Qimei orcidid: 0000-0003-2497-8911 surname: Chen fullname: Chen, Qimei email: chenqimei@whu.edu.cn organization: School of Electronic Information, Wuhan University, Wuhan, China – sequence: 3 givenname: Guangxu orcidid: 0000-0001-9532-9201 surname: Zhu fullname: Zhu, Guangxu email: gxzhu@sribd.cn organization: Shenzhen Research Institute of Big Data, Shenzhen, China – sequence: 4 givenname: Dingzhu orcidid: 0000-0003-0538-5811 surname: Wen fullname: Wen, Dingzhu email: wendzh@shanghaitech.edu.cn organization: School of Information Science and Technology, ShanghaiTech University, Shanghai, China – sequence: 5 givenname: Hao orcidid: 0000-0002-8533-1612 surname: Jiang fullname: Jiang, Hao email: jh@whu.edu.cn organization: School of Electronic Information, Wuhan University, Wuhan, China – sequence: 6 givenname: Jun surname: Zeng fullname: Zeng, Jun email: 2021182120064@whu.edu.cn organization: School of Electronic Information, Wuhan University, Wuhan, China – sequence: 7 givenname: Ming surname: Li fullname: Li, Ming email: mli@fiberhome.com organization: School of Electronic Information, Wuhan University, Wuhan, China |
| BookMark | eNpNUD1PwzAQtVCRaAs7A4MlZhd_xglbFbWAVMTQIsbISS80pXWC7Qz8exzaAd1w707vQ3oTNLKtBYRuGZ0xRrOHzUc-45SrmVBSCpZeoDFTKiWcy3Q0YJEQxnVyhSbe7yllOlFqjPb5ofcBHGzxErbgTIjotT-EhmyM_8IrMM429vMRz3HeHo-9bSoTmtYSY7ckfro-_N14UddN1YANeN0Z5wGvd8ZFJZ53nWtNtbtGl7U5eLg57yl6Xy42-TNZvT295PMVqbhUgTAKKacJrTUziahUrVVpmMhSVUrNMqAMZCkzyktd0kRk1Mishm1CpaiY4kxM0f3JN8Z-9-BDsW97Z2NkITjTWidUDCx6YlWu9d5BXXSuORr3UzBaDI0WsdFiaLQ4NxoldydJAwD_6Fmc6PgLtPFzAA |
| CODEN | ITWCAX |
| Cites_doi | 10.1109/TWC.2021.3111707 10.1109/TCOMM.2021.3124961 10.1109/TWC.2021.3108197 10.1109/JIOT.2022.3228893 10.1109/TNNLS.2022.3224252 10.1109/TWC.2023.3342626 10.1145/3453142.3492909 10.1109/LCOMM.2022.3174295 10.1109/JIOT.2021.3136205 10.1109/JSAC.2023.3242727 10.1109/TWC.2019.2946245 10.1109/TVT.2023.3287355 10.1109/JSAC.2015.2481209 10.1109/JSAC.2021.3126076 10.1109/ICASSP.2014.6855096 10.1109/TWC.2020.3039309 10.1109/TNNLS.2019.2944481 10.1109/LCOMM.2023.3308334 10.1109/JIOT.2021.3095077 10.1109/TKDE.2021.3124599 10.1609/aaai.v34i05.6424 10.1109/JIOT.2022.3201310 10.1109/ICDE53745.2022.00077 10.1109/TMLCN.2023.3303292 10.1109/MVT.2020.3019650 10.1109/COMST.2020.2986024 10.1145/2872427.2883041 10.1007/s11432-022-3652-2 10.1109/ICC45855.2022.9838880 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2025 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2025 |
| DBID | 97E RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D |
| DOI | 10.1109/TWC.2025.3544318 |
| DatabaseName | IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional |
| DatabaseTitle | CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Technology Research Database |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 1558-2248 |
| EndPage | 4838 |
| ExternalDocumentID | 10_1109_TWC_2025_3544318 10909031 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: Shanghai Sailing Program grantid: 23YF1427400 funderid: 10.13039/100016096 – fundername: Key Research and Development Plan of Hubei Province grantid: 2023BCB041 – fundername: National Natural Science Foundation of China grantid: 62371313; 62401369 – fundername: Shenzhen-Hong Kong-Macau Technology Research Programme (Type C) grantid: SGDX20230821091559018 – fundername: National Key Research and Development Program of China grantid: 2024YFE0115900; 2024YFB2908001 funderid: 10.13039/501100012166 – fundername: Wuhan Science and Technology Achievement Transformation Project grantid: 2024030803010178 |
| GroupedDBID | -~X 0R~ 29I 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACGFS ACIWK AENEX AETIX AGQYO AGSQL AHBIQ AIBXA AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD HZ~ H~9 IES IFIPE IPLJI JAVBF LAI M43 O9- OCL P2P RIA RIE RNS AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D |
| ID | FETCH-LOGICAL-c245t-10e82060f71a63c5f75ba13985b4719e01e4b4902b7b06390a49fed6043c15213 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 0 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001506722000049&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1536-1276 |
| IngestDate | Thu Sep 25 23:40:54 EDT 2025 Sat Nov 29 07:48:16 EST 2025 Wed Aug 27 01:47:15 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 6 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c245t-10e82060f71a63c5f75ba13985b4719e01e4b4902b7b06390a49fed6043c15213 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0002-8533-1612 0000-0003-2497-8911 0000-0001-9532-9201 0000-0003-0538-5811 |
| PQID | 3217776031 |
| PQPubID | 105736 |
| PageCount | 15 |
| ParticipantIDs | proquest_journals_3217776031 ieee_primary_10909031 crossref_primary_10_1109_TWC_2025_3544318 |
| PublicationCentury | 2000 |
| PublicationDate | 2025-06-01 |
| PublicationDateYYYYMMDD | 2025-06-01 |
| PublicationDate_xml | – month: 06 year: 2025 text: 2025-06-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationPlace | New York |
| PublicationPlace_xml | – name: New York |
| PublicationTitle | IEEE transactions on wireless communications |
| PublicationTitleAbbrev | TWC |
| PublicationYear | 2025 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref35 ref12 ref34 ref15 ref14 ref36 ref31 ref30 ref33 ref32 ref2 ref1 ref17 ref19 McMahan (ref6) ref18 Fifty (ref16); 34 Ma (ref28) 2022 ref24 ref23 ref26 ref25 Smith (ref11) ref20 ref22 ref21 Arivazhagan (ref10) 2019 ref27 ref29 ref8 ref7 Shen (ref13) ref4 ref3 Li (ref9) ref5 |
| References_xml | – start-page: 429 volume-title: Proc. 3rd Mach. Learn. Syst. Conf. ident: ref9 article-title: Federated optimization in heterogeneous networks – year: 2022 ident: ref28 article-title: On the convergence of clustered federated learning publication-title: arXiv:2202.06187 – ident: ref33 doi: 10.1109/TWC.2021.3111707 – ident: ref19 doi: 10.1109/TCOMM.2021.3124961 – ident: ref8 doi: 10.1109/TWC.2021.3108197 – ident: ref25 doi: 10.1109/JIOT.2022.3228893 – ident: ref14 doi: 10.1109/TNNLS.2022.3224252 – ident: ref31 doi: 10.1109/TWC.2023.3342626 – ident: ref18 doi: 10.1145/3453142.3492909 – ident: ref26 doi: 10.1109/LCOMM.2022.3174295 – ident: ref23 doi: 10.1109/JIOT.2021.3136205 – volume: 34 start-page: 27503 volume-title: Proc. Adv. Neural Inf. Process. Syst. (NIPS) ident: ref16 article-title: Efficiently identifying task groupings for multi-task learning – ident: ref17 doi: 10.1109/JSAC.2023.3242727 – ident: ref21 doi: 10.1109/TWC.2019.2946245 – ident: ref20 doi: 10.1109/TVT.2023.3287355 – ident: ref32 doi: 10.1109/JSAC.2015.2481209 – ident: ref2 doi: 10.1109/JSAC.2021.3126076 – ident: ref34 doi: 10.1109/ICASSP.2014.6855096 – ident: ref22 doi: 10.1109/TWC.2020.3039309 – ident: ref30 doi: 10.1109/TNNLS.2019.2944481 – ident: ref29 doi: 10.1109/LCOMM.2023.3308334 – ident: ref12 doi: 10.1109/JIOT.2021.3095077 – ident: ref5 doi: 10.1109/TKDE.2021.3124599 – year: 2019 ident: ref10 article-title: Federated learning with personalization layers publication-title: arXiv:1912.00818 – ident: ref15 doi: 10.1609/aaai.v34i05.6424 – ident: ref24 doi: 10.1109/JIOT.2022.3201310 – ident: ref7 doi: 10.1109/ICDE53745.2022.00077 – ident: ref27 doi: 10.1109/TMLCN.2023.3303292 – ident: ref1 doi: 10.1109/MVT.2020.3019650 – start-page: 1273 volume-title: Proc. Int. Conf. Artif. Intell. Stat. (AISTATS) ident: ref6 article-title: Communication-efficient learning of deep networks from decentralized data – ident: ref4 doi: 10.1109/COMST.2020.2986024 – ident: ref36 doi: 10.1145/2872427.2883041 – start-page: 4427 volume-title: Proc. Int. Conf. Adv. Neural Inf. Process. Syst. ident: ref11 article-title: Federated multi-task learning – ident: ref3 doi: 10.1007/s11432-022-3652-2 – start-page: 1 volume-title: Proc. Adv. Neural Inf. Process. Syst. (NIPS) ident: ref13 article-title: Variational multi-task learning with Gumbel-Softmax priors – ident: ref35 doi: 10.1109/ICC45855.2022.9838880 |
| SSID | ssj0017655 |
| Score | 2.4848514 |
| Snippet | Federated multi-task learning (FMTL) is a promising technology to tackle one of the most severe non-independent and identically distributed (non-IID) data... |
| SourceID | proquest crossref ieee |
| SourceType | Aggregation Database Index Database Publisher |
| StartPage | 4824 |
| SubjectTerms | Adaptation models Algorithms bandwidth allocation Bandwidths Channel allocation Clients clustering Communication Computation Computational modeling Convergence Correlation Customization Data models Federated learning Federated multi-task learning model pruning Multitasking Optimization sparse sharing Sparsity Training Wireless communication |
| Title | Clustered Federated Multi-Task Learning: A Communication-and-Computation Efficient Sparse Sharing Approach |
| URI | https://ieeexplore.ieee.org/document/10909031 https://www.proquest.com/docview/3217776031 |
| Volume | 24 |
| WOSCitedRecordID | wos001506722000049&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1558-2248 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0017655 issn: 1536-1276 databaseCode: RIE dateStart: 20020101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwELagYoCBZxGFgjywMLh1Yyeu2aqqFVOFRBHdIttxEA-lVR_8fu6ctCpCDGyJEkeWz75H7u77CLmVmVGRdYJZHwsmPQSsYAU9y6x1lhswIB0XyCbUaNSdTPRj1aweemG896H4zLfwMuTys6lb4a-yNhYRao5d07tKJWWz1iZloJJAcQonGIll1CYnyXV7_NKHSDCKWwLR3pDfY8sGBVKVX5o4mJfh0T8ndkwOKz-S9krBn5AdX5ySgy10wTPy3v9cIQyCz-gQISPAq8xoaLhlY7P4oBW06us97dEffSLMFBkr6R7CPR0EnAmYAn2aQSDsKcI8w0jaqxDJ6-R5OBj3H1hFrcBcJOMlKF-PwO08Vx2TCBfnKrYGnMFubMFaaRCZl1ZqHlll0YnhRurcZwmXwqHFF-ekVkwLf0GoNKBreZ6o3ORSGKu1QWzRiIvYWuWTBrlbL3Y6KxE00hB5cJ2CYFIUTFoJpkHquLhb75Xr2iDNtXjS6owtUgHRFGwDeHz5x7Arso9fLyu7mqS2nK_8NdlzX8u3xfwmbJ9vqLjCVQ |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3NS8MwFA8yBfXg58Tp1By8eMiWNWmzeBtjY-IcghN3K0maih90Yx_-_eal3ZiIB28tNDS817yPvvd-P4SueaJEoA0j2oaMcOsSVucFLUm0Npoq50AaxpNNiMGgORrJx2JY3c_CWGt985mtwaWv5Sdjs4BfZXVoIpQUpqY3Q84Dmo9rrYoGIvIkp-4MA7WMWFUlqawPX9ouFwzCGgO8N2D4WPNCnlblly32Dqa7_8-tHaC9IpLErVz1h2jDZkdodw1f8Bi9tz8XAIRgE9wF0AgXVybYj9ySoZp94AJc9fUWt_CPSRGisoTkhA_-Hnc80oTbAn6auFTYYgB6ditxq8AkL6PnbmfY7pGCXIGYgIdzZ34tQLfTVDRUxEyYilArFw42Q-38lXRKs1xzSQMtNIQxVHGZ2iSinBnw-ewElbJxZk8R5spZW5pGIlUpZ0pLqQBdNKAs1FrYqIJulsKOJzmGRuxzDypjp5gYFBMXiqmgMgh37blcrhVUXaonLk7ZLGYunxICeLLP_lh2hbZ7w4d-3L8b3J-jHXhT3udVRaX5dGEv0Jb5mr_Nppf-U_oG8YnFnA |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Clustered+Federated+Multi-Task+Learning%3A+A+Communication-and-Computation+Efficient+Sparse+Sharing+Approach&rft.jtitle=IEEE+transactions+on+wireless+communications&rft.au=Ai%2C+Yuhan&rft.au=Chen%2C+Qimei&rft.au=Zhu%2C+Guangxu&rft.au=Wen%2C+Dingzhu&rft.date=2025-06-01&rft.pub=The+Institute+of+Electrical+and+Electronics+Engineers%2C+Inc.+%28IEEE%29&rft.issn=1536-1276&rft.eissn=1558-2248&rft.volume=24&rft.issue=6&rft.spage=4824&rft.epage=4838&rft_id=info:doi/10.1109%2FTWC.2025.3544318&rft.externalDBID=NO_FULL_TEXT |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1536-1276&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1536-1276&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1536-1276&client=summon |