An asynchronous distributed training algorithm based on Gossip communication and Stochastic Gradient Descent
Cyber–Physical Systems (CPS) applications are playing an increasingly important role in our lives, hence the use of centralized distributed machine learning in CPS to secure applications to start widespread use. However, existing centralized distributed machine learning (ML) algorithms have signific...
Gespeichert in:
| Veröffentlicht in: | Computer communications Jg. 195; S. 416 - 423 |
|---|---|
| Hauptverfasser: | , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
Elsevier B.V
01.11.2022
|
| Schlagworte: | |
| ISSN: | 0140-3664, 1873-703X |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | Cyber–Physical Systems (CPS) applications are playing an increasingly important role in our lives, hence the use of centralized distributed machine learning in CPS to secure applications to start widespread use. However, existing centralized distributed machine learning (ML) algorithms have significant shortcomings in CPS scenarios. As a result, its synchronization algorithm has high latency and sensitivity to drop-off, which affects the security of CPS. Therefore, this paper combining the Gossip protocol with Stochastic Gradient Descent (SGD), this paper proposes a communication framework Gossip Ring SGD (GR-SGD) for machine learning. GR-SGD is decentralized and asynchronous, and solves the problem of long communication waiting time. This paper uses the ImageNet data set and the ResNet model to verify the feasibility of the algorithm and compares it with Ring AllReduce and D-PSGD. Moreover, this paper also indicates that some data redundancy can reduce communication overhead and increase system fault tolerance, it can be better applied to CPS and all kinds of machine learning models. |
|---|---|
| AbstractList | Cyber–Physical Systems (CPS) applications are playing an increasingly important role in our lives, hence the use of centralized distributed machine learning in CPS to secure applications to start widespread use. However, existing centralized distributed machine learning (ML) algorithms have significant shortcomings in CPS scenarios. As a result, its synchronization algorithm has high latency and sensitivity to drop-off, which affects the security of CPS. Therefore, this paper combining the Gossip protocol with Stochastic Gradient Descent (SGD), this paper proposes a communication framework Gossip Ring SGD (GR-SGD) for machine learning. GR-SGD is decentralized and asynchronous, and solves the problem of long communication waiting time. This paper uses the ImageNet data set and the ResNet model to verify the feasibility of the algorithm and compares it with Ring AllReduce and D-PSGD. Moreover, this paper also indicates that some data redundancy can reduce communication overhead and increase system fault tolerance, it can be better applied to CPS and all kinds of machine learning models. |
| Author | Tu, Jun Zhou, Jia Ren, Donglin |
| Author_xml | – sequence: 1 givenname: Jun surname: Tu fullname: Tu, Jun email: tujun@mail.hbut.edu.cn – sequence: 2 givenname: Jia surname: Zhou fullname: Zhou, Jia – sequence: 3 givenname: Donglin surname: Ren fullname: Ren, Donglin |
| BookMark | eNqFkMFqGzEQhkVIoHaSN-hBL7Db0UpeeXsoGLd1AoEcmkBuQh7NxjK2FCS5kLevXOeUQwIDP8zwz8z_Tdl5iIEY-yqgFSD6b9sW475W20HXtTC0IOCMTcRcy0aDfDpnExAKGtn36gub5rwFAKW1nLDdInCbXwNuUgzxkLnzuSS_PhRyvCTrgw_P3O6eY_Jls-drm-sgBr6KOfsXXq_uD8GjLb42bXD8T4m4sbl45KtknadQ-E_KWPWKXYx2l-n6TS_Z4-9fD8ub5u5-dbtc3DUooS8Ndh1o2WlS6MgCkupBzgYapOhQa1RudCOQ00SKQIm5XLuZ7nEu7TCKYS4vmTrtxVS_TDSal-T3Nr0aAeZIzGzNiZg5EjMwmEqs2r6_s6Ev_4MdQew-M_84makG--spmYw1O5LzibAYF_3HC_4BZ2GO9A |
| CitedBy_id | crossref_primary_10_1109_TNSM_2023_3263542 crossref_primary_10_1016_j_neucom_2024_127258 |
| Cites_doi | 10.1016/j.sysconle.2012.06.004 10.1109/JPROC.2018.2817461 10.1007/s10208-012-9135-7 10.1109/TSP.2012.2223692 10.1016/j.comcom.2020.04.039 10.1109/TPDS.2020.3046440 10.1109/TIT.2006.874516 10.1109/ACCESS.2021.3083639 10.1016/j.neucom.2018.11.002 10.1109/TBDATA.2015.2472014 10.1109/COMST.2020.3036778 |
| ContentType | Journal Article |
| Copyright | 2022 Elsevier B.V. |
| Copyright_xml | – notice: 2022 Elsevier B.V. |
| DBID | AAYXX CITATION |
| DOI | 10.1016/j.comcom.2022.09.010 |
| DatabaseName | CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 1873-703X |
| EndPage | 423 |
| ExternalDocumentID | 10_1016_j_comcom_2022_09_010 S0140366422003498 |
| GroupedDBID | --K --M .DC .~1 0R~ 1B1 1~. 1~5 4.4 457 4G. 5GY 5VS 7-5 71M 77K 8P~ 9JN AABNK AACTN AAEDT AAEDW AAIAV AAIKJ AAKOC AALRI AAOAW AAQFI AAXUO AAYFN ABBOA ABFNM ABMAC ABYKQ ACDAQ ACGFS ACRLP ACZNC ADBBV ADEZE ADTZH AEBSH AECPX AEKER AENEX AFKWA AFTJW AGHFR AGUBO AGYEJ AHHHB AHJVU AHZHX AIALX AIEXJ AIKHN AITUG AJOXV ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ AOUOD AXJTR BJAXD BKOJK BLXMC CS3 DU5 EBS EFJIC EFLBG EO8 EO9 EP2 EP3 FDB FIRID FNPLU FYGXN G-Q GBLVA GBOLZ IHE J1W JJJVA KOM LG9 M41 MO0 MS~ N9A O-L O9- OAUVE OZT P-8 P-9 P2P PC. Q38 ROL RPZ RXW SDF SDG SDP SES SPC SPCBC SST SSV SSZ T5K WH7 ZMT ~G- 07C 29F 77I 9DU AAQXK AATTM AAXKI AAYWO AAYXX ABJNI ABWVN ABXDB ACLOT ACNNM ACRPL ACVFH ADCNI ADJOM ADMUD ADNMO AEIPS AEUPX AFJKZ AFPUW AGQPQ AI. AIGII AIIUN AKBMS AKRWK AKYEP ANKPU APXCP ASPBG AVWKF AZFZN CITATION EFKBS EJD F0J FEDTE FGOYB HLZ HVGLF HZ~ R2- SBC SEW TAE UHS VH1 VOH WUQ XPP ZY4 ~HD |
| ID | FETCH-LOGICAL-c306t-c2207327e4cdea0ce460359e9312c77c4dfdf0ed7ee4e04183bd576c83a9f1983 |
| ISICitedReferencesCount | 3 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000869009100005&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0140-3664 |
| IngestDate | Tue Nov 18 22:16:22 EST 2025 Sat Nov 29 07:24:04 EST 2025 Fri Feb 23 02:39:46 EST 2024 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Keywords | Decentralized distribution Asynchronous Gossip Redundancy CPS |
| Language | English |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-c306t-c2207327e4cdea0ce460359e9312c77c4dfdf0ed7ee4e04183bd576c83a9f1983 |
| PageCount | 8 |
| ParticipantIDs | crossref_primary_10_1016_j_comcom_2022_09_010 crossref_citationtrail_10_1016_j_comcom_2022_09_010 elsevier_sciencedirect_doi_10_1016_j_comcom_2022_09_010 |
| PublicationCentury | 2000 |
| PublicationDate | 2022-11-01 2022-11-00 |
| PublicationDateYYYYMMDD | 2022-11-01 |
| PublicationDate_xml | – month: 11 year: 2022 text: 2022-11-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationTitle | Computer communications |
| PublicationYear | 2022 |
| Publisher | Elsevier B.V |
| Publisher_xml | – name: Elsevier B.V |
| References | Chandrasekaran, Recht, Parrilo, Willsky (b4) 2012; 12 Daily, Vishnu, Siegel, Warfel, Amatya (b14) 2018 Abeyrathna, Bhattarai, Goodwin, Gorji, Granmo, Jiao, Saha, Yadav (b46) 2020 Ben-Nun, Hoefler (b18) 2018 Bayatpour, Chakraborty, Subramoni, Lu, Panda (b22) 2017 Li, Andersen, Smola, Yu (b6) 2014 Assran, Loizou, Ballas, Rabbat (b32) 2019 Han, Li, Wang, Liu, Xin, Chen (b16) 2021; 32 Boyd, Ghosh, Prabhakar, Shah (b23) 2006; 52 Li, Andersen, Park, Smola, Ahmed, Josifovski, Long, Shekita, Su (b5) 2014 Xiao, Boyd, Lall (b28) 2005 Thanou, Kokiopoulou, Pu, Frossard (b29) 2013; 61 Seneta (b36) 2008 Chen, Li, Li, Lin, Wang, Wang, Xiao, Xu, Zhang, Zhang (b10) 2015 Georgiev, Gurov (b17) 2019 Colin, Bellet, Salmon, Clémençon (b24) 2016 Blot, Picard, Thome, Cord (b25) 2019; 330 Olowononi, Rawat, Liu (b3) 2021; 23 Yuan, Xu, Zhao, Rong (b27) 2012; 61 Lu, Sa (b42) 2022 Abadi, Barham, Chen, Chen, Davis, Dean, Devin, Ghemawat, Irving, Isard, Kudlur, Levenberg, Monga, Moore, Murray, Steiner, Tucker, Vasudevan, Warden, Wicke, Yu, Zheng (b8) 2016 Kostas, Nota, Thomas (b44) 2019 Xing, Ho, Dai, Kim, Wei, Lee, Zheng, Xie, Kumar, Yu (b7) 2015; 1 Xie, Koyejo, Gupta (b47) 2019 Nedić, Olshevsky, Rabbat (b33) 2018; 106 Blot, Picard, Cord, Thome (b26) 2016 Yang, Li (b45) 2020 Haddadpour, Kamani, Mahdavi, Cadambe (b34) 2019; vol. 97 Mamidala, Kollias, Ward, Artico (b21) 2018 L. (b37) 2013 Seide, Agarwal (b9) 2016 Oguni, Shudo (b15) 2021; 9 Awan, Hamidouche, Hashmi, Panda (b20) 2017 Kempe, Dobra, Gehrke (b38) 2003 Zhang, Ran, Lai, Liu (b12) 2021; 43 Wang, Joshi (b31) 2019 Wang (b1) 2020; 157 Hamidouche, Venkatesh, Awan, Subramoni, Chu, Panda (b19) 2015 Lian, Zhang, Zhang, Hsieh, Zhang, Liu (b39) 2017 Sergeev, Balso (b11) 2018 McMahan, Moore, Ramage, Hampson, y Arcas (b13) 2017 Haddadpour, Yang, Cadambe, Grover (b35) 2018 Paszke, Chintala, Collobert, Kavukcuoglu, Farabet, Bengio, Melvin, Weston, Mariethoz (b41) 2017 Xin, Khan, Kar (b43) 2021 Wang, Sayadi, Sasan, Rafatirad, Mohsenin, Homayoun (b2) 2020 Koloskova, Stich, Jaggi (b30) 2019 He, Zhang, Ren, Sun (b40) 2015 He (10.1016/j.comcom.2022.09.010_b40) 2015 Blot (10.1016/j.comcom.2022.09.010_b26) 2016 Colin (10.1016/j.comcom.2022.09.010_b24) 2016 Thanou (10.1016/j.comcom.2022.09.010_b29) 2013; 61 Wang (10.1016/j.comcom.2022.09.010_b31) 2019 Xing (10.1016/j.comcom.2022.09.010_b7) 2015; 1 Awan (10.1016/j.comcom.2022.09.010_b20) 2017 Seneta (10.1016/j.comcom.2022.09.010_b36) 2008 Wang (10.1016/j.comcom.2022.09.010_b2) 2020 Chen (10.1016/j.comcom.2022.09.010_b10) 2015 Nedić (10.1016/j.comcom.2022.09.010_b33) 2018; 106 Sergeev (10.1016/j.comcom.2022.09.010_b11) 2018 Assran (10.1016/j.comcom.2022.09.010_b32) 2019 Kostas (10.1016/j.comcom.2022.09.010_b44) 2019 Yuan (10.1016/j.comcom.2022.09.010_b27) 2012; 61 Chandrasekaran (10.1016/j.comcom.2022.09.010_b4) 2012; 12 Zhang (10.1016/j.comcom.2022.09.010_b12) 2021; 43 Daily (10.1016/j.comcom.2022.09.010_b14) 2018 Bayatpour (10.1016/j.comcom.2022.09.010_b22) 2017 Haddadpour (10.1016/j.comcom.2022.09.010_b34) 2019; vol. 97 Blot (10.1016/j.comcom.2022.09.010_b25) 2019; 330 Haddadpour (10.1016/j.comcom.2022.09.010_b35) 2018 Georgiev (10.1016/j.comcom.2022.09.010_b17) 2019 Lu (10.1016/j.comcom.2022.09.010_b42) 2022 Xiao (10.1016/j.comcom.2022.09.010_b28) 2005 Wang (10.1016/j.comcom.2022.09.010_b1) 2020; 157 Xin (10.1016/j.comcom.2022.09.010_b43) 2021 Li (10.1016/j.comcom.2022.09.010_b5) 2014 Ben-Nun (10.1016/j.comcom.2022.09.010_b18) 2018 Yang (10.1016/j.comcom.2022.09.010_b45) 2020 Han (10.1016/j.comcom.2022.09.010_b16) 2021; 32 Olowononi (10.1016/j.comcom.2022.09.010_b3) 2021; 23 McMahan (10.1016/j.comcom.2022.09.010_b13) 2017 Mamidala (10.1016/j.comcom.2022.09.010_b21) 2018 L. (10.1016/j.comcom.2022.09.010_b37) 2013 Lian (10.1016/j.comcom.2022.09.010_b39) 2017 Abadi (10.1016/j.comcom.2022.09.010_b8) 2016 Koloskova (10.1016/j.comcom.2022.09.010_b30) 2019 Abeyrathna (10.1016/j.comcom.2022.09.010_b46) 2020 Paszke (10.1016/j.comcom.2022.09.010_b41) 2017 Xie (10.1016/j.comcom.2022.09.010_b47) 2019 Li (10.1016/j.comcom.2022.09.010_b6) 2014 Boyd (10.1016/j.comcom.2022.09.010_b23) 2006; 52 Hamidouche (10.1016/j.comcom.2022.09.010_b19) 2015 Seide (10.1016/j.comcom.2022.09.010_b9) 2016 Oguni (10.1016/j.comcom.2022.09.010_b15) 2021; 9 Kempe (10.1016/j.comcom.2022.09.010_b38) 2003 |
| References_xml | – year: 2019 ident: b31 article-title: Cooperative SGD: A unified framework for the design and analysis of communication-efficient SGD algorithms – year: 2015 ident: b40 article-title: Deep residual learning for image recognition – start-page: 193 year: 2017 end-page: 205 ident: b20 article-title: S-caffe: Co-designing MPI runtimes and caffe for scalable deep learning on modern GPU clusters publication-title: Proceedings of the 22nd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming – start-page: 220 year: 2019 end-page: 227 ident: b17 article-title: Distributed deep learning on heterogeneous computing resources using gossip communication publication-title: Large-Scale Scientific Computing: 12th International Conference, LSSC 2019, Sozopol, Bulgaria, June 10–14, 2019, Revised Selected Papers – volume: 106 start-page: 953 year: 2018 end-page: 976 ident: b33 article-title: Network topology and communication-computation tradeoffs in decentralized optimization publication-title: Proc. IEEE – volume: 157 start-page: 336 year: 2020 end-page: 342 ident: b1 article-title: Research on real-time reliability evaluation of CPS system based on machine learning publication-title: Comput. Commun. – volume: 23 start-page: 524 year: 2021 end-page: 552 ident: b3 article-title: Resilient machine learning for networked cyber physical systems: A survey for machine learning security to securing machine learning for CPS publication-title: IEEE Commun. Surv. Tutor. – year: 2016 ident: b26 article-title: Gossip training for deep learning – volume: 52 start-page: 2508 year: 2006 end-page: 2530 ident: b23 article-title: Randomized gossip algorithms publication-title: IEEE Trans. Inform. Theory – start-page: 63 year: 2005 end-page: 70 ident: b28 article-title: A scheme for robust distributed sensor fusion based on average consensus publication-title: IPSN 2005. Fourth International Symposium on Information Processing in Sensor Networks, 2005. – volume: 61 start-page: 1053 year: 2012 end-page: 1061 ident: b27 article-title: Distributed dual averaging method for multi-agent optimization with quantized communication publication-title: Systems Control Lett. – volume: 32 start-page: 1591 year: 2021 end-page: 1602 ident: b16 article-title: Accelerating gossip-based deep learning in heterogeneous edge computing platforms publication-title: IEEE Trans. Parallel Distrib. Syst. – start-page: 482 year: 2003 end-page: 491 ident: b38 article-title: Gossip-based computation of aggregate information publication-title: 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings – year: 2018 ident: b18 article-title: Demystifying parallel and distributed deep learning: An in-depth concurrency analysis – year: 2008 ident: b36 article-title: Non-negative matrices and Markov chains – start-page: 583 year: 2014 end-page: 598 ident: b5 article-title: Scaling distributed machine learning with the parameter server publication-title: Proceedings of the 11th USENIX Conference on Operating Systems Design and Implementation – start-page: 78 year: 2015 end-page: 87 ident: b19 article-title: Exploiting GPUDirect RDMA in designing high performance OpenSHMEM for NVIDIA GPU clusters publication-title: 2015 IEEE International Conference on Cluster Computing – year: 2021 ident: b43 article-title: A hybrid variance-reduced method for decentralized stochastic non-convex optimization – year: 2016 ident: b24 article-title: Gossip dual averaging for decentralized optimization of pairwise functions – year: 2019 ident: b30 article-title: Decentralized stochastic optimization and gossip algorithms with compressed communication – volume: 43 start-page: 416 year: 2021 end-page: 425 ident: b12 article-title: Performance analysis of distributed deep learning communication architecture publication-title: Comput. Eng. Sci. – year: 2017 ident: b13 article-title: Communication-efficient learning of deep networks from decentralized data – start-page: 265 year: 2016 end-page: 283 ident: b8 article-title: TensorFlow: A system for large-scale machine learning publication-title: Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation – year: 2022 ident: b42 article-title: Optimal complexity in decentralized training – start-page: 19 year: 2014 end-page: 27 ident: b6 article-title: Communication efficient distributed machine learning with the parameter server publication-title: Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 1 – year: 2013 ident: b37 article-title: Distributed average consensus problem based on Gossip algorithm – volume: vol. 97 start-page: 2545 year: 2019 end-page: 2554 ident: b34 article-title: Trading redundancy for communication: Speeding up distributed SGD for non-convex optimization publication-title: Proceedings of the 36th International Conference on Machine Learning – year: 2020 ident: b45 article-title: Buffered asynchronous SGD for Byzantine learning – year: 2019 ident: b44 article-title: Asynchronous coagent networks – volume: 9 start-page: 77873 year: 2021 end-page: 77881 ident: b15 article-title: Communication scheduling for gossip SGD in a wide area network publication-title: IEEE Access – year: 2019 ident: b32 article-title: Stochastic gradient push for distributed deep learning – year: 2018 ident: b14 article-title: GossipGraD: Scalable deep learning using gossip communication based asynchronous gradient descent – year: 2019 ident: b47 article-title: Zeno++: Robust fully asynchronous SGD – volume: 12 start-page: 805 year: 2012 end-page: 849 ident: b4 article-title: The convex geometry of linear inverse problems publication-title: Found. Comput. Math. – year: 2017 ident: b41 article-title: Pytorch: Tensors and dynamic neural networks in python with strong gpu acceleration – year: 2018 ident: b11 article-title: Horovod: fast and easy distributed deep learning in TensorFlow – volume: 61 start-page: 194 year: 2013 end-page: 205 ident: b29 article-title: Distributed average consensus with quantization refinement publication-title: IEEE Trans. Signal Process. – year: 2020 ident: b46 article-title: Massively parallel and asynchronous tsetlin machine architecture supporting almost constant-time scaling – year: 2018 ident: b21 article-title: MXNET-MPI: Embedding MPI parallelism in parameter server task model for scaling deep learning – volume: 1 start-page: 49 year: 2015 end-page: 67 ident: b7 article-title: Petuum: A new platform for distributed machine learning on big data publication-title: IEEE Trans. Big Data – year: 2015 ident: b10 article-title: MXNet: A flexible and efficient machine learning library for heterogeneous distributed systems – start-page: 2135 year: 2016 ident: b9 article-title: CNTK: Microsoft’s open-source deep-learning toolkit publication-title: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining – volume: 330 start-page: 287 year: 2019 end-page: 296 ident: b25 article-title: Distributed optimization for deep learning with gossip exchange publication-title: Neurocomputing – start-page: 181 year: 2020 end-page: 186 ident: b2 article-title: Comprehensive evaluation of machine learning countermeasures for detecting microarchitectural side-channel attacks publication-title: Proceedings of the 2020 on Great Lakes Symposium on VLSI – year: 2017 ident: b39 article-title: Can decentralized algorithms outperform centralized algorithms? A case study for decentralized parallel stochastic gradient descent – year: 2017 ident: b22 article-title: Scalable reduction collectives with data partitioning-based multi-leader design publication-title: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis – start-page: 196 year: 2018 end-page: 203 ident: b35 article-title: Cross-iteration coded computing publication-title: 2018 56th Annual Allerton Conference on Communication, Control, and Computing (Allerton) – year: 2017 ident: 10.1016/j.comcom.2022.09.010_b13 – volume: 61 start-page: 1053 issue: 11 year: 2012 ident: 10.1016/j.comcom.2022.09.010_b27 article-title: Distributed dual averaging method for multi-agent optimization with quantized communication publication-title: Systems Control Lett. doi: 10.1016/j.sysconle.2012.06.004 – start-page: 78 year: 2015 ident: 10.1016/j.comcom.2022.09.010_b19 article-title: Exploiting GPUDirect RDMA in designing high performance OpenSHMEM for NVIDIA GPU clusters – volume: 106 start-page: 953 issue: 5 year: 2018 ident: 10.1016/j.comcom.2022.09.010_b33 article-title: Network topology and communication-computation tradeoffs in decentralized optimization publication-title: Proc. IEEE doi: 10.1109/JPROC.2018.2817461 – year: 2019 ident: 10.1016/j.comcom.2022.09.010_b47 – start-page: 193 year: 2017 ident: 10.1016/j.comcom.2022.09.010_b20 article-title: S-caffe: Co-designing MPI runtimes and caffe for scalable deep learning on modern GPU clusters – year: 2016 ident: 10.1016/j.comcom.2022.09.010_b24 – start-page: 63 year: 2005 ident: 10.1016/j.comcom.2022.09.010_b28 article-title: A scheme for robust distributed sensor fusion based on average consensus – year: 2015 ident: 10.1016/j.comcom.2022.09.010_b40 – volume: 12 start-page: 805 issue: 6 year: 2012 ident: 10.1016/j.comcom.2022.09.010_b4 article-title: The convex geometry of linear inverse problems publication-title: Found. Comput. Math. doi: 10.1007/s10208-012-9135-7 – volume: 61 start-page: 194 year: 2013 ident: 10.1016/j.comcom.2022.09.010_b29 article-title: Distributed average consensus with quantization refinement publication-title: IEEE Trans. Signal Process. doi: 10.1109/TSP.2012.2223692 – year: 2017 ident: 10.1016/j.comcom.2022.09.010_b39 – year: 2020 ident: 10.1016/j.comcom.2022.09.010_b46 – year: 2019 ident: 10.1016/j.comcom.2022.09.010_b30 – year: 2013 ident: 10.1016/j.comcom.2022.09.010_b37 – volume: 157 start-page: 336 year: 2020 ident: 10.1016/j.comcom.2022.09.010_b1 article-title: Research on real-time reliability evaluation of CPS system based on machine learning publication-title: Comput. Commun. doi: 10.1016/j.comcom.2020.04.039 – volume: 32 start-page: 1591 issue: 7 year: 2021 ident: 10.1016/j.comcom.2022.09.010_b16 article-title: Accelerating gossip-based deep learning in heterogeneous edge computing platforms publication-title: IEEE Trans. Parallel Distrib. Syst. doi: 10.1109/TPDS.2020.3046440 – year: 2018 ident: 10.1016/j.comcom.2022.09.010_b21 – volume: 52 start-page: 2508 issue: 6 year: 2006 ident: 10.1016/j.comcom.2022.09.010_b23 article-title: Randomized gossip algorithms publication-title: IEEE Trans. Inform. Theory doi: 10.1109/TIT.2006.874516 – year: 2018 ident: 10.1016/j.comcom.2022.09.010_b18 – start-page: 19 year: 2014 ident: 10.1016/j.comcom.2022.09.010_b6 article-title: Communication efficient distributed machine learning with the parameter server – volume: 9 start-page: 77873 year: 2021 ident: 10.1016/j.comcom.2022.09.010_b15 article-title: Communication scheduling for gossip SGD in a wide area network publication-title: IEEE Access doi: 10.1109/ACCESS.2021.3083639 – year: 2021 ident: 10.1016/j.comcom.2022.09.010_b43 – start-page: 196 year: 2018 ident: 10.1016/j.comcom.2022.09.010_b35 article-title: Cross-iteration coded computing – year: 2017 ident: 10.1016/j.comcom.2022.09.010_b41 – start-page: 482 year: 2003 ident: 10.1016/j.comcom.2022.09.010_b38 article-title: Gossip-based computation of aggregate information – volume: 43 start-page: 416 issue: 03 year: 2021 ident: 10.1016/j.comcom.2022.09.010_b12 article-title: Performance analysis of distributed deep learning communication architecture publication-title: Comput. Eng. Sci. – year: 2016 ident: 10.1016/j.comcom.2022.09.010_b26 – year: 2008 ident: 10.1016/j.comcom.2022.09.010_b36 – start-page: 181 year: 2020 ident: 10.1016/j.comcom.2022.09.010_b2 article-title: Comprehensive evaluation of machine learning countermeasures for detecting microarchitectural side-channel attacks – year: 2018 ident: 10.1016/j.comcom.2022.09.010_b14 – volume: 330 start-page: 287 year: 2019 ident: 10.1016/j.comcom.2022.09.010_b25 article-title: Distributed optimization for deep learning with gossip exchange publication-title: Neurocomputing doi: 10.1016/j.neucom.2018.11.002 – year: 2017 ident: 10.1016/j.comcom.2022.09.010_b22 article-title: Scalable reduction collectives with data partitioning-based multi-leader design – start-page: 583 year: 2014 ident: 10.1016/j.comcom.2022.09.010_b5 article-title: Scaling distributed machine learning with the parameter server – year: 2020 ident: 10.1016/j.comcom.2022.09.010_b45 – year: 2019 ident: 10.1016/j.comcom.2022.09.010_b44 – volume: vol. 97 start-page: 2545 year: 2019 ident: 10.1016/j.comcom.2022.09.010_b34 article-title: Trading redundancy for communication: Speeding up distributed SGD for non-convex optimization – volume: 1 start-page: 49 issue: 2 year: 2015 ident: 10.1016/j.comcom.2022.09.010_b7 article-title: Petuum: A new platform for distributed machine learning on big data publication-title: IEEE Trans. Big Data doi: 10.1109/TBDATA.2015.2472014 – year: 2019 ident: 10.1016/j.comcom.2022.09.010_b31 – start-page: 220 year: 2019 ident: 10.1016/j.comcom.2022.09.010_b17 article-title: Distributed deep learning on heterogeneous computing resources using gossip communication – start-page: 2135 year: 2016 ident: 10.1016/j.comcom.2022.09.010_b9 article-title: CNTK: Microsoft’s open-source deep-learning toolkit – year: 2019 ident: 10.1016/j.comcom.2022.09.010_b32 – year: 2022 ident: 10.1016/j.comcom.2022.09.010_b42 – start-page: 265 year: 2016 ident: 10.1016/j.comcom.2022.09.010_b8 article-title: TensorFlow: A system for large-scale machine learning – volume: 23 start-page: 524 issue: 1 year: 2021 ident: 10.1016/j.comcom.2022.09.010_b3 article-title: Resilient machine learning for networked cyber physical systems: A survey for machine learning security to securing machine learning for CPS publication-title: IEEE Commun. Surv. Tutor. doi: 10.1109/COMST.2020.3036778 – year: 2015 ident: 10.1016/j.comcom.2022.09.010_b10 – year: 2018 ident: 10.1016/j.comcom.2022.09.010_b11 |
| SSID | ssj0004773 |
| Score | 2.386545 |
| Snippet | Cyber–Physical Systems (CPS) applications are playing an increasingly important role in our lives, hence the use of centralized distributed machine learning in... |
| SourceID | crossref elsevier |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 416 |
| SubjectTerms | Asynchronous CPS Decentralized distribution Gossip Redundancy |
| Title | An asynchronous distributed training algorithm based on Gossip communication and Stochastic Gradient Descent |
| URI | https://dx.doi.org/10.1016/j.comcom.2022.09.010 |
| Volume | 195 |
| WOSCitedRecordID | wos000869009100005&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVESC databaseName: Elsevier SD Freedom Collection Journals 2021 customDbUrl: eissn: 1873-703X dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0004773 issn: 0140-3664 databaseCode: AIEXJ dateStart: 19950101 isFulltext: true titleUrlDefault: https://www.sciencedirect.com providerName: Elsevier |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV3db9MwELdKxwN7mPgU2wD5gbcqqImd2n6s0BggNCFUUMVL5NgO7VScqsumTfvnOcdxmqiIL4mXqLLqxL375e7s3v0OoZfFhBWpISTiSV5ElGgZ5VwWEZOploYKk9RdFL58YGdnfD4XHweD21ALc7Vi1vLra7H-r6qGMVC2K539C3W3N4UB-AxKhyuoHa5_pPipHcmLG6sc663Lb9WOGtd1tYLQMjSEGMnVt3KzrBbfR86NafeXwSm4y-Xa5ZhvK0Z8YmdVqoV0fM6j002dIVaBmapZoLqhbegP0b9DG7LPLn0RSAvGr4vSDy1bz_CpsYGlqy223QMJ2MvGvQOJ3UqZ5uAS7P3EM5a_Mt7YckYisDjznjUWacee0njScc3UlybvWH1_AHHulOZSgNyiavLaJmG2z6ddp7O5lSRJTc7D76C9hKWCD9He9N3J_P22rJb5BIWw9FB5WacH7j7r55FNJ1qZ3UcHzTYDTz08HqCBsQ_Rfod88hFaTS3uAgV3gIIDUHALFFwDBZcWe6DgnpoxAAVvgYIDUHADlMfo85uT2eu3UdN7I1KwiawiBdJhJGGGKm3kWBk6cWSPRpA4UYwpqgtdjI1mxlAzpuAYcg1bV8WJFEUsOHmChra05inCKldUaFfbRzTVWvGcyYLURH08Jqk6RCTILVMNMb37lassZCCeZ17amZN2NhYZSPsQRe2stSdm-c33WVBJ1gSXPmjMAEW_nHn0zzOP0b3tC_IMDavNpXmO7qqranmxedHA7QeDBqUw |
| linkProvider | Elsevier |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=An+asynchronous+distributed+training+algorithm+based+on+Gossip+communication+and+Stochastic+Gradient+Descent&rft.jtitle=Computer+communications&rft.au=Tu%2C+Jun&rft.au=Zhou%2C+Jia&rft.au=Ren%2C+Donglin&rft.date=2022-11-01&rft.pub=Elsevier+B.V&rft.issn=0140-3664&rft.eissn=1873-703X&rft.volume=195&rft.spage=416&rft.epage=423&rft_id=info:doi/10.1016%2Fj.comcom.2022.09.010&rft.externalDocID=S0140366422003498 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0140-3664&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0140-3664&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0140-3664&client=summon |