Online Learning from data streams via decentralized and asynchronous SGD
Online Learning (OL) is a sub-field of Machine Learning (ML) which focuses on solving time-sensitive problems through iterative learning from data streams. This emerging field is characterized by the challenge of concept drifts, where the underlying distribution of the incoming data values evolves o...
Uložené v:
| Vydané v: | Future generation computer systems Ročník 175; s. 108052 |
|---|---|
| Hlavní autori: | , |
| Médium: | Journal Article |
| Jazyk: | English |
| Vydavateľské údaje: |
Elsevier B.V
01.02.2026
|
| Predmet: | |
| ISSN: | 0167-739X |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | Online Learning (OL) is a sub-field of Machine Learning (ML) which focuses on solving time-sensitive problems through iterative learning from data streams. This emerging field is characterized by the challenge of concept drifts, where the underlying distribution of the incoming data values evolves over time. Traditional OL algorithms, while efficient and less resource-intensive than conventional ML methods, often fall short in solving non-linear, high-dimensional problems. This prevalent gap has recently led to the integration of Artificial Neural Networks (ANN) into OL settings. These models support real-time inference. However, because they rely on offline training, their performance often degrades during or shortly after concept drifts. In this paper, we extend TensAIR, an online stream-processing engine that we specifically designed for the distributed training of ANN models. Our extensions allow TensAIR to automatically identify concept drifts using the OPTWIN drift detector algorithm, triggering the retraining of the ANN models as soon as drifts are detected. Additionally, we propose a novel decentralized and asynchronous stochastic gradient descent (DASGD) algorithm, which is central to TensAIR’s performance improvements over existing methods, and we formally prove its convergence under the specified conditions. We assessed TensAIR both in single-server and HPC settings, evaluating its distributed training performance over various multi-CPU and multi-GPU scenarios. As result, we show TensAIR to converge within the best known theoretical bounds while achieving up to 78× higher sustainable throughput than state-of-the-art baselines. Based on our results, we expect to inspire further research and applications exploiting the distributed training of ANN models in HPC platforms for a wide range of OL settings. |
|---|---|
| AbstractList | Online Learning (OL) is a sub-field of Machine Learning (ML) which focuses on solving time-sensitive problems through iterative learning from data streams. This emerging field is characterized by the challenge of concept drifts, where the underlying distribution of the incoming data values evolves over time. Traditional OL algorithms, while efficient and less resource-intensive than conventional ML methods, often fall short in solving non-linear, high-dimensional problems. This prevalent gap has recently led to the integration of Artificial Neural Networks (ANN) into OL settings. These models support real-time inference. However, because they rely on offline training, their performance often degrades during or shortly after concept drifts. In this paper, we extend TensAIR, an online stream-processing engine that we specifically designed for the distributed training of ANN models. Our extensions allow TensAIR to automatically identify concept drifts using the OPTWIN drift detector algorithm, triggering the retraining of the ANN models as soon as drifts are detected. Additionally, we propose a novel decentralized and asynchronous stochastic gradient descent (DASGD) algorithm, which is central to TensAIR’s performance improvements over existing methods, and we formally prove its convergence under the specified conditions. We assessed TensAIR both in single-server and HPC settings, evaluating its distributed training performance over various multi-CPU and multi-GPU scenarios. As result, we show TensAIR to converge within the best known theoretical bounds while achieving up to 78× higher sustainable throughput than state-of-the-art baselines. Based on our results, we expect to inspire further research and applications exploiting the distributed training of ANN models in HPC platforms for a wide range of OL settings. |
| ArticleNumber | 108052 |
| Author | Theobald, Martin Tosi, Mauro Dalle Lucca |
| Author_xml | – sequence: 1 givenname: Mauro Dalle Lucca orcidid: 0000-0002-0218-2413 surname: Tosi fullname: Tosi, Mauro Dalle Lucca email: mauro.dalle-lucca-tosi@list.lu organization: Human Centered AI, Data & Software Unit, Luxembourg Institute of Science and Technology, Esch-sur-Alzette, Luxembourg – sequence: 2 givenname: Martin surname: Theobald fullname: Theobald, Martin email: martin.theobald@uni.lu organization: Department of Computer Science, University of Luxembourg, Esch-sur-Alzette, Luxembourg |
| BookMark | eNp9kFFLwzAUhfMwwW36D3zIH-hMmrapL4JM3YTCHlTwLSQ3N5rRpZK0g_nr7ajPPlwO3MM5HL4FmYUuICE3nK0449XtfuWGfoi4yllejq-alfmMzEdLZlLcfVySRUp7xhiXgs_JdhdaH5A2qGPw4ZO62B2o1b2mqY-oD4kevaYWAUMfdet_0FIdxkunAF-xC92Q6Ovm8YpcON0mvP7TJXl_fnpbb7Nmt3lZPzQZCCb6jFcoS2E45haKsjJYWckAgUEl0Bmba-lYbUwtQPLCFELb0gCvhTMlA-7EkhRTL8QupYhOfUd_0PGkOFNnAmqvJgLqTEBNBMbY_RTDcdvRY1QJPAZA6yNCr2zn_y_4BaSMa98 |
| Cites_doi | 10.1109/TAC.2020.2981035 10.1109/JSTSP.2011.2118740 10.1016/j.jpdc.2022.04.022 10.1016/j.jpdc.2020.11.005 10.1145/3320060 10.1109/Allerton49937.2022.9929409 10.1145/3647750.3647762 10.1145/3363554 10.1109/ICDEW61823.2024.00049 10.1109/SBAC-PAD49847.2020.00018 10.1007/s11042-021-11007-7 10.1093/beheco/ark016 10.1023/A:1010933404324 10.1186/s40537-019-0206-3 10.1109/MSP.2017.2765695 10.1016/j.tics.2020.09.004 10.1186/s40537-014-0007-7 10.1016/j.neucom.2021.04.112 10.1287/opre.2020.612 |
| ContentType | Journal Article |
| Copyright | 2025 The Authors |
| Copyright_xml | – notice: 2025 The Authors |
| DBID | 6I. AAFTH AAYXX CITATION |
| DOI | 10.1016/j.future.2025.108052 |
| DatabaseName | ScienceDirect Open Access Titles Elsevier:ScienceDirect:Open Access CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| ExternalDocumentID | 10_1016_j_future_2025_108052 S0167739X25003474 |
| GroupedDBID | --K --M -~X .DC .~1 0R~ 1B1 1~. 1~5 29H 4.4 457 4G. 5GY 5VS 6I. 7-5 71M 8P~ 9DU 9JN AAEDT AAEDW AAFTH AAIKJ AAKOC AALRI AAOAW AAQFI AAQXK AATTM AAXKI AAXUO AAYFN AAYWO ABBOA ABDPE ABFNM ABJNI ABMAC ABWVN ABXDB ACDAQ ACGFS ACLOT ACNNM ACRLP ACRPL ACZNC ADBBV ADEZE ADJOM ADMUD ADNMO AEBSH AEIPS AEKER AFJKZ AFTJW AGHFR AGQPQ AGUBO AGYEJ AHHHB AHZHX AIALX AIEXJ AIIUN AIKHN AITUG ALMA_UNASSIGNED_HOLDINGS AMRAJ ANKPU AOUOD APXCP ASPBG AVWKF AXJTR AZFZN BKOJK BLXMC CS3 EBS EFJIC EFKBS EFLBG EJD EO8 EO9 EP2 EP3 F5P FDB FEDTE FGOYB FIRID FNPLU FYGXN G-Q GBLVA GBOLZ HLZ HVGLF HZ~ IHE J1W KOM LG9 M41 MO0 MS~ N9A O-L O9- OAUVE OZT P-8 P-9 PC. Q38 R2- ROL RPZ SBC SDF SDG SES SEW SPC SPCBC SSV SSZ T5K UHS WUQ XPP ZMT ~G- ~HD AAYXX CITATION |
| ID | FETCH-LOGICAL-c303t-16e753b1e2dc456be6d70cec0c63efbd2a7f08bb83c714b43ad5bc183fb50c1f3 |
| ISICitedReferencesCount | 1 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001554673400001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0167-739X |
| IngestDate | Sat Nov 29 06:57:47 EST 2025 Sat Nov 29 17:08:20 EST 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Keywords | Online Learning Data streams Concept drifts Asynchronous & decentralized SGD Artificial neural networks |
| Language | English |
| License | This is an open access article under the CC BY license. |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-c303t-16e753b1e2dc456be6d70cec0c63efbd2a7f08bb83c714b43ad5bc183fb50c1f3 |
| ORCID | 0000-0002-0218-2413 |
| OpenAccessLink | https://dx.doi.org/10.1016/j.future.2025.108052 |
| ParticipantIDs | crossref_primary_10_1016_j_future_2025_108052 elsevier_sciencedirect_doi_10_1016_j_future_2025_108052 |
| PublicationCentury | 2000 |
| PublicationDate | February 2026 2026-02-00 |
| PublicationDateYYYYMMDD | 2026-02-01 |
| PublicationDate_xml | – month: 02 year: 2026 text: February 2026 |
| PublicationDecade | 2020 |
| PublicationTitle | Future generation computer systems |
| PublicationYear | 2026 |
| Publisher | Elsevier B.V |
| Publisher_xml | – name: Elsevier B.V |
| References | (b66) 2022 Zhang, Tan, Li (b21) 2018 Mahbobi, Tiemann (b56) 2015 (b13) 2022 Mishchenko, Bach, Even, Woodworth (b36) 2022; 35 M.T. Toghani, C.A. Uribe, Unbounded Gradients in Federated Learning with Buffered Asynchronous Aggregation, in: 2022 58th Annual Allerton Conference on Communication, Control, and Computing (Allerton), 2022, pp. 1–8. Ben-Nun, Hoefler (b33) 2019; 52 Cohen, Daniely, Drori, Koren, Schain (b37) 2021; 34 Najafabadi, Villanustre, Khoshgoftaar, Seliya, Wald, Muharemagic (b12) 2015; 2 Eedala (b67) 2023 Lu, Liu, Dong, Gu, Gama, Zhang (b9) 2018; 31 Wang, Lu, Wang, Zhang (b22) 2022; 81 Varrette, Cartiaux, Peter, Kieffer, Valette, Olloh (b61) 2022 Duchi, Hazan, Singer (b27) 2011; 12 Dalle Lucca Tosi (b52) 2024 Verma (b57) 2025; 10 Venugopal, Theobald, Tassetti, Chaychi, Tawakuli (b20) 2022 Carbone, Katsifodimos, Ewen, Markl, Haridi, Tzoumas (b18) 2015; 36 Recht, Re, Wright, Niu (b34) 2011; 24 Hariri, Fredericks, Bowers (b2) 2019; 6 Paladini, Bernasconi de Luca, Carminati, Polino, Trovò, Zanero (b4) 2023 Thompson, Ge, Manso (b3) 2022 Ouyang, Dong, Xu, Xiao (b29) 2021; 149 Simonyan, Zisserman (b64) 2015 Chen, Liu, Hong (b5) 2023 Webb (b10) 2010 . Kingma, Ba (b26) 2014 V.E. Venugopal, M. Theobald, S. Chaychi, A. Tawakuli, AIR: A light-weight yet high-performance dataflow engine based on asynchronous iterative routing, in: 2020 IEEE 32nd International Symposium on Computer Architecture and High Performance Computing, SBAC-PAD, 2020, pp. 51–58. Ram, Nedić, Veeravalli (b47) 2009 Le, Yang (b65) 2015; 7 Wu, Yuan, Ling, Yin, Sayed (b44) 2017; 4 Lian, Zhang, Zhang, Liu (b48) 2018 Even, Hendrikx, Massoulié (b45) 2024 Srivastava, Nedic (b46) 2011; 5 Krizhevsky, Hinton (b63) 2009 A. Koloskova, N. Loizou, S. Boreiri, M. Jaggi, S. Stich, A unified theory of decentralized SGD with changing topology and local updates, in: International Conference on Machine Learning, 2020, pp. 5381–5393. Jiang, Zhang, Gu, Zhu (b50) 2021; 34 Arjevani, Shamir, Srebro (b38) 2020 Assran, Rabbat (b43) 2020; 66 M. Bornstein, T. Rabbani, E.Z. Wang, A. Bedi, F. Huang, SWIFT: Rapid Decentralized Federated Learning via Wait-Free Model Communication, in: The Eleventh International Conference on Learning Representations, 2023. Goodfellow, Bengio, Courville (b8) 2016 Mahbobi, Tiemann (b54) 2016 Ruxton (b53) 2006; 17 Basterrech, Kasprzak, Platoš, Woźniak (b7) 2023 Xie, Koyejo, Gupta (b39) 2019 Bifet, Gavalda (b55) 2007 Dean, Corrado, Monga, Chen, Devin, Mao, Ranzato, Senior, Tucker, Yang (b59) 2012; 25 Assran, Loizou, Ballas, Rabbat (b49) 2019 Koloskova, Stich, Jaggi (b35) 2022 Kreps, Narkhede, Rao (b17) 2011; vol. 11 Hadsell, Rao, Rusu, Pascanu (b24) 2020; 24 Hoi, Sahoo, Lu, Zhao (b1) 2021; 459 Tosi, Venugopal, Theobald (b60) 2022 (b14) 2022 Nguyen, Ly, Ho, Al-Ansari, Le, Tran, Prakash, Pham (b23) 2021; 2021 Tantisripreecha, Soonthomphisaj (b6) 2018 Lian, Zhang, Zhang, Hsieh, Zhang, Liu (b31) 2017; 30 J. Nguyen, K. Malik, H. Zhan, A. Yousefpour, M. Rabbat, M. Malek, D. Huba, Federated learning with buffered asynchronous aggregation, in: International Conference on Artificial Intelligence and Statistics, 2022, pp. 3581–3607. M.D.L. Tosi, V.E. Venugopal, M. Theobald, TensAIR: Real-Time Training of Neural Networks from Data-streams, in: Proceedings of the 2024 8th International Conference on Machine Learning and Soft Computing, 2024, pp. 73–82. Agarwal, Duchi (b58) 2011; 24 LeCun, Bottou, Orr, Müller (b25) 2002 Cheng, Wang, Zhou, Zhang (b28) 2018; 35 Breiman (b11) 2001; 45 M.D.L. Tosi, M. Theobald, OPTWIN: Drift identification with optimal sub-windows, in: 2024 IEEE 40th International Conference on Data Engineering Workshops (ICDEW 2024), 2024, pp. 331–337 Mayer, Jacobsen (b30) 2020; 53 Wu, Liu, Magnusson, Johansson (b42) 2023 (b62) 2022 Cohen (10.1016/j.future.2025.108052_b37) 2021; 34 Mayer (10.1016/j.future.2025.108052_b30) 2020; 53 Hadsell (10.1016/j.future.2025.108052_b24) 2020; 24 (10.1016/j.future.2025.108052_b66) 2022 Simonyan (10.1016/j.future.2025.108052_b64) 2015 10.1016/j.future.2025.108052_b40 10.1016/j.future.2025.108052_b41 Krizhevsky (10.1016/j.future.2025.108052_b63) 2009 Verma (10.1016/j.future.2025.108052_b57) 2025; 10 (10.1016/j.future.2025.108052_b62) 2022 Tantisripreecha (10.1016/j.future.2025.108052_b6) 2018 Kingma (10.1016/j.future.2025.108052_b26) 2014 Paladini (10.1016/j.future.2025.108052_b4) 2023 Hariri (10.1016/j.future.2025.108052_b2) 2019; 6 Mahbobi (10.1016/j.future.2025.108052_b56) 2015 Dean (10.1016/j.future.2025.108052_b59) 2012; 25 10.1016/j.future.2025.108052_b15 Thompson (10.1016/j.future.2025.108052_b3) 2022 10.1016/j.future.2025.108052_b16 Wu (10.1016/j.future.2025.108052_b42) 2023 10.1016/j.future.2025.108052_b51 Tosi (10.1016/j.future.2025.108052_b60) 2022 Breiman (10.1016/j.future.2025.108052_b11) 2001; 45 Jiang (10.1016/j.future.2025.108052_b50) 2021; 34 Lian (10.1016/j.future.2025.108052_b48) 2018 Cheng (10.1016/j.future.2025.108052_b28) 2018; 35 Nguyen (10.1016/j.future.2025.108052_b23) 2021; 2021 Ram (10.1016/j.future.2025.108052_b47) 2009 Hoi (10.1016/j.future.2025.108052_b1) 2021; 459 Eedala (10.1016/j.future.2025.108052_b67) 2023 Basterrech (10.1016/j.future.2025.108052_b7) 2023 Ouyang (10.1016/j.future.2025.108052_b29) 2021; 149 Lian (10.1016/j.future.2025.108052_b31) 2017; 30 Ben-Nun (10.1016/j.future.2025.108052_b33) 2019; 52 Najafabadi (10.1016/j.future.2025.108052_b12) 2015; 2 Srivastava (10.1016/j.future.2025.108052_b46) 2011; 5 (10.1016/j.future.2025.108052_b14) 2022 Kreps (10.1016/j.future.2025.108052_b17) 2011; vol. 11 Mahbobi (10.1016/j.future.2025.108052_b54) 2016 10.1016/j.future.2025.108052_b19 Wu (10.1016/j.future.2025.108052_b44) 2017; 4 Lu (10.1016/j.future.2025.108052_b9) 2018; 31 Duchi (10.1016/j.future.2025.108052_b27) 2011; 12 Assran (10.1016/j.future.2025.108052_b49) 2019 Varrette (10.1016/j.future.2025.108052_b61) 2022 Chen (10.1016/j.future.2025.108052_b5) 2023 Assran (10.1016/j.future.2025.108052_b43) 2020; 66 10.1016/j.future.2025.108052_b32 Xie (10.1016/j.future.2025.108052_b39) 2019 Webb (10.1016/j.future.2025.108052_b10) 2010 Even (10.1016/j.future.2025.108052_b45) 2024 Recht (10.1016/j.future.2025.108052_b34) 2011; 24 Venugopal (10.1016/j.future.2025.108052_b20) 2022 Koloskova (10.1016/j.future.2025.108052_b35) 2022 Arjevani (10.1016/j.future.2025.108052_b38) 2020 Le (10.1016/j.future.2025.108052_b65) 2015; 7 Ruxton (10.1016/j.future.2025.108052_b53) 2006; 17 Wang (10.1016/j.future.2025.108052_b22) 2022; 81 Zhang (10.1016/j.future.2025.108052_b21) 2018 Dalle Lucca Tosi (10.1016/j.future.2025.108052_b52) 2024 Carbone (10.1016/j.future.2025.108052_b18) 2015; 36 Goodfellow (10.1016/j.future.2025.108052_b8) 2016 Mishchenko (10.1016/j.future.2025.108052_b36) 2022; 35 Bifet (10.1016/j.future.2025.108052_b55) 2007 (10.1016/j.future.2025.108052_b13) 2022 LeCun (10.1016/j.future.2025.108052_b25) 2002 Agarwal (10.1016/j.future.2025.108052_b58) 2011; 24 |
| References_xml | – year: 2016 ident: b8 article-title: Deep Learning – reference: M. Bornstein, T. Rabbani, E.Z. Wang, A. Bedi, F. Huang, SWIFT: Rapid Decentralized Federated Learning via Wait-Free Model Communication, in: The Eleventh International Conference on Learning Representations, 2023. – volume: 24 year: 2011 ident: b58 article-title: Distributed delayed stochastic optimization publication-title: Adv. Neural Inf. Process. Syst. – volume: 17 start-page: 688 year: 2006 end-page: 690 ident: b53 article-title: The unequal variance t-test is an underused alternative to student’s t-test and the Mann–Whitney U test publication-title: Behav. Ecol. – volume: 7 start-page: 3 year: 2015 ident: b65 article-title: Tiny imagenet visual recognition challenge publication-title: CS 231N – volume: 35 start-page: 126 year: 2018 end-page: 136 ident: b28 article-title: Model compression and acceleration for deep neural networks: The principles, progress, and challenges publication-title: IEEE Signal Process. Mag. – year: 2022 ident: b13 article-title: Deep learning on flink – year: 2019 ident: b39 article-title: Asynchronous federated optimization – year: 2022 ident: b20 article-title: Targeting a light-weight and multi-channel approach for distributed stream processing publication-title: J. Parallel Distrib. Comput. – start-page: 111 year: 2020 end-page: 132 ident: b38 article-title: A tight convergence analysis for stochastic gradient descent with delayed updates publication-title: Algorithmic Learning Theory – volume: 5 start-page: 772 year: 2011 end-page: 790 ident: b46 article-title: Distributed asynchronous constrained stochastic optimization publication-title: IEEE J. Sel. Top. Signal Process. – start-page: 443 year: 2007 end-page: 448 ident: b55 article-title: Learning from time-changing data with adaptive windowing publication-title: Proceedings of the 2007 SIAM International Conference on Data Mining – volume: vol. 11 start-page: 1 year: 2011 end-page: 7 ident: b17 article-title: Kafka: A distributed messaging system for log processing publication-title: Proceedings of the NetDB – year: 2009 ident: b63 article-title: Learning multiple layers of features from tiny images – volume: 6 start-page: 1 year: 2019 end-page: 16 ident: b2 article-title: Uncertainty in big data analytics: survey, opportunities, and challenges publication-title: J. Big Data – start-page: 55 year: 2016 end-page: 63 ident: b54 article-title: Introductory business statistics with interactive spreadsheets-1st Canadian edition – volume: 66 start-page: 168 year: 2020 end-page: 183 ident: b43 article-title: Asynchronous gradient push publication-title: IEEE Trans. Autom. Control – start-page: 713 year: 2010 end-page: 714 ident: b10 article-title: Naïve Bayes publication-title: Encyclopedia of Machine Learning – year: 2015 ident: b56 article-title: Regression basics publication-title: Introductory Business Statistics with Interactive Spreadsheets-1st Canadian Edition – reference: M.T. Toghani, C.A. Uribe, Unbounded Gradients in Federated Learning with Buffered Asynchronous Aggregation, in: 2022 58th Annual Allerton Conference on Communication, Control, and Computing (Allerton), 2022, pp. 1–8. – volume: 81 start-page: 41611 year: 2022 end-page: 41660 ident: b22 article-title: A review on extreme learning machine publication-title: Multimedia Tools Appl. – start-page: 314 year: 2022 end-page: 315 ident: b60 article-title: Convergence-time analysis of asynchronous distributed artificial neural networks publication-title: 5th Joint International Conference on Data Science & Management of Data – reference: M.D.L. Tosi, V.E. Venugopal, M. Theobald, TensAIR: Real-Time Training of Neural Networks from Data-streams, in: Proceedings of the 2024 8th International Conference on Machine Learning and Soft Computing, 2024, pp. 73–82. – start-page: 1 year: 2023 end-page: 10 ident: b7 article-title: A continual learning system with self domain shift adaptation for fake news detection publication-title: 2023 IEEE 10th International Conference on Data Science and Advanced Analytics – start-page: 9 year: 2002 end-page: 50 ident: b25 article-title: Efficient backprop publication-title: Neural Networks: Tricks of the Trade – volume: 12 year: 2011 ident: b27 article-title: Adaptive subgradient methods for online learning and stochastic optimization publication-title: J. Mach. Learn. Res. – year: 2022 ident: b61 article-title: Management of an academic HPC & research computing facility: The ULHPC experience 2.0 publication-title: Proc. of the 6th ACM High Performance Computing and Cluster Technologies Conf. – volume: 10 start-page: 34 year: 2025 end-page: 46 ident: b57 article-title: A comprehensive framework for residual analysis in regression and machine learning publication-title: J. Inf. Syst. Eng. Manag. – year: 2022 ident: b3 article-title: The importance of (exponentially more) computing power – volume: 30 year: 2017 ident: b31 article-title: Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent publication-title: Adv. Neural Inf. Process. Syst. – volume: 52 start-page: 1 year: 2019 end-page: 43 ident: b33 article-title: Demystifying parallel and distributed deep learning: An in-depth concurrency analysis publication-title: ACM Comput. Surv. – reference: J. Nguyen, K. Malik, H. Zhan, A. Yousefpour, M. Rabbat, M. Malek, D. Huba, Federated learning with buffered asynchronous aggregation, in: International Conference on Artificial Intelligence and Statistics, 2022, pp. 3581–3607. – volume: 2021 start-page: 1 year: 2021 end-page: 15 ident: b23 article-title: Influence of data splitting on performance of machine learning models in prediction of shear strength of soil publication-title: Math. Probl. Eng. – volume: 36 year: 2015 ident: b18 article-title: Apache Flink: Stream and batch processing in a single engine publication-title: Bull. the IEEE Comput. Soc. Tech. Comm. Data Eng. – year: 2024 ident: b45 article-title: Asynchronous speedup in decentralized optimization publication-title: IEEE Trans. Autom. Control – year: 2022 ident: b62 article-title: Convolutional neural network (CNN): Tensorflow core – year: 2014 ident: b26 article-title: Adam: A method for stochastic optimization – volume: 2 start-page: 1 year: 2015 end-page: 21 ident: b12 article-title: Deep learning applications and challenges in big data analytics publication-title: J. Big Data – volume: 24 year: 2011 ident: b34 article-title: Hogwild!: A lock-free approach to parallelizing stochastic gradient descent publication-title: Adv. Neural Inf. Process. Syst. – volume: 4 start-page: 293 year: 2017 end-page: 307 ident: b44 article-title: Decentralized consensus optimization with asynchrony and delays publication-title: IEEE Trans. Signal Inf. Process. over Netw. – volume: 24 start-page: 1028 year: 2020 end-page: 1040 ident: b24 article-title: Embracing change: Continual learning in deep neural networks publication-title: Trends Cogn. Sci. – start-page: 1082 year: 2023 end-page: 1087 ident: b42 article-title: Delay-agnostic asynchronous distributed optimization publication-title: 2023 62nd IEEE Conference on Decision and Control – start-page: 344 year: 2019 end-page: 353 ident: b49 article-title: Stochastic gradient push for distributed deep learning publication-title: International Conference on Machine Learning – year: 2023 ident: b5 article-title: An online learning approach to dynamic pricing and capacity sizing in service systems publication-title: Oper. Res. – volume: 31 start-page: 2346 year: 2018 end-page: 2363 ident: b9 article-title: Learning under concept drift: A review publication-title: IEEE Trans. Knowl. Data Eng. – year: 2022 ident: b35 article-title: Sharper convergence guarantees for asynchronous SGD for distributed and federated learning publication-title: Advances in Neural Information Processing Systems – volume: 34 start-page: 20185 year: 2021 end-page: 20196 ident: b50 article-title: Asynchronous decentralized online learning publication-title: Adv. Neural Inf. Process. Syst. – start-page: 3043 year: 2018 end-page: 3052 ident: b48 article-title: Asynchronous decentralized parallel stochastic gradient descent publication-title: International Conference on Machine Learning – year: 2015 ident: b64 article-title: Very deep convolutional networks for large-scale image recognition publication-title: 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings – volume: 25 year: 2012 ident: b59 article-title: Large scale distributed deep networks publication-title: Adv. Neural Inf. Process. Syst. – start-page: 45 year: 2018 end-page: 50 ident: b21 article-title: A survey of online sequential extreme learning machine publication-title: 2018 5th International Conference on Control, Decision and Information Technologies – year: 2023 ident: b67 article-title: Flight delay data — kaggle.com – volume: 45 start-page: 5 year: 2001 end-page: 32 ident: b11 article-title: Random forests publication-title: Mach. Learn. – reference: A. Koloskova, N. Loizou, S. Boreiri, M. Jaggi, S. Stich, A unified theory of decentralized SGD with changing topology and local updates, in: International Conference on Machine Learning, 2020, pp. 5381–5393. – volume: 53 start-page: 1 year: 2020 end-page: 37 ident: b30 article-title: Scalable deep learning on distributed infrastructures: Challenges, techniques, and tools publication-title: ACM Comput. Surv. – reference: M.D.L. Tosi, M. Theobald, OPTWIN: Drift identification with optimal sub-windows, in: 2024 IEEE 40th International Conference on Data Engineering Workshops (ICDEW 2024), 2024, pp. 331–337, – start-page: 275 year: 2023 end-page: 292 ident: b4 article-title: Advancing fraud detection systems through online learning publication-title: Joint European Conference on Machine Learning and Knowledge Discovery in Databases – year: 2022 ident: b14 article-title: Robust machine learning on streaming data using kafka and Tensorflow-IO – start-page: 3581 year: 2009 end-page: 3586 ident: b47 article-title: Asynchronous gossip algorithms for stochastic optimization publication-title: Proceedings of the 48h IEEE Conference on Decision and Control (CDC) Held Jointly with 2009 28th Chinese Control Conference – year: 2024 ident: b52 article-title: Online Learning Using Distributed Neural Networks – year: 2022 ident: b66 article-title: TensorFlow – reference: . – volume: 34 start-page: 9024 year: 2021 end-page: 9035 ident: b37 article-title: Asynchronous stochastic optimization robust to arbitrary delays publication-title: Adv. Neural Inf. Process. Syst. – start-page: 135 year: 2018 end-page: 139 ident: b6 article-title: Stock market movement prediction using LDA-online learning model publication-title: 2018 19th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing – volume: 149 start-page: 52 year: 2021 end-page: 65 ident: b29 article-title: Communication optimization strategies for distributed deep neural network training: A survey publication-title: J. Parallel Distrib. Comput. – volume: 459 start-page: 249 year: 2021 end-page: 289 ident: b1 article-title: Online learning: A comprehensive survey publication-title: Neurocomputing – reference: V.E. Venugopal, M. Theobald, S. Chaychi, A. Tawakuli, AIR: A light-weight yet high-performance dataflow engine based on asynchronous iterative routing, in: 2020 IEEE 32nd International Symposium on Computer Architecture and High Performance Computing, SBAC-PAD, 2020, pp. 51–58. – volume: 35 start-page: 420 year: 2022 end-page: 433 ident: b36 article-title: Asynchronous SGD beats minibatch SGD under arbitrary delays publication-title: Adv. Neural Inf. Process. Syst. – volume: 66 start-page: 168 issue: 1 year: 2020 ident: 10.1016/j.future.2025.108052_b43 article-title: Asynchronous gradient push publication-title: IEEE Trans. Autom. Control doi: 10.1109/TAC.2020.2981035 – start-page: 3581 year: 2009 ident: 10.1016/j.future.2025.108052_b47 article-title: Asynchronous gossip algorithms for stochastic optimization – year: 2016 ident: 10.1016/j.future.2025.108052_b8 – year: 2015 ident: 10.1016/j.future.2025.108052_b64 article-title: Very deep convolutional networks for large-scale image recognition – volume: 24 year: 2011 ident: 10.1016/j.future.2025.108052_b58 article-title: Distributed delayed stochastic optimization publication-title: Adv. Neural Inf. Process. Syst. – ident: 10.1016/j.future.2025.108052_b40 – year: 2022 ident: 10.1016/j.future.2025.108052_b66 – start-page: 3043 year: 2018 ident: 10.1016/j.future.2025.108052_b48 article-title: Asynchronous decentralized parallel stochastic gradient descent – volume: 12 issue: 7 year: 2011 ident: 10.1016/j.future.2025.108052_b27 article-title: Adaptive subgradient methods for online learning and stochastic optimization publication-title: J. Mach. Learn. Res. – year: 2022 ident: 10.1016/j.future.2025.108052_b61 article-title: Management of an academic HPC & research computing facility: The ULHPC experience 2.0 – start-page: 344 year: 2019 ident: 10.1016/j.future.2025.108052_b49 article-title: Stochastic gradient push for distributed deep learning – volume: 24 year: 2011 ident: 10.1016/j.future.2025.108052_b34 article-title: Hogwild!: A lock-free approach to parallelizing stochastic gradient descent publication-title: Adv. Neural Inf. Process. Syst. – year: 2024 ident: 10.1016/j.future.2025.108052_b45 article-title: Asynchronous speedup in decentralized optimization publication-title: IEEE Trans. Autom. Control – volume: 5 start-page: 772 issue: 4 year: 2011 ident: 10.1016/j.future.2025.108052_b46 article-title: Distributed asynchronous constrained stochastic optimization publication-title: IEEE J. Sel. Top. Signal Process. doi: 10.1109/JSTSP.2011.2118740 – year: 2022 ident: 10.1016/j.future.2025.108052_b13 – volume: 2021 start-page: 1 year: 2021 ident: 10.1016/j.future.2025.108052_b23 article-title: Influence of data splitting on performance of machine learning models in prediction of shear strength of soil publication-title: Math. Probl. Eng. – year: 2022 ident: 10.1016/j.future.2025.108052_b20 article-title: Targeting a light-weight and multi-channel approach for distributed stream processing publication-title: J. Parallel Distrib. Comput. doi: 10.1016/j.jpdc.2022.04.022 – start-page: 45 year: 2018 ident: 10.1016/j.future.2025.108052_b21 article-title: A survey of online sequential extreme learning machine – year: 2022 ident: 10.1016/j.future.2025.108052_b3 – year: 2009 ident: 10.1016/j.future.2025.108052_b63 – volume: 149 start-page: 52 year: 2021 ident: 10.1016/j.future.2025.108052_b29 article-title: Communication optimization strategies for distributed deep neural network training: A survey publication-title: J. Parallel Distrib. Comput. doi: 10.1016/j.jpdc.2020.11.005 – volume: 35 start-page: 420 year: 2022 ident: 10.1016/j.future.2025.108052_b36 article-title: Asynchronous SGD beats minibatch SGD under arbitrary delays publication-title: Adv. Neural Inf. Process. Syst. – year: 2024 ident: 10.1016/j.future.2025.108052_b52 – start-page: 1 year: 2023 ident: 10.1016/j.future.2025.108052_b7 article-title: A continual learning system with self domain shift adaptation for fake news detection – volume: 52 start-page: 1 issue: 4 year: 2019 ident: 10.1016/j.future.2025.108052_b33 article-title: Demystifying parallel and distributed deep learning: An in-depth concurrency analysis publication-title: ACM Comput. Surv. doi: 10.1145/3320060 – ident: 10.1016/j.future.2025.108052_b41 doi: 10.1109/Allerton49937.2022.9929409 – year: 2022 ident: 10.1016/j.future.2025.108052_b14 – ident: 10.1016/j.future.2025.108052_b15 doi: 10.1145/3647750.3647762 – volume: 53 start-page: 1 issue: 1 year: 2020 ident: 10.1016/j.future.2025.108052_b30 article-title: Scalable deep learning on distributed infrastructures: Challenges, techniques, and tools publication-title: ACM Comput. Surv. doi: 10.1145/3363554 – ident: 10.1016/j.future.2025.108052_b16 doi: 10.1109/ICDEW61823.2024.00049 – volume: vol. 11 start-page: 1 year: 2011 ident: 10.1016/j.future.2025.108052_b17 article-title: Kafka: A distributed messaging system for log processing – year: 2022 ident: 10.1016/j.future.2025.108052_b35 article-title: Sharper convergence guarantees for asynchronous SGD for distributed and federated learning – year: 2023 ident: 10.1016/j.future.2025.108052_b67 – start-page: 275 year: 2023 ident: 10.1016/j.future.2025.108052_b4 article-title: Advancing fraud detection systems through online learning – volume: 7 start-page: 3 issue: 7 year: 2015 ident: 10.1016/j.future.2025.108052_b65 article-title: Tiny imagenet visual recognition challenge publication-title: CS 231N – volume: 31 start-page: 2346 issue: 12 year: 2018 ident: 10.1016/j.future.2025.108052_b9 article-title: Learning under concept drift: A review publication-title: IEEE Trans. Knowl. Data Eng. – start-page: 111 year: 2020 ident: 10.1016/j.future.2025.108052_b38 article-title: A tight convergence analysis for stochastic gradient descent with delayed updates – volume: 34 start-page: 20185 year: 2021 ident: 10.1016/j.future.2025.108052_b50 article-title: Asynchronous decentralized online learning publication-title: Adv. Neural Inf. Process. Syst. – volume: 25 year: 2012 ident: 10.1016/j.future.2025.108052_b59 article-title: Large scale distributed deep networks publication-title: Adv. Neural Inf. Process. Syst. – ident: 10.1016/j.future.2025.108052_b19 doi: 10.1109/SBAC-PAD49847.2020.00018 – volume: 30 year: 2017 ident: 10.1016/j.future.2025.108052_b31 article-title: Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent publication-title: Adv. Neural Inf. Process. Syst. – year: 2022 ident: 10.1016/j.future.2025.108052_b62 – volume: 81 start-page: 41611 issue: 29 year: 2022 ident: 10.1016/j.future.2025.108052_b22 article-title: A review on extreme learning machine publication-title: Multimedia Tools Appl. doi: 10.1007/s11042-021-11007-7 – volume: 17 start-page: 688 issue: 4 year: 2006 ident: 10.1016/j.future.2025.108052_b53 article-title: The unequal variance t-test is an underused alternative to student’s t-test and the Mann–Whitney U test publication-title: Behav. Ecol. doi: 10.1093/beheco/ark016 – ident: 10.1016/j.future.2025.108052_b32 – volume: 45 start-page: 5 year: 2001 ident: 10.1016/j.future.2025.108052_b11 article-title: Random forests publication-title: Mach. Learn. doi: 10.1023/A:1010933404324 – volume: 10 start-page: 34 issue: 31s year: 2025 ident: 10.1016/j.future.2025.108052_b57 article-title: A comprehensive framework for residual analysis in regression and machine learning publication-title: J. Inf. Syst. Eng. Manag. – volume: 6 start-page: 1 issue: 1 year: 2019 ident: 10.1016/j.future.2025.108052_b2 article-title: Uncertainty in big data analytics: survey, opportunities, and challenges publication-title: J. Big Data doi: 10.1186/s40537-019-0206-3 – start-page: 443 year: 2007 ident: 10.1016/j.future.2025.108052_b55 article-title: Learning from time-changing data with adaptive windowing – year: 2015 ident: 10.1016/j.future.2025.108052_b56 article-title: Regression basics – ident: 10.1016/j.future.2025.108052_b51 – volume: 34 start-page: 9024 year: 2021 ident: 10.1016/j.future.2025.108052_b37 article-title: Asynchronous stochastic optimization robust to arbitrary delays publication-title: Adv. Neural Inf. Process. Syst. – start-page: 1082 year: 2023 ident: 10.1016/j.future.2025.108052_b42 article-title: Delay-agnostic asynchronous distributed optimization – volume: 35 start-page: 126 issue: 1 year: 2018 ident: 10.1016/j.future.2025.108052_b28 article-title: Model compression and acceleration for deep neural networks: The principles, progress, and challenges publication-title: IEEE Signal Process. Mag. doi: 10.1109/MSP.2017.2765695 – start-page: 135 year: 2018 ident: 10.1016/j.future.2025.108052_b6 article-title: Stock market movement prediction using LDA-online learning model – start-page: 9 year: 2002 ident: 10.1016/j.future.2025.108052_b25 article-title: Efficient backprop – volume: 24 start-page: 1028 issue: 12 year: 2020 ident: 10.1016/j.future.2025.108052_b24 article-title: Embracing change: Continual learning in deep neural networks publication-title: Trends Cogn. Sci. doi: 10.1016/j.tics.2020.09.004 – year: 2014 ident: 10.1016/j.future.2025.108052_b26 – year: 2019 ident: 10.1016/j.future.2025.108052_b39 – start-page: 713 year: 2010 ident: 10.1016/j.future.2025.108052_b10 article-title: Naïve Bayes – volume: 2 start-page: 1 issue: 1 year: 2015 ident: 10.1016/j.future.2025.108052_b12 article-title: Deep learning applications and challenges in big data analytics publication-title: J. Big Data doi: 10.1186/s40537-014-0007-7 – volume: 36 issue: 4 year: 2015 ident: 10.1016/j.future.2025.108052_b18 article-title: Apache Flink: Stream and batch processing in a single engine publication-title: Bull. the IEEE Comput. Soc. Tech. Comm. Data Eng. – start-page: 55 year: 2016 ident: 10.1016/j.future.2025.108052_b54 – volume: 459 start-page: 249 year: 2021 ident: 10.1016/j.future.2025.108052_b1 article-title: Online learning: A comprehensive survey publication-title: Neurocomputing doi: 10.1016/j.neucom.2021.04.112 – year: 2023 ident: 10.1016/j.future.2025.108052_b5 article-title: An online learning approach to dynamic pricing and capacity sizing in service systems publication-title: Oper. Res. doi: 10.1287/opre.2020.612 – volume: 4 start-page: 293 issue: 2 year: 2017 ident: 10.1016/j.future.2025.108052_b44 article-title: Decentralized consensus optimization with asynchrony and delays publication-title: IEEE Trans. Signal Inf. Process. over Netw. – start-page: 314 year: 2022 ident: 10.1016/j.future.2025.108052_b60 article-title: Convergence-time analysis of asynchronous distributed artificial neural networks |
| SSID | ssj0001731 |
| Score | 2.4573104 |
| Snippet | Online Learning (OL) is a sub-field of Machine Learning (ML) which focuses on solving time-sensitive problems through iterative learning from data streams.... |
| SourceID | crossref elsevier |
| SourceType | Index Database Publisher |
| StartPage | 108052 |
| SubjectTerms | Artificial neural networks Asynchronous & decentralized SGD Concept drifts Data streams Online Learning |
| Title | Online Learning from data streams via decentralized and asynchronous SGD |
| URI | https://dx.doi.org/10.1016/j.future.2025.108052 |
| Volume | 175 |
| WOSCitedRecordID | wos001554673400001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVESC databaseName: Elsevier SD Freedom Collection Journals 2021 issn: 0167-739X databaseCode: AIEXJ dateStart: 19950201 customDbUrl: isFulltext: true dateEnd: 99991231 titleUrlDefault: https://www.sciencedirect.com omitProxy: false ssIdentifier: ssj0001731 providerName: Elsevier |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtZ3fS9xAEMeX9uxDX6xtFW1t2QffSkqS3WSzj-KP2lKk4BXuLWRnN3IiObkfov3rnc1s7s6eFBX6EkIge2E-y9zMMvMdxvZM7awGZSLMlm0kdS2iCrIqqlzhRF1rIYn0T3V6WgwG-leY2T5pxwmopilubvTVf0WNzxC2b519Au75ovgA7xE6XhE7Xh8FnsRDO-HUc2og8YWgbV-IF2C4HlZfrAtlmcM_zpJi6-S2AS-V64tiz74dLoetx63yiB-37MKOgTANIkhBzyPz_mgypB6g2XiEe-ryEj9lBouCIC8GYMJxNGkYLB88pPNa5e40bKUjhg4o0fEq0Y7HXXhYGo6y4q3p4ODiK8mnYLKeZm3NI2na_qWDfeaX9itj0BYLqeRLtpaqTBc9trb__WjwY_4HnKgwhjJ8Stcx2Zb1rf7WwxHJUpTR32DrIT3g-4T1LXvhmnfsTTd6gwdP_J6dEGXeUeaeMveUeaDMkTK_R5kjZb5MmSPlTfb7-Kh_cBKFoRgRYLQxjZLcYYZpEpdawODXuNyqGBzEkAtXG5tWqo4LYwoBKpFGispmBtBx1yaLIanFFus1o8ZtMy61zZM8s5g1Cqkd6CqDVLk0BymVSYodFnWGKa9I-6TsigIvSjJk6Q1ZkiF3mOqsV4b4jeKyEoH_880Pz37zI3u92Ju7rDcdz9wn9gqup8PJ-HPYGXcRuG-x |
| linkProvider | Elsevier |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Online+Learning+from+data+streams+via+decentralized+and+asynchronous+SGD&rft.jtitle=Future+generation+computer+systems&rft.au=Tosi%2C+Mauro+Dalle+Lucca&rft.au=Theobald%2C+Martin&rft.date=2026-02-01&rft.pub=Elsevier+B.V&rft.issn=0167-739X&rft.volume=175&rft_id=info:doi/10.1016%2Fj.future.2025.108052&rft.externalDocID=S0167739X25003474 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0167-739X&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0167-739X&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0167-739X&client=summon |