Self-Adaptive Micro-Batching for Low-Latency GPU-Accelerated Stream Processing

Stream processing is a computing paradigm enabling the continuous processing of unbounded data streams. Some classes of stream processing applications can greatly benefit from the parallel processing power and affordability offered by GPUs. However, efficient GPU utilization with stream processing a...

Full description

Saved in:

Bibliographic Details
Published in:	International journal of parallel programming Vol. 53; no. 2; p. 14
Main Authors:	Leonarczyk, Ricardo, Mencagli, Gabriele, Griebler, Dalvan
Format:	Journal Article
Language:	English
Published:	New York Springer US 01.04.2025 Springer Nature B.V
Subjects:	Adaptive algorithms Algorithms Computer Science Data transfer (computers) Data transmission Effectiveness Graphics processing units Parallel processing Processor Architectures Segments Self adaptive control systems Software Engineering/Programming and Operating Systems Theory of Computation Workload Workloads Parallel programming GPU programming Self-adaptive algorithms Heterogeneous architectures
ISSN:	0885-7458, 1573-7640
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Abstract	Stream processing is a computing paradigm enabling the continuous processing of unbounded data streams. Some classes of stream processing applications can greatly benefit from the parallel processing power and affordability offered by GPUs. However, efficient GPU utilization with stream processing applications often requires micro-batching techniques, i.e., the continuous processing of data batches to expose data parallelism opportunities and amortize host-device data transfer overheads. Micro-batching further introduces the challenge of finding suitable micro-batch sizes to maintain low-latency processing under highly dynamic workloads. The research field of self-adaptive software provides different techniques to address such a challenge. Our goal is to assess the performance of six self-adaptive algorithms in meeting latency requirements through micro-batch size adaptation. The algorithms are applied to a GPU-accelerated stream processing benchmark with a highly dynamic workload. Four of the six algorithms have already been evaluated using a smaller workload with the same application. We propose two new algorithms to address the shortcomings detected in the former four. The results demonstrate that a highly dynamic workload is challenging for the evaluated algorithms, as they could not meet the most strict latency requirements for more than 38.5% of the stream data items. Overall, all algorithms performed similarly in meeting the latency requirements. However, one of our proposed algorithms met the requirements for 4% more data items than the best of the previously studied algorithms, demonstrating more effectiveness in highly variable workloads. This effectiveness is particularly evident in segments of the workload with abrupt transitions between low- and high-latency regions, where our proposed algorithms met the requirements for 79% of the data items in those segments, compared to 33% for the best of the earlier algorithms.
AbstractList	Stream processing is a computing paradigm enabling the continuous processing of unbounded data streams. Some classes of stream processing applications can greatly benefit from the parallel processing power and affordability offered by GPUs. However, efficient GPU utilization with stream processing applications often requires micro-batching techniques, i.e., the continuous processing of data batches to expose data parallelism opportunities and amortize host-device data transfer overheads. Micro-batching further introduces the challenge of finding suitable micro-batch sizes to maintain low-latency processing under highly dynamic workloads. The research field of self-adaptive software provides different techniques to address such a challenge. Our goal is to assess the performance of six self-adaptive algorithms in meeting latency requirements through micro-batch size adaptation. The algorithms are applied to a GPU-accelerated stream processing benchmark with a highly dynamic workload. Four of the six algorithms have already been evaluated using a smaller workload with the same application. We propose two new algorithms to address the shortcomings detected in the former four. The results demonstrate that a highly dynamic workload is challenging for the evaluated algorithms, as they could not meet the most strict latency requirements for more than 38.5% of the stream data items. Overall, all algorithms performed similarly in meeting the latency requirements. However, one of our proposed algorithms met the requirements for 4% more data items than the best of the previously studied algorithms, demonstrating more effectiveness in highly variable workloads. This effectiveness is particularly evident in segments of the workload with abrupt transitions between low- and high-latency regions, where our proposed algorithms met the requirements for 79% of the data items in those segments, compared to 33% for the best of the earlier algorithms.
ArticleNumber	14
Author	Leonarczyk, Ricardo Griebler, Dalvan Mencagli, Gabriele
Author_xml	– sequence: 1 givenname: Ricardo surname: Leonarczyk fullname: Leonarczyk, Ricardo email: ricardo.leonarczyk@edu.pucrs.br organization: School of Technology, Pontifical Catholic University of Rio Grande do Sul (PUCRS), Computer Science Department, University of Pisa – sequence: 2 givenname: Gabriele surname: Mencagli fullname: Mencagli, Gabriele organization: Computer Science Department, University of Pisa – sequence: 3 givenname: Dalvan surname: Griebler fullname: Griebler, Dalvan organization: School of Technology, Pontifical Catholic University of Rio Grande do Sul (PUCRS)
BookMark	eNp9kMFOwzAMhiM0JLbBC3CqxDmQNGkTH8cEA6nApLFzlLbO2LS1I-lAe3sCReLGybL1f7b8jcigaRsk5JKza86YugmcqTynLM1obEFQeUKGPFOCqlyyARkyrTOqZKbPyCiEDWMMlNZD8rzAraOT2u679QcmT-vKt_TWdtXbulklrvVJ0X7SwnbYVMdkNl_SSVXhFn2c1Mmi82h3ydy3FYYQiXNy6uw24MVvHZPl_d3r9IEWL7PH6aSgVcpYRx1wqxhkKZa8znIUGS8RHJROQu6Ug1qA1oAgFE9r6UqQrJK5Bc5QSlWKMbnq9-59-37A0JlNe_BNPGkEV6lQoAFiKu1T8akQPDqz9-ud9UfDmfn2ZnpvJnozP96MjJDooRDDzQr93-p_qC-sr3D5
Cites_doi	10.1002/cpe.6759 10.1145/3132747.3132750 10.1017/CBO9781139058940 10.1109/PDP55904.2022.00011 10.1002/047166880X 10.1016/j.jpdc.2023.104782 10.1109/ACCESS.2019.2910312 10.1002/9781119332015.ch13 10.1016/j.csi.2024.103922 10.1007/978-0-387-71003-7_9 10.1145/3269961.3282845 10.1109/TPDS.2018.2846234 10.1002/cpe.5786 10.1145/2670979.2670995 10.1007/978-3-031-50684-0_7 10.1017/9781009089517 10.1109/ICAC.2016.27
ContentType	Journal Article
Copyright	The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2025 Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law. Copyright Springer Nature B.V. Apr 2025
Copyright_xml	– notice: The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2025 Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law. – notice: Copyright Springer Nature B.V. Apr 2025
DBID	AAYXX CITATION 7SC 8FD JQ2 L7M L~C L~D
DOI	10.1007/s10766-025-00793-4
DatabaseName	CrossRef Computer and Information Systems Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional
DatabaseTitle	CrossRef Computer and Information Systems Abstracts Technology Research Database Computer and Information Systems Abstracts – Academic Advanced Technologies Database with Aerospace ProQuest Computer Science Collection Computer and Information Systems Abstracts Professional
DatabaseTitleList	Computer and Information Systems Abstracts
DeliveryMethod	fulltext_linktorsrc
Discipline	Computer Science
EISSN	1573-7640
ExternalDocumentID	10_1007_s10766_025_00793_4
GrantInformation_xml	– fundername: European Commission grantid: 2022BAL2F3 funderid: http://dx.doi.org/10.13039/501100000780 – fundername: Conselho Nacional das Fundações Estaduais de Amparo à Pesquisa grantid: 306511/2021-5 funderid: http://dx.doi.org/10.13039/501100019831 – fundername: Fundação de Amparo à Pesquisa do Estado do Rio Grande do Sul grantid: 24/2551-0001400-4 funderid: http://dx.doi.org/10.13039/501100004263
GroupedDBID	-Y2 -~X .4S .86 .DC .VR 06D 0R~ 0VY 199 1N0 2.D 203 28- 29J 2J2 2JN 2JY 2KG 2LR 2P1 2VQ 2~H 30V 4.4 406 408 409 40D 40E 5GY 5QI 5VS 67Z 6NX 78A 7WY 8FE 8FG 8FL 8G5 8TC 8UJ 95- 95. 95~ 96X AABHQ AACDK AAHNG AAIAL AAJBT AAJKR AANZL AAOBN AAPKM AARHV AARTL AASML AATNV AATVU AAUYE AAWCG AAYIU AAYJJ AAYQN AAYTO AAYZH ABAKF ABBBX ABBXA ABDBE ABDBF ABDPE ABDZT ABECU ABFSI ABFTD ABFTV ABHLI ABHQN ABJNI ABJOX ABKCH ABKTR ABMNI ABMQK ABNWP ABQBU ABQSL ABSXP ABTAH ABTEG ABTHY ABTKH ABTMW ABULA ABUWG ABWNU ABXPI ACAOD ACBXY ACDTI ACGFO ACGFS ACHSB ACHXU ACIHN ACKNC ACMDZ ACMLO ACNCT ACOKC ACOMO ACPIV ACREN ACUHS ACZOJ ADHIR ADHKG ADKNI ADKPE ADMLS ADRFC ADTPH ADURQ ADYFF ADYOE ADZKW AEAQA AEBTG AEFIE AEFQL AEGAL AEGNC AEJHL AEJRE AEKMD AEMSY AENEX AEOHA AEPYU AESKC AETLH AEVLU AEXYK AFBBN AFDZB AFEXP AFGCZ AFKRA AFLOW AFQWF AFWTZ AFYQB AFZKB AGAYW AGDGC AGGDS AGJBK AGMZJ AGQEE AGQMX AGRTI AGWIL AGWZB AGYKE AHAVH AHBYD AHKAY AHPBZ AHSBF AHYZX AIAKS AIGIU AIIXL AILAN AITGF AJBLW AJRNO AJZVZ ALMA_UNASSIGNED_HOLDINGS ALWAN AMKLP AMTXH AMXSW AMYLF AOCGG ARAPS ARCSS ARMRJ AXYYD AYFIA AYJHY AZFZN AZQEC B-. B0M BA0 BBWZM BDATZ BENPR BEZIV BGLVJ BGNMA BKOMP BPHCQ BSONS CAG CCPQU COF CS3 CSCUP DDRTE DL5 DNIVK DPUIP DU5 DWQXO E.L EAD EAP EAS EBLON EBS EDO EIOEI EJD EMK EPL ESBYG ESX FEDTE FERAY FFXSO FIGPU FINBP FNLPD FRNLG FRRFC FSGXE FWDCC GGCAI GGRSB GJIRD GNUQQ GNWQR GQ7 GQ8 GROUPED_ABI_INFORM_RESEARCH GUQSH GXS H13 HCIFZ HF~ HG5 HG6 HMJXF HQYDN HRMNR HVGLF HZ~ H~9 I-F I09 IHE IJ- IKXTQ ITM IWAJR IXC IZIGR IZQ I~X I~Z J-C J0Z JBSCW JCJTX JZLTJ K60 K6V K6~ K7- KDC KOV KOW LAK LLZTM M0C M2O M4Y MA- MS~ N2Q NB0 NDZJH NPVJJ NQJWS NU0 O9- O93 O9G O9I O9J OAM OVD P19 P62 P9O PF0 PHGZT PQBIZ PQBZA PQQKQ PROAC PT4 PT5 Q2X QOK QOS R89 R9I RHV RNI RNS ROL RPX RSV RZC RZE RZK S16 S1Z S26 S27 S28 S3B SAP SCJ SCLPG SCO SDH SDM SHX SISQX SJYHP SNE SNPRN SNX SOHCF SOJ SPISZ SRMVM SSLCW STPWE SZN T13 T16 TAE TEORI TN5 TSG TSK TSV TUC TUS U2A U5U UG4 UOJIU UTJUX UZXMN VC2 VFIZW W23 W48 WH7 WK8 YLTOR Z45 ZMTXR ZY4 ~8M ~EX AAYXX ABBRH ABFSG ABRTQ ACSTC AEZWR AFFHD AFHIU AFOHR AGQPQ AHWEU AIXLP ATHPR CITATION PHGZM PQGLB 7SC 8FD JQ2 L7M L~C L~D
ID	FETCH-LOGICAL-c200t-f91a70952eb1d56e351be9f9bf496f7f9d39889e93712d4fb940c46a910e447b3
IEDL.DBID	RSV
ISICitedReferencesCount	0
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001434447100002&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0885-7458
IngestDate	Wed Nov 05 08:39:57 EST 2025 Sat Nov 29 08:06:09 EST 2025 Sat Apr 05 01:12:49 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Issue	2
Keywords	Parallel programming GPU programming Self-adaptive algorithms Heterogeneous architectures
Language	English
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c200t-f91a70952eb1d56e351be9f9bf496f7f9d39889e93712d4fb940c46a910e447b3
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
PQID	3172379899
PQPubID	48389
ParticipantIDs	proquest_journals_3172379899 crossref_primary_10_1007_s10766_025_00793_4 springer_journals_10_1007_s10766_025_00793_4
PublicationCentury	2000
PublicationDate	20250400 2025-04-00 20250401
PublicationDateYYYYMMDD	2025-04-01
PublicationDate_xml	– month: 4 year: 2025 text: 20250400
PublicationDecade	2020
PublicationPlace	New York
PublicationPlace_xml	– name: New York
PublicationTitle	International journal of parallel programming
PublicationTitleAbbrev	Int J Parallel Prog
PublicationYear	2025
Publisher	Springer US Springer Nature B.V
Publisher_xml	– name: Springer US – name: Springer Nature B.V
References	793_CR19 793_CR1 HCM Andrade (793_CR2) 2014 793_CR18 793_CR4 793_CR3 A Vogel (793_CR6) 2022; 34 793_CR15 793_CR17 793_CR16 793_CR9 793_CR5 793_CR8 793_CR7 JL Hellerstein (793_CR13) 2004 D Cheng (793_CR12) 2018; 29 793_CR11 G Mencagli (793_CR14) 2024; 184 T De Matteis (793_CR10) 2019; 7
References_xml	– volume: 34 start-page: 6759 issue: 6 year: 2022 ident: 793_CR6 publication-title: Concurrency Comput. Practice Exp. doi: 10.1002/cpe.6759 – ident: 793_CR11 doi: 10.1145/3132747.3132750 – volume-title: Fundamentals of stream processing: application design, systems, and analytics year: 2014 ident: 793_CR2 doi: 10.1017/CBO9781139058940 – ident: 793_CR9 doi: 10.1109/PDP55904.2022.00011 – volume-title: Feedback control of computing systems year: 2004 ident: 793_CR13 doi: 10.1002/047166880X – volume: 184 start-page: 104782 year: 2024 ident: 793_CR14 publication-title: Journal of parallel and distributed computing doi: 10.1016/j.jpdc.2023.104782 – volume: 7 start-page: 48753 year: 2019 ident: 793_CR10 publication-title: IEEE Access doi: 10.1109/ACCESS.2019.2910312 – ident: 793_CR17 doi: 10.1002/9781119332015.ch13 – ident: 793_CR18 doi: 10.1016/j.csi.2024.103922 – ident: 793_CR1 doi: 10.1007/978-0-387-71003-7_9 – ident: 793_CR3 doi: 10.1145/3269961.3282845 – volume: 29 start-page: 2672 issue: 12 year: 2018 ident: 793_CR12 publication-title: IEEE Trans. Parallel Distrib. Syst. doi: 10.1109/TPDS.2018.2846234 – ident: 793_CR4 doi: 10.1002/cpe.5786 – ident: 793_CR7 doi: 10.1145/2670979.2670995 – ident: 793_CR5 doi: 10.1007/978-3-031-50684-0_7 – ident: 793_CR16 – ident: 793_CR15 doi: 10.1017/9781009089517 – ident: 793_CR19 – ident: 793_CR8 doi: 10.1109/ICAC.2016.27
SSID	ssj0009788
Score	2.3464706
Snippet	Stream processing is a computing paradigm enabling the continuous processing of unbounded data streams. Some classes of stream processing applications can...
SourceID	proquest crossref springer
SourceType	Aggregation Database Index Database Publisher
StartPage	14
SubjectTerms	Adaptive algorithms Algorithms Computer Science Data transfer (computers) Data transmission Effectiveness Graphics processing units Parallel processing Processor Architectures Segments Self adaptive control systems Software Engineering/Programming and Operating Systems Theory of Computation Workload Workloads
Title	Self-Adaptive Micro-Batching for Low-Latency GPU-Accelerated Stream Processing
URI	https://link.springer.com/article/10.1007/s10766-025-00793-4 https://www.proquest.com/docview/3172379899
Volume	53
WOSCitedRecordID	wos001434447100002&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVAVX databaseName: Springer Nature - Connect here FIRST to enable access customDbUrl: eissn: 1573-7640 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0009788 issn: 0885-7458 databaseCode: RSV dateStart: 19970101 isFulltext: true titleUrlDefault: https://link.springer.com/search?facet-content-type=%22Journal%22 providerName: Springer Nature
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1NT8MwDLVgcODC-BSDgXLgBpHWJm2S40AMDmOaGEO7VU2aSEiwTdsA8e9xulYFBAc4p40qJ7afa_sZ4JQF2sXSahrK2FIuMtQ5FxgqdYrOEqNmYUw-bEL0enI0Uv2iKWxeVruXKcncUn9qdhOxL5iNaMuzulG-Cmvo7qQf2HA3eKiodkU-bRLVJ6KCR7Jolfl5j6_uqMKY39Kiubfp1P_3nVuwWaBL0l5eh21YseMdqJeTG0ihyLvQG9gnR9tZOvXWjtz6qjx6gVbZ_48iiGNJd_JGu6kH1O_kuj-kbWPQQXleiYz4THb6TIoeA3xjD4adq_vLG1pMVqAGtWJBnQpSgeAqREudRbFlUaCtcko7rmInnMqYklJZT5YXZtxpxVuGxyliC8u50GwfauPJ2B4A8avcxMKxCKPsVKrQOc2YkTKOnA5cA85KASfTJYFGUlEle1ElKKokF1XCG9AszyAplGmeIMQJmVAYGTbgvJR5tfz7bod_e_wINsL82HxdThNqi9mLPYZ187p4nM9O8kv2AYFTynw
linkProvider	Springer Nature
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV3fT8IwEL4omuiL-DOiqH3wTZu4rVvbRzQiRiBEwPC2rF2bmCgQQI3_vdexBTX6oM_dmuXau_tud_cdwGngKRsJo6gvIkMZT1HnrKepUAk6S4yaudbZsAnebovBQHbyprBpUe1epCQzS_2p2Y1HrmA2pBeO1Y2yZVhh6LEcY_5992FBtcuzaZOoPiHlLBR5q8zPe3x1RwuM-S0tmnmbevl_37kJGzm6JLX5ddiCJTPchnIxuYHkirwD7a55srSWJmNn7UjLVeXRS7TK7n8UQRxLmqM32kwcoH4nN50-rWmNDsrxSqTEZbKTZ5L3GOAbu9CvX_euGjSfrEA1asWMWuklHMGVj5Y6DSMThJ4y0kplmYwstzINpBDSOLI8P2VWSXahWZQgtjCMcRXsQWk4Gpp9IG6V6YjbIMQoOxHSt1YFgRYiCq3ybAXOCgHH4zmBRrygSnaiilFUcSaqmFWgWpxBnCvTNEaI4wdcYmRYgfNC5ovl33c7-NvjJ7DW6LWacfO2fXcI6352hK5Gpwql2eTFHMGqfp09TifH2YX7AAkozWA
linkToPdf	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LT8MwDLZgIMSF8RSDATlwg2i0TZvkOB4DxKgmjaHdqiZNJCTopq2A-PckXasOBAfEOW1UObH9ubY_Axx7jtABUwK7LFCY0MTonHYkZiI2ztJEzVTKfNgEDUM2HPLeXBd_Xu1epiRnPQ2WpSnNWuNEt-Ya32hgi2d9fGYZ3jBZhCViC-ltvN5_rGh3aT550qiSjynxWdE28_MeX11ThTe_pUhzz9Op__-b12GtQJ2oPbsmG7Cg0k2olxMdUKHgWxD21bPG7SQeWyuI7m21Hj431tr-p0IG36Lu6B13Ywu0P9B1b4DbUhrHZfkmEmQz3PELKnoPzBvbMOhcPVzc4GLiApZGWzKsuRNTA7pcY8ETP1Ce7wjFNRea8EBTzROPM8aVJdFzE6IFJ2eSBLHBHIoQKrwdqKWjVO0CsqtEBlR7vom-Y8ZdrYXnScYCXwtHN-CkFHY0nhFrRBWFshVVZEQV5aKKSAOa5XlEhZJNIwN9XI9yEzE24LSUf7X8-257f3v8CFZ6l52oexve7cOqm5-gLd1pQi2bvKoDWJZv2dN0cpjfvU-VD9ZE
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Self-Adaptive+Micro-Batching+for+Low-Latency+GPU-Accelerated+Stream+Processing&rft.jtitle=International+journal+of+parallel+programming&rft.au=Leonarczyk%2C+Ricardo&rft.au=Mencagli%2C+Gabriele&rft.au=Griebler%2C+Dalvan&rft.date=2025-04-01&rft.issn=0885-7458&rft.eissn=1573-7640&rft.volume=53&rft.issue=2&rft_id=info:doi/10.1007%2Fs10766-025-00793-4&rft.externalDBID=n%2Fa&rft.externalDocID=10_1007_s10766_025_00793_4
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0885-7458&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0885-7458&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0885-7458&client=summon