CStream: Parallel Data Stream Compression on Multicore Edge Devices
In the burgeoning realm of Internet of Things (IoT) applications on edge devices, data stream compression has become increasingly pertinent. The integration of added compression overhead and limited hardware resources on these devices calls for a nuanced software-hardware co-design. This paper intro...
Saved in:
| Published in: | IEEE transactions on knowledge and data engineering Vol. 36; no. 11; pp. 5889 - 5904 |
|---|---|
| Main Authors: | , |
| Format: | Journal Article |
| Language: | English |
| Published: |
IEEE
01.11.2024
|
| Subjects: | |
| ISSN: | 1041-4347, 1558-2191 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | In the burgeoning realm of Internet of Things (IoT) applications on edge devices, data stream compression has become increasingly pertinent. The integration of added compression overhead and limited hardware resources on these devices calls for a nuanced software-hardware co-design. This paper introduces CStream , a pioneering framework crafted for parallelizing stream compression on multicore edge devices. CStream grapples with the distinct challenges of delivering a high compression ratio, high throughput, low latency, and low energy consumption. Notably, CStream distinguishes itself by accommodating an array of stream compression algorithms, a variety of hardware architectures and configurations, and an innovative set of parallelization strategies, some of which are proposed herein for the first time. Our evaluation showcases the efficacy of a thoughtful co-design involving a lossy compression algorithm, asymmetric multicore processors, and our novel, hardware-conscious parallelization strategies. This approach achieves a <inline-formula><tex-math notation="LaTeX">2.8 \times</tex-math> <mml:math><mml:mrow><mml:mn>2</mml:mn><mml:mo>.</mml:mo><mml:mn>8</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zeng-ieq1-3386862.gif"/> </inline-formula> compression ratio with only marginal information loss, <inline-formula><tex-math notation="LaTeX">4.3 \times</tex-math> <mml:math><mml:mrow><mml:mn>4</mml:mn><mml:mo>.</mml:mo><mml:mn>3</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zeng-ieq2-3386862.gif"/> </inline-formula> throughput, 65% latency reduction and 89% energy consumption reduction, compared to designs lacking such strategic integration. |
|---|---|
| AbstractList | In the burgeoning realm of Internet of Things (IoT) applications on edge devices, data stream compression has become increasingly pertinent. The integration of added compression overhead and limited hardware resources on these devices calls for a nuanced software-hardware co-design. This paper introduces CStream , a pioneering framework crafted for parallelizing stream compression on multicore edge devices. CStream grapples with the distinct challenges of delivering a high compression ratio, high throughput, low latency, and low energy consumption. Notably, CStream distinguishes itself by accommodating an array of stream compression algorithms, a variety of hardware architectures and configurations, and an innovative set of parallelization strategies, some of which are proposed herein for the first time. Our evaluation showcases the efficacy of a thoughtful co-design involving a lossy compression algorithm, asymmetric multicore processors, and our novel, hardware-conscious parallelization strategies. This approach achieves a <inline-formula><tex-math notation="LaTeX">2.8 \times</tex-math> <mml:math><mml:mrow><mml:mn>2</mml:mn><mml:mo>.</mml:mo><mml:mn>8</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zeng-ieq1-3386862.gif"/> </inline-formula> compression ratio with only marginal information loss, <inline-formula><tex-math notation="LaTeX">4.3 \times</tex-math> <mml:math><mml:mrow><mml:mn>4</mml:mn><mml:mo>.</mml:mo><mml:mn>3</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zeng-ieq2-3386862.gif"/> </inline-formula> throughput, 65% latency reduction and 89% energy consumption reduction, compared to designs lacking such strategic integration. |
| Author | Zeng, Xianzhi Zhang, Shuhao |
| Author_xml | – sequence: 1 givenname: Xianzhi orcidid: 0009-0002-4613-9297 surname: Zeng fullname: Zeng, Xianzhi email: xianzhi_xianzhi@mymail.sutd.edu.sg organization: Singapore University of Technology and Design, Singapore – sequence: 2 givenname: Shuhao orcidid: 0000-0002-9927-6925 surname: Zhang fullname: Zhang, Shuhao email: shuhao.zhang@ntu.edu.sg organization: Nanyang Technological University, Singapore |
| BookMark | eNp9kM9Kw0AQhxepYFt9AMHDvkDqzv7LrjdJWxUrCtZzmKYTWUmbshsF396U9iAehIEZfvDNMN-IDbbtlhi7BDEBEP56-TidTaSQeqKUs87KEzYEY1wmwcOgn4WGTCudn7FRSh9CCJc7GLKieO0i4eaGv2DEpqGGT7FDfkh50W52kVIK7Zb39fTZdKFqI_HZ-p34lL5CRemcndbYJLo49jF7m8-WxX22eL57KG4XWSWt7TK3EliDX4FZe5nLtc99LYz1oNGKSpGXUtceJXlbgbQGNeUa0a-MdFpBrsYMDnur2KYUqS53MWwwfpcgyr2Fcm-h3FsojxZ6Jv_DVKHDrv-nixiaf8mrAxmI6NclI6ywTv0AkAFq2g |
| CODEN | ITKEEH |
| CitedBy_id | crossref_primary_10_1109_ACCESS_2025_3586539 |
| Cites_doi | 10.1109/DCC.2015.66 10.14778/2021017.2021021 10.1109/DCC.2019.00123 10.1145/3448016.3452793 10.1145/3387902.3394037 10.1145/3514221.3517836 10.1109/TWC.2021.3088910 10.1145/3583678.3596885 10.1109/HPCA.2013.6522303 10.1145/3299869.3300067 10.1109/JRPROC.1952.273898 10.1016/j.apr.2020.11.010 10.1145/3035918.3064007 10.1145/1620585.1620588 10.1109/ISCA.2005.51 10.1109/TPDS.2020.3045279 10.1145/2856125 10.1145/1498765.1498785 10.1007/s00778-014-0368-8 10.1109/TPDS.2021.3119402 10.1145/3372799.3394370 10.1145/2508148.2485927 10.1145/3342555 10.1145/2654822.2541971 10.1145/3447993.3448625 10.1109/ICECCT.2017.8117850 10.1145/1142473.1142548 10.1109/TC.2020.2984607 10.1145/3419634 10.1145/3323991 10.1109/ICASSP40776.2020.9053178 10.1145/1290672.1290686 10.1109/PATMOS.2015.7347594 10.1016/j.jbiomech.2018.08.015 10.1109/TIT.1978.1055934 10.1109/ISCA.2012.6237019 10.1109/CASES.2013.6662519 10.5555/1863103.1863113 10.1609/aaai.v33i01.33018893 10.1145/3297280.3297552 10.1145/362084.362137 10.1145/163090.163096 10.1145/2654822.2541974 10.1109/ICFPT52863.2021.9609952 10.1109/ICDE48307.2020.00136 10.1109/ICDE55515.2023.00078 10.1109/MS.2009.183 10.1145/3444943 10.1007/978-3-319-17248-4_7 10.14778/3402707.3402725 10.1109/TIT.1975.1055349 10.1016/j.jpdc.2020.12.004 10.1109/TVT.2019.2912942 10.1186/s12984-015-0081-x 10.1145/3264903 10.1145/3447993.3448628 10.1145/3296957.3173184 |
| ContentType | Journal Article |
| DBID | 97E RIA RIE AAYXX CITATION |
| DOI | 10.1109/TKDE.2024.3386862 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE/IET Electronic Library CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE/IET Electronic Library url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering Computer Science |
| EISSN | 1558-2191 |
| EndPage | 5904 |
| ExternalDocumentID | 10_1109_TKDE_2024_3386862 10506068 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: Future Communications Research and Development Programme grantid: FCP-SUTD-RG-2022-005 – fundername: MoE AcRF Tier 2 grantid: MOE-T2EP20122-0010 – fundername: Info-communications Media Development Authority; Infocomm Media Development Authority funderid: 10.13039/501100008629 – fundername: National Research Foundation Singapore; National Research Foundation, Singapore funderid: 10.13039/501100001381 – fundername: NTU grantid: 023452-00001 |
| GroupedDBID | -~X .DC 0R~ 1OL 29I 4.4 5GY 5VS 6IK 97E 9M8 AAJGR AARMG AASAJ AAWTH ABAZT ABFSI ABQJQ ABVLG ACGFO ACIWK AENEX AETIX AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 E.L EBS EJD F5P HZ~ H~9 ICLAB IEDLZ IFIPE IFJZH IPLJI JAVBF LAI M43 MS~ O9- OCL P2P PQQKQ RIA RIE RNI RNS RXW RZB TAE TAF TN5 UHB VH1 AAYXX CITATION |
| ID | FETCH-LOGICAL-c266t-8b0af19b15d9272d979f056914a60c3e9224f9a2e96c1265a4e74aa9b52843173 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 3 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001336378400071&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1041-4347 |
| IngestDate | Sat Nov 29 02:36:09 EST 2025 Tue Nov 18 21:25:32 EST 2025 Wed Aug 27 02:20:02 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 11 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c266t-8b0af19b15d9272d979f056914a60c3e9224f9a2e96c1265a4e74aa9b52843173 |
| ORCID | 0009-0002-4613-9297 0000-0002-9927-6925 |
| PageCount | 16 |
| ParticipantIDs | ieee_primary_10506068 crossref_primary_10_1109_TKDE_2024_3386862 crossref_citationtrail_10_1109_TKDE_2024_3386862 |
| PublicationCentury | 2000 |
| PublicationDate | 2024-11-01 |
| PublicationDateYYYYMMDD | 2024-11-01 |
| PublicationDate_xml | – month: 11 year: 2024 text: 2024-11-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationTitle | IEEE transactions on knowledge and data engineering |
| PublicationTitleAbbrev | TKDE |
| PublicationYear | 2024 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| References | Zeuch (ref6) ref13 ref59 ref14 ref53 ref52 ref54 ref17 ref16 ref19 Lu (ref20) ref51 ref50 ref46 ref45 ref48 ref47 ref42 ref86 ref41 ref85 ref43 ref49 ref8 ref7 (ref22) 2019 ref4 ref3 Pekhimenko (ref1) ref5 ref82 ref81 Haque (ref83) ref40 ref84 ref80 ref35 ref34 ref78 ref36 ref31 ref75 ref74 ref77 ref32 ref76 ref2 ref39 Reinsel (ref26) 2018; 16 ref71 ref70 ref73 ref72 ref68 ref69 ref64 ref63 Pekhimenko (ref37) ref28 ref27 ref29 Wei (ref55) Bharde (ref79) 2018 ref60 ref62 ref61 |
| References_xml | – volume-title: Proc. 10th Conf. Innov. Data Syst. Res. ident: ref6 article-title: The nebulastream platform for data and application management in the internet of things – ident: ref29 doi: 10.1109/DCC.2015.66 – ident: ref86 doi: 10.14778/2021017.2021021 – ident: ref50 doi: 10.1109/DCC.2019.00123 – ident: ref64 doi: 10.1145/3448016.3452793 – ident: ref49 doi: 10.1145/3387902.3394037 – ident: ref5 doi: 10.1145/3514221.3517836 – ident: ref84 doi: 10.1109/TWC.2021.3088910 – ident: ref3 doi: 10.1145/3583678.3596885 – ident: ref85 doi: 10.1109/HPCA.2013.6522303 – ident: ref27 doi: 10.1145/3299869.3300067 – ident: ref34 doi: 10.1109/JRPROC.1952.273898 – start-page: 625 volume-title: Proc. 50th Annu. IEEE/ACM Int. Symp. Microarchitecture ident: ref83 article-title: Exploiting heterogeneity for tail latency and energy efficiency – ident: ref61 doi: 10.1016/j.apr.2020.11.010 – ident: ref17 doi: 10.1145/3035918.3064007 – ident: ref28 doi: 10.1145/1620585.1620588 – ident: ref70 doi: 10.1109/ISCA.2005.51 – ident: ref72 doi: 10.1109/TPDS.2020.3045279 – ident: ref41 doi: 10.1145/2856125 – ident: ref68 doi: 10.1145/1498765.1498785 – ident: ref80 doi: 10.1007/s00778-014-0368-8 – ident: ref43 doi: 10.1109/TPDS.2021.3119402 – ident: ref46 doi: 10.1145/3372799.3394370 – ident: ref74 doi: 10.1145/2508148.2485927 – ident: ref36 doi: 10.1145/3342555 – start-page: 377 volume-title: Proc. 21st Int. Conf. Parallel Architectures Compilation Techn. ident: ref37 article-title: Base-delta-immediate compression: Practical data compression for on-chip caches – ident: ref47 doi: 10.1145/2654822.2541971 – ident: ref53 doi: 10.1145/3447993.3448625 – start-page: 307 volume-title: Proc. USENIX Annu. Tech. Conf. ident: ref1 article-title: Tersecades: Efficient data compression in stream processing – ident: ref35 doi: 10.1109/ICECCT.2017.8117850 – ident: ref13 doi: 10.1145/1142473.1142548 – ident: ref82 doi: 10.1109/TC.2020.2984607 – ident: ref8 doi: 10.1145/3419634 – ident: ref77 doi: 10.1145/3323991 – ident: ref78 doi: 10.1109/ICASSP40776.2020.9053178 – ident: ref52 doi: 10.1145/1290672.1290686 – ident: ref45 doi: 10.1109/PATMOS.2015.7347594 – ident: ref63 doi: 10.1016/j.jbiomech.2018.08.015 – ident: ref16 doi: 10.1109/TIT.1978.1055934 – ident: ref73 doi: 10.1109/ISCA.2012.6237019 – ident: ref71 doi: 10.1109/CASES.2013.6662519 – ident: ref51 doi: 10.5555/1863103.1863113 – volume-title: Proc. Adv. Neural Inf. Process. Syst. ident: ref55 article-title: Online reinforcement learning in stochastic games – ident: ref60 doi: 10.1609/aaai.v33i01.33018893 – ident: ref7 doi: 10.1145/3297280.3297552 – volume: 16 start-page: 1 year: 2018 ident: ref26 article-title: The digitization of the world from edge to core publication-title: Framingham: Int. Data Corporation – ident: ref76 doi: 10.1145/362084.362137 – ident: ref75 doi: 10.1145/163090.163096 – ident: ref48 doi: 10.1145/2654822.2541974 – ident: ref2 doi: 10.1109/ICFPT52863.2021.9609952 – ident: ref54 doi: 10.1109/ICDE48307.2020.00136 – ident: ref4 doi: 10.1109/ICDE55515.2023.00078 – ident: ref40 doi: 10.1109/MS.2009.183 – year: 2019 ident: ref22 article-title: Creator of the angry birds game – ident: ref31 doi: 10.1145/3444943 – ident: ref69 doi: 10.1007/978-3-319-17248-4_7 – ident: ref19 doi: 10.14778/3402707.3402725 – ident: ref32 doi: 10.1109/TIT.1975.1055349 – volume-title: Proc. 3rd USENIX Workshop Hot Top. Edge Comput. ident: ref20 article-title: Adaptively compressing {IoT} data on the resource-constrained edge – ident: ref39 doi: 10.1016/j.jpdc.2020.12.004 – volume-title: Proc. USENIX Workshop Hot Top. Edge Comput. year: 2018 ident: ref79 article-title: $\lbrace${Store-Edge $\rbrace$}$\lbrace${ RippleStream$\rbrace$}: Versatile infrastructure for $\lbrace${IoT$\rbrace$} data transfer, – ident: ref59 doi: 10.1109/TVT.2019.2912942 – ident: ref62 doi: 10.1186/s12984-015-0081-x – ident: ref14 doi: 10.1145/3264903 – ident: ref42 doi: 10.1145/3447993.3448628 – ident: ref81 doi: 10.1145/3296957.3173184 |
| SSID | ssj0008781 |
| Score | 2.4636786 |
| Snippet | In the burgeoning realm of Internet of Things (IoT) applications on edge devices, data stream compression has become increasingly pertinent. The integration of... |
| SourceID | crossref ieee |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 5889 |
| SubjectTerms | Asymmetric hardware Compression algorithms edge computing and IoT Energy consumption Hardware Internet of Things Multicore processing Program processors stream compression Throughput |
| Title | CStream: Parallel Data Stream Compression on Multicore Edge Devices |
| URI | https://ieeexplore.ieee.org/document/10506068 |
| Volume | 36 |
| WOSCitedRecordID | wos001336378400071&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE/IET Electronic Library customDbUrl: eissn: 1558-2191 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0008781 issn: 1041-4347 databaseCode: RIE dateStart: 19890101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEB5s8aAHq7VifZGDJ2HrPtLNxpv0gSCUHir0tiSbRIS6lbr19ztJtqUeFIQ9LCHDLjM7r53MfAC3FK1iGEkdZIqygJpIBTLhAjWei9ioQlAqHNgEm0yy-ZxP62Z11wujtXaHz3TP3rpavloWa_urDDXcjsNLswY0GGO-WWtrdjPmEEkxvcCkKKGsLmHiQ-9nz8MRpoIx7WFCZlsifjihHVQV51TGrX--zjEc1dEjefTiPoE9XbahtUFmILWituFwZ8zgKQwGtvYs3h_IVKwsdsqCDEUliF8lltyfhi0JXq4l1w63JCP1qslQO1vSgZfxaDZ4CmrwhKBAn1sFmQyFibiM-orHLFaccYPBDo-oSMMi0Rx9t0F5aJ4WUZz2BdWMCsFlHx0WBhXJGTTLZanPgYTScKNEYoNJ1PdIJlSZmDOJwZ2Wqe5CuOFmXtSTxS3AxSJ3GUbIcyuA3AogrwXQhbstyYcfq_HX5o5l_s5Gz_eLX9Yv4cCS-47BK2hWq7W-hv3iq3r7XN24r-YbFlm81Q |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEB60CurBaq1Ynzl4ErbuI_uIN-mDSmvpoUJvS7JJRKit1K2_30l2W-pBQdjDEpLdZWbnlcnMB3BLUSu6nlBOImnsUO1JRwSMo8Qz7muZcUq5BZuIh8NkMmGjsljd1sIopezhM9U0tzaXL-fZ0myVoYSbdnhRsg07IaW-V5RrrRVvEltMUgwwMCwKaFwmMfG19-N-u4PBoE-bGJKZoogfZmgDV8WalW71nx90BIel_0geC4Yfw5aa1aC6wmYgpajW4GCj0eAJtFom-8zfH8iILwx6ypS0ec5JMUrM8uI87IzgZYtyTXtL0pGvirSV1SZ1eOl2xq2eU8InOBla3dxJhMu1x4QXSubHvmQx0-juMI_yyM0CxdB6a-SIYlHm-VHIqYop50yEaLLQrQhOoTKbz9QZEFdopiUPjDuJEu-JgErts1ige6dEpBrgrqiZZmVvcQNxMU1tjOGy1DAgNQxISwY04G695KNorPHX5Loh_sbEgu7nv4zfwF5v_DxIB0_D_gXsm0cV9YOXUMkXS3UFu9lX_va5uLZ_0DcZccAc |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=CStream%3A+Parallel+Data+Stream+Compression+on+Multicore+Edge+Devices&rft.jtitle=IEEE+transactions+on+knowledge+and+data+engineering&rft.au=Zeng%2C+Xianzhi&rft.au=Zhang%2C+Shuhao&rft.date=2024-11-01&rft.pub=IEEE&rft.issn=1041-4347&rft.volume=36&rft.issue=11&rft.spage=5889&rft.epage=5904&rft_id=info:doi/10.1109%2FTKDE.2024.3386862&rft.externalDocID=10506068 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1041-4347&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1041-4347&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1041-4347&client=summon |