Faster and Stronger Lossless Compression with Optimized Autoregressive Framework
Neural AutoRegressive (AR) framework has been applied in general-purpose lossless compression recently to improve compression performance. However, this paper found that directly applying the original AR framework causes the duplicated processing problem and the in-batch distribution variation probl...
Uložené v:
| Vydané v: | 2023 60th ACM/IEEE Design Automation Conference (DAC) s. 1 - 6 |
|---|---|
| Hlavní autori: | , , , |
| Médium: | Konferenčný príspevok.. |
| Jazyk: | English |
| Vydavateľské údaje: |
IEEE
09.07.2023
|
| Predmet: | |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | Neural AutoRegressive (AR) framework has been applied in general-purpose lossless compression recently to improve compression performance. However, this paper found that directly applying the original AR framework causes the duplicated processing problem and the in-batch distribution variation problem, which leads to deteriorated compression performance. The key to address the duplicated processing problem is to disentangle the processing of the history symbol set at the input side. Two new types of neural blocks are first proposed. An individual-block performs separate feature extraction on each history symbol while a mix-block models the correlation between extracted features and estimates the probability. A progressive AR-based compression framework (PAC) is then proposed, which only requires one history symbol from the host at a time rather than the whole history symbol set. In addition, we introduced a trainable matrix multiplication to model the ordered importance, replacing previous hardware-unfriendly Gumble-Softmax sampling. The in-batch distribution variation problem is caused by AR-based compression's structured batch construction. Based on this observation, a batch-location-aware individual block is proposed to capture the heterogeneous in-batch distributions precisely, improving the performance without efficiency losses. Experimental results show the proposed framework can achieve an average of 130% speed improvement with an average of 3% compression ratio gain across data domains compared to the state-of-the-art. |
|---|---|
| AbstractList | Neural AutoRegressive (AR) framework has been applied in general-purpose lossless compression recently to improve compression performance. However, this paper found that directly applying the original AR framework causes the duplicated processing problem and the in-batch distribution variation problem, which leads to deteriorated compression performance. The key to address the duplicated processing problem is to disentangle the processing of the history symbol set at the input side. Two new types of neural blocks are first proposed. An individual-block performs separate feature extraction on each history symbol while a mix-block models the correlation between extracted features and estimates the probability. A progressive AR-based compression framework (PAC) is then proposed, which only requires one history symbol from the host at a time rather than the whole history symbol set. In addition, we introduced a trainable matrix multiplication to model the ordered importance, replacing previous hardware-unfriendly Gumble-Softmax sampling. The in-batch distribution variation problem is caused by AR-based compression's structured batch construction. Based on this observation, a batch-location-aware individual block is proposed to capture the heterogeneous in-batch distributions precisely, improving the performance without efficiency losses. Experimental results show the proposed framework can achieve an average of 130% speed improvement with an average of 3% compression ratio gain across data domains compared to the state-of-the-art. |
| Author | Li, Jingzong Xue, Jason Chun Mao, Yu Cui, Yufei |
| Author_xml | – sequence: 1 givenname: Yu surname: Mao fullname: Mao, Yu email: yumao7-c@my.cityu.edu.hk organization: City University of Hong Kong,Department of Computer Science – sequence: 2 givenname: Jingzong surname: Li fullname: Li, Jingzong email: jingzong.li@my.cityu.edu.hk organization: City University of Hong Kong,Department of Computer Science – sequence: 3 givenname: Yufei surname: Cui fullname: Cui, Yufei email: yufei.cui@mail.mcgill.ca organization: McGill University,School of Computer Science – sequence: 4 givenname: Jason Chun surname: Xue fullname: Xue, Jason Chun email: jasonxue@cityu.edu.hk organization: City University of Hong Kong,Department of Computer Science |
| BookMark | eNo1j9tKxDAYhCMoqGvfQCQvsDVJm9NlqVaFwgrq9ZI2f9fgtilJdNGnt3i4mZmPgYE5R8eTnwChK0pySom-vqlqLjTTOSOsyClhpVRCHKFMS60KTgpWlIqeoixG1xFBuCqJKM_QY2NigoDNZPFTCn7aLdD6GPcQI679OIclOD_hg0uveDMnN7ovsLh6Tz7A7qf9ANwEM8LBh7cLdDKYfYTsz1fopbl9ru_X7ebuoa7atWGapHXJdW8UB8EtlQPr9SJSEtorwxTVQ9-LwequM4xQRa3kJR_kYI3ltNPQy2KFLn93HQBs5-BGEz63_8eLbxKqUow |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IH CBEJK RIE RIO |
| DOI | 10.1109/DAC56929.2023.10247866 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP) 1998-present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| EISBN | 9798350323481 |
| EndPage | 6 |
| ExternalDocumentID | 10247866 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IH ACM ALMA_UNASSIGNED_HOLDINGS CBEJK RIE RIO |
| ID | FETCH-LOGICAL-a290t-459ca85e65d17f2c97f27701c8a2819fcc6fd9bba20181d7545f7fdad51b9ec73 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 3 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001073487300178&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 02:50:59 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-a290t-459ca85e65d17f2c97f27701c8a2819fcc6fd9bba20181d7545f7fdad51b9ec73 |
| PageCount | 6 |
| ParticipantIDs | ieee_primary_10247866 |
| PublicationCentury | 2000 |
| PublicationDate | 2023-July-9 |
| PublicationDateYYYYMMDD | 2023-07-09 |
| PublicationDate_xml | – month: 07 year: 2023 text: 2023-July-9 day: 09 |
| PublicationDecade | 2020 |
| PublicationTitle | 2023 60th ACM/IEEE Design Automation Conference (DAC) |
| PublicationTitleAbbrev | DAC |
| PublicationYear | 2023 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssib060584064 |
| Score | 2.3278801 |
| Snippet | Neural AutoRegressive (AR) framework has been applied in general-purpose lossless compression recently to improve compression performance. However, this paper... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 1 |
| SubjectTerms | auto-regressive computational efficient Correlation Design automation Feature extraction general-purpose hardware friendly History lossless data compression neural networks Picture archiving and communication systems Symbols |
| Title | Faster and Stronger Lossless Compression with Optimized Autoregressive Framework |
| URI | https://ieeexplore.ieee.org/document/10247866 |
| WOSCitedRecordID | wos001073487300178&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8NAEF5s8eBJxYpv9uA1NY_NPo6lGjxILajQW9nM7kpBU6mtB3-9M9tG8eDBS0gCSWCSnS_z-L5h7NIrmwpnIdF5ZjFACSExIrVJLbR0IJ1MC4jDJtRopCcTM96Q1SMXxnsfm898n3ZjLd_NYUWpMlzhuVBayg7rKKXWZK3246HyHoKT2LCAs9RcXQ-GpUT479OI8H578a8xKhFFqt1_Pn-P9X74eHz8jTT7bMs3B2xcWRI54LZx_IES2s94cIeY94LOi9M6X7e4NpxyrfwefcPr7NM7PiDZAh_jbHR1vGrbs3rsqbp5HN4mm_kIic1NukxEacDq0svSZSrkYHCjVJqBtlQeCwAyOFPXNidVLqfwZymo4Kwrs9p4UMUh6zbzxh8xXoBFOMPwT2ckRxPwrk65vCgAclHa-pj1yBzTt7UExrS1xMkf50_ZDhk99rWaM9ZdLlb-nG3Dx3L2vriIL-4L1EybFg |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3LSsNAFB20CrpSseLbWbhNTSbzXJZqqFhrwQrdlck8pKCp1NaFX-_caaO4cOEmJIGEMJO5Z-7jnIvQpRM6pVabRJJMBwfF-0TRVCclldwabnmam9hsQvT7cjRSgxVZPXJhnHOx-My14DTm8u3ULCBUFlY4oUJyvo42GKUkW9K16t8HEnwBnuiKB5yl6uq63WE8bABa0CS8VT_-q5FKxJFi559fsIuaP4w8PPjGmj205qp9NCg0yBxgXVn8CCHt53DRC6j3EswXhpW-LHKtMERb8UOwDq-TT2dxG4QLXPS0g7HDRV2g1URPxc2w001WHRISTVQ6TyhTRkvmOLOZ8MSocBAizYzUkCDzxnBvVVlqArpcVoTtkhfeasuyUjkj8gPUqKaVO0Q4NzoAWnAAZQaCND681QpL8twYQpkuj1AThmP8thTBGNcjcfzH_Qu01R3e98a92_7dCdqGCYhVruoUNeazhTtDm-ZjPnmfncdJ_AIU5J5d |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2023+60th+ACM%2FIEEE+Design+Automation+Conference+%28DAC%29&rft.atitle=Faster+and+Stronger+Lossless+Compression+with+Optimized+Autoregressive+Framework&rft.au=Mao%2C+Yu&rft.au=Li%2C+Jingzong&rft.au=Cui%2C+Yufei&rft.au=Xue%2C+Jason+Chun&rft.date=2023-07-09&rft.pub=IEEE&rft.spage=1&rft.epage=6&rft_id=info:doi/10.1109%2FDAC56929.2023.10247866&rft.externalDocID=10247866 |