Accelerating XOR-Based Erasure Coding using Program Optimization Techniques

Erasure coding (EC) affords data redundancy for large-scale systems. XOR-based EC is an easy-to-implement method for optimizing EC. This paper addresses a significant performance gap between the state-of-the-art XOR-based EC approach (~4.9 GB/s coding through-put) and Intel's high-performance E...

Full description

Saved in:
Bibliographic Details
Published in:SC21: International Conference for High Performance Computing, Networking, Storage and Analysis pp. 1 - 15
Main Author: Uezato, Yuya
Format: Conference Proceeding
Language:English
Published: ACM 14.11.2021
Subjects:
Encoding
Games
High performance computing
Optimization methods
Program processors
Redundancy
Throughput
ISSN:2167-4337
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Erasure coding (EC) affords data redundancy for large-scale systems. XOR-based EC is an easy-to-implement method for optimizing EC. This paper addresses a significant performance gap between the state-of-the-art XOR-based EC approach (~4.9 GB/s coding through-put) and Intel's high-performance EC library based on another approach (~6.7 GB/s). We propose a novel approach based on our observation that XOR-based EC virtually generates programs of a Domain Specific Language for XORing byte arrays. We formalize such programs as straight-line programs (SLPs) of compiler construction and optimize SLPs using various program optimization techniques. Our optimization flow is three-fold: 1) reducing the number of XORs using grammar compression algorithms; 2) reducing memory accesses using deforestation, a functional program optimization method; and 3) reducing cache misses using the (red-blue) pebble game of program analysis. We provide an experimentallibrary, which outperforms Intel's library with an \sim 8.92 GB/s throughput.
AbstractList Erasure coding (EC) affords data redundancy for large-scale systems. XOR-based EC is an easy-to-implement method for optimizing EC. This paper addresses a significant performance gap between the state-of-the-art XOR-based EC approach (~4.9 GB/s coding through-put) and Intel's high-performance EC library based on another approach (~6.7 GB/s). We propose a novel approach based on our observation that XOR-based EC virtually generates programs of a Domain Specific Language for XORing byte arrays. We formalize such programs as straight-line programs (SLPs) of compiler construction and optimize SLPs using various program optimization techniques. Our optimization flow is three-fold: 1) reducing the number of XORs using grammar compression algorithms; 2) reducing memory accesses using deforestation, a functional program optimization method; and 3) reducing cache misses using the (red-blue) pebble game of program analysis. We provide an experimentallibrary, which outperforms Intel's library with an \sim 8.92 GB/s throughput.
Author Uezato, Yuya
Author_xml – sequence: 1
  givenname: Yuya
  surname: Uezato
  fullname: Uezato, Yuya
  email: yuuya_uezato@dwango.co.jp
  organization: Dwango, Co., Ltd.,Japan
BookMark eNotj8tOwzAURA0CiVKyZsEmP5Bi-_q5LFF5iEpBqEjsKse5KUFNUuxkAV9PKtjMLI7OSHNJzrq-Q0KuGV0wJuQtCGkM0wsQWnEqTkhitZkABSMEZ6dkxpnSmQDQFySJ8ZNSyo1mwOmMPC-9xz0GNzTdLn0vXrM7F7FKV8HFMWCa99URjPGYL6HfBdemxWFo2uZncvou3aD_6JqvEeMVOa_dPmLy33Pydr_a5I_Zunh4ypfrzHGhh8xLwbgEWQO3FrXk3CHltSqdV1hahVBqxqcn4MB4rY0XzFdGWQWltw5gTm7-dhtE3B5C07rwvbWWUSoBfgEiXE5w
ContentType Conference Proceeding
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1145/3458817.3476204
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Xplore POP ALL
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Xplore Digital Libary (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISBN 9781450384421
1450384420
EISSN 2167-4337
EndPage 15
ExternalDocumentID 9910053
Genre orig-research
GroupedDBID 6IE
6IF
6IH
6IK
6IL
6IN
AAWTH
ABLEC
ADZIZ
ALMA_UNASSIGNED_HOLDINGS
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CBEJK
CHZPO
IEGSK
IPLJI
OCL
RIE
RIL
ID FETCH-LOGICAL-a247t-c5412535f3299e7522ae02f6bac6eb96e3b7122043a38c778c41cd86963bc9a33
IEDL.DBID RIE
ISICitedReferencesCount 13
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000946520100014&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Wed Aug 27 02:18:42 EDT 2025
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-a247t-c5412535f3299e7522ae02f6bac6eb96e3b7122043a38c778c41cd86963bc9a33
PageCount 15
ParticipantIDs ieee_primary_9910053
PublicationCentury 2000
PublicationDate 2021-Nov.-14
PublicationDateYYYYMMDD 2021-11-14
PublicationDate_xml – month: 11
  year: 2021
  text: 2021-Nov.-14
  day: 14
PublicationDecade 2020
PublicationTitle SC21: International Conference for High Performance Computing, Networking, Storage and Analysis
PublicationTitleAbbrev SC
PublicationYear 2021
Publisher ACM
Publisher_xml – name: ACM
SSID ssj0002871320
ssj0003204180
Score 1.8683007
Snippet Erasure coding (EC) affords data redundancy for large-scale systems. XOR-based EC is an easy-to-implement method for optimizing EC. This paper addresses a...
SourceID ieee
SourceType Publisher
StartPage 1
SubjectTerms Encoding
Games
High performance computing
Optimization methods
Program processors
Redundancy
Throughput
Title Accelerating XOR-Based Erasure Coding using Program Optimization Techniques
URI https://ieeexplore.ieee.org/document/9910053
WOSCitedRecordID wos000946520100014&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NS8NAEB1q8eCpait-swePpu1mN7ubo5YWQWmLVOit7G4mxYOp9MPf706aVgQvkssmkBB2M5mXyXtvAO5sYrtpnugoRS4jGcssMtgN4e6McF6pzGRl15IXPRya6TQd1-B-r4VBxJJ8hm0alv_ys4XfUKmsE7AMPTQHcKC12mq19vUUQv6igj60H8aSm27l5sNl0hEkyuS6LaQmE_Zf7VTKbDJo_O8-jqH1I8tj433COYEaFqfQ2PVlYFWYNuH5wfuQTWhtizmbjl6jx5CrMtZflvVA1lvQ-Ywo73O6IBG02Ci8Oz4qUSab7JxdVy14G_QnvaeoapoQ2VjqdeQTGTCLSHIREg3qAK8sduNcOesVulShcJrHpIi1wnitjZfcZ0aFQHQ-tUKcQb1YFHgODLnLFXeJI0OZ8PlqrZVCYC7DZnNjL6BJczP73PpizKppufz78BUcxcQHIQqdvIb6ernBGzj0X-v31fK2XMxv8VCefg
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PT8IwFH5BNNETKhh_24NHB-vatd1RCQQDAjGYcCNt1xEPguGHf799Y2BMvJhduiVblnZv37e3930P4F7HOkyyWAaJozzgEU8D5UIf7kYxY4VIVZp3LenJfl-Nx8mwBA87LYxzLi8-c3Uc5v_y07ldY6qs4bkMPjR7sB9zHoUbtdYuo4LcnxXkB_f9mFMVFn4-lMcNhrJMKuuMS7Rh_9VQJceTduV_d3IMtR9hHhnuIOcESm52CpVtZwZSBGoVuo_WejzB1Z1NyXjwGjx5tEpJa5FnBElzjucTLHqf4gWxRIsM_Nvjo5BlktHW23VZg7d2a9TsBEXbhEBHXK4CG3PPWlicMQ81TnqCpV0YZcJoK5xJhGNG0gg1sZopK6WynNpUCR-KxiaasTMoz-Yzdw7EUZMJamKDljL-A1ZrzRlzGfebzpS-gCrOzeRz44wxKabl8u_Dd3DYGb30Jr3nfvcKjmiiNjUw4hrKq8Xa3cCB_Vq9Lxe3-cJ-A2Nvodk
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=SC21%3A+International+Conference+for+High+Performance+Computing%2C+Networking%2C+Storage+and+Analysis&rft.atitle=Accelerating+XOR-Based+Erasure+Coding+using+Program+Optimization+Techniques&rft.au=Uezato%2C+Yuya&rft.date=2021-11-14&rft.pub=ACM&rft.eissn=2167-4337&rft.spage=1&rft.epage=15&rft_id=info:doi/10.1145%2F3458817.3476204&rft.externalDocID=9910053