Revisiting I/O Behavior in Large-Scale Storage Systems: The Expected and the Unexpected

Large-scale applications typically spend a large fraction of their execution time performing I/O to a parallel storage system. How-ever, with rapid progress in compute and storage system stack of large-scale systems, it is critical to investigate and update our understanding of the I/O behavior of l...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:SC19: International Conference for High Performance Computing, Networking, Storage and Analysis S. 1 - 13
Hauptverfasser: Patel, Tirthak, Byna, Suren, Lockwood, Glenn K., Tiwari, Devesh
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: ACM 17.11.2019
Schlagworte:
High performance computing
Large-scale systems
Monitoring
Parallel processing
Processor scheduling
Production
ISSN:2167-4337
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Abstract Large-scale applications typically spend a large fraction of their execution time performing I/O to a parallel storage system. How-ever, with rapid progress in compute and storage system stack of large-scale systems, it is critical to investigate and update our understanding of the I/O behavior of large-scale applications. Toward that end, in this work, we monitor, collect and analyze a year worth of storage system data from a large-scale production parallel stor-age system. We perform temporal, spatial and correlative analysis of the system and uncover surprising patterns which defy existing assumptions and have important implications for future systems.
AbstractList Large-scale applications typically spend a large fraction of their execution time performing I/O to a parallel storage system. How-ever, with rapid progress in compute and storage system stack of large-scale systems, it is critical to investigate and update our understanding of the I/O behavior of large-scale applications. Toward that end, in this work, we monitor, collect and analyze a year worth of storage system data from a large-scale production parallel stor-age system. We perform temporal, spatial and correlative analysis of the system and uncover surprising patterns which defy existing assumptions and have important implications for future systems.
Author Patel, Tirthak
Lockwood, Glenn K.
Byna, Suren
Tiwari, Devesh
Author_xml – sequence: 1
  givenname: Tirthak
  surname: Patel
  fullname: Patel, Tirthak
  organization: Northeastern University,USA
– sequence: 2
  givenname: Suren
  surname: Byna
  fullname: Byna, Suren
  organization: Lawrence Berkeley National Laboratory,USA
– sequence: 3
  givenname: Glenn K.
  surname: Lockwood
  fullname: Lockwood, Glenn K.
  organization: Lawrence Berkeley National Laboratory,USA
– sequence: 4
  givenname: Devesh
  surname: Tiwari
  fullname: Tiwari, Devesh
  organization: Northeastern University,USA
BookMark eNotjF1LAkEUhqcoqMzrbrqYP7B6Zs58dldiJQhCKl3K7HhGJ3SV3UXy37eQVw8878P7wG6qQ0WMPQkYCKH0EKXXGmCAqI1weMX63rpuADRSerhm91IYWyhEe8f6TfMDAChBCQf37PuLTrnJba42fDKc8TfahlM-1DxXfBrqDRXzGHbE5-2hDpuO56alffPCF1vi498jxZbWPFRr3nZiWdFFPbLbFHYN9S_sseX7eDH6LKazj8nodVoEqVxbhFCWMUlntLfCBycwKZJkUooikpMqoLBar0uK3qqYjCi7IFlpfCojeuyx5__fTESrY533oT6vBHiQ0gL-AYowU3M
ContentType Conference Proceeding
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1145/3295500.3356183
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Xplore POP ALL
IEEE Xplore All Conference Proceedings
IEEE/IET Electronic Library
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISBN 9781450362290
145036229X
EISSN 2167-4337
EndPage 13
ExternalDocumentID 10902270
Genre orig-research
GrantInformation_xml – fundername: U.S. Department of Energy
  grantid: DE-AC02-05CHl1231
  funderid: 10.13039/100000015
GroupedDBID 6IE
6IF
6IH
6IK
6IL
6IN
AAWTH
ABLEC
ADZIZ
ALMA_UNASSIGNED_HOLDINGS
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CBEJK
CHZPO
IEGSK
IPLJI
OCL
RIE
RIL
ID FETCH-LOGICAL-a248t-aabbcf28659719a813f4e2e6ffc1ce824a31755dbec974cf61bf4ef7269fbc393
IEDL.DBID RIE
ISICitedReferencesCount 40
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000545976800065&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Wed Mar 12 06:17:11 EDT 2025
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-a248t-aabbcf28659719a813f4e2e6ffc1ce824a31755dbec974cf61bf4ef7269fbc393
PageCount 13
ParticipantIDs ieee_primary_10902270
PublicationCentury 2000
PublicationDate 2019-Nov.-17
PublicationDateYYYYMMDD 2019-11-17
PublicationDate_xml – month: 11
  year: 2019
  text: 2019-Nov.-17
  day: 17
PublicationDecade 2010
PublicationTitle SC19: International Conference for High Performance Computing, Networking, Storage and Analysis
PublicationTitleAbbrev SC
PublicationYear 2019
Publisher ACM
Publisher_xml – name: ACM
SSID ssj0003204180
ssj0002871321
Score 1.9715044
Snippet Large-scale applications typically spend a large fraction of their execution time performing I/O to a parallel storage system. How-ever, with rapid progress in...
SourceID ieee
SourceType Publisher
StartPage 1
SubjectTerms High performance computing
Large-scale systems
Monitoring
Parallel processing
Processor scheduling
Production
Title Revisiting I/O Behavior in Large-Scale Storage Systems: The Expected and the Unexpected
URI https://ieeexplore.ieee.org/document/10902270
WOSCitedRecordID wos000545976800065&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV09T8MwELWgYmAqH0V8ywOrafzROGZFVCChUlEqulW2Y0tdUlRafj93rltYGNgSK0Pkc_Le2ffeEXIj8WjLacvq2gsGEGAZ4I5nsFh8bX2BDZJSswk9GFSTiRlmsXrSwoQQUvFZuMXLdJZfz_0Kt8q6qYhQaMjQd7Uu12Kt7YYKUn-ZsQjvpSgUr4ps58NVryuFAT4OaaoE0oA-gb_6qSQ46bf_-SIHpPMjzKPDLeQckp3QHJH2pjMDzR_qMXl_TaJxLGmmT90Xml0QF3TW0Ges_WYjiE2gI8i44YdCs2_5HYVVQ9H92AMRpbapKfBDOm5CHuqQcf_h7f6R5RYKzApVLZm1zvmI6lOjubEVl1EFEcoYPfehEsoif-jVEElILHwsuYMHohalic5LI09Iq5k34ZRQADERXRmLIgoVnDMKGxKWVmljo-PFGengRE0_1i4Z080cnf8xfkH2gXwY1PVxfUlay8UqXJE9_7WcfS6uU2y_ASZbo08
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV09T8MwELVQQYKpfBTxjQdW0_ijScyKQK0opYJWdKtsx5a6pKi0_H7ujAssDGyJlSHyObl39r33CLmSeLRlC8OqygkGKcAwyDuOwWJxlXEZGiRFs4liMCgnEz1MZPXIhfHex-Yzf42X8Sy_mrsVbpW1YxOhKKBC30TrrETX-t5SQfAvUzbCeykyxcssCfpw1WlLoQGRQ6EqATagUuAvR5WYUO6b_3yVXdL6oebR4XfS2SMbvt4nzbU3A02f6gF5fY60cWxqpr32E006iAs6q2kfu7_ZC0TH0xeoueGXQpNy-Q2FdUNR_9gBFKWmriggRDqufRpqkfH93ei2y5KJAjNClUtmjLUuIP9UF1ybksugvPB5CI47XwplEEF0KogllBYu5NzCA6EQuQ7WSS0PSaOe1_6IUEhjItg8ZFkQylurFVoS5kYV2gTLs2PSwomavn3pZEzXc3Tyx_gl2e6OHvvTfm_wcEp2AIpoZPnx4ow0louVPydb7mM5e19cxDh_Atjyppg
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=SC19%3A+International+Conference+for+High+Performance+Computing%2C+Networking%2C+Storage+and+Analysis&rft.atitle=Revisiting+I%2FO+Behavior+in+Large-Scale+Storage+Systems%3A+The+Expected+and+the+Unexpected&rft.au=Patel%2C+Tirthak&rft.au=Byna%2C+Suren&rft.au=Lockwood%2C+Glenn+K.&rft.au=Tiwari%2C+Devesh&rft.date=2019-11-17&rft.pub=ACM&rft.eissn=2167-4337&rft.spage=1&rft.epage=13&rft_id=info:doi/10.1145%2F3295500.3356183&rft.externalDocID=10902270