Revisiting I/O Behavior in Large-Scale Storage Systems: The Expected and the Unexpected

Large-scale applications typically spend a large fraction of their execution time performing I/O to a parallel storage system. How-ever, with rapid progress in compute and storage system stack of large-scale systems, it is critical to investigate and update our understanding of the I/O behavior of l...

Full description

Saved in:
Bibliographic Details
Published in:SC19: International Conference for High Performance Computing, Networking, Storage and Analysis pp. 1 - 13
Main Authors: Patel, Tirthak, Byna, Suren, Lockwood, Glenn K., Tiwari, Devesh
Format: Conference Proceeding
Language:English
Published: ACM 17.11.2019
Subjects:
High performance computing
Large-scale systems
Monitoring
Parallel processing
Processor scheduling
Production
ISSN:2167-4337
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Large-scale applications typically spend a large fraction of their execution time performing I/O to a parallel storage system. How-ever, with rapid progress in compute and storage system stack of large-scale systems, it is critical to investigate and update our understanding of the I/O behavior of large-scale applications. Toward that end, in this work, we monitor, collect and analyze a year worth of storage system data from a large-scale production parallel stor-age system. We perform temporal, spatial and correlative analysis of the system and uncover surprising patterns which defy existing assumptions and have important implications for future systems.
ISSN:2167-4337
DOI:10.1145/3295500.3356183