Regen: An object layout regenerator on large-scale production HPC systems

This article proposes an object layout regenerator called Regen which regenerates and removes the object layout dynamically to improve the read performance of applications. Regen first detects frequent access patterns from the I/O requests of the applications. Second, Regen reorganizes the objects a...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Future generation computer systems Jg. 171; H. C; S. 107830
Hauptverfasser: Sung, Dong Kyu, Kim, Sunggon, Lee, Sangjin, Tang, Houjun, Sim, Alex, Wu, Kesheng, Byna, Suren, Son, Yongseok
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Netherlands Elsevier B.V 01.10.2025
Elsevier
Schlagworte:
Distributed file system
High-performance computing
Object storage
Pattern detection
Pattern detection
Object storage
High-performance computing
Distributed file system
ISSN:0167-739X
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:This article proposes an object layout regenerator called Regen which regenerates and removes the object layout dynamically to improve the read performance of applications. Regen first detects frequent access patterns from the I/O requests of the applications. Second, Regen reorganizes the objects and regenerates or preallocates new object layouts according to the identified access patterns. Finally, Regen removes or reuses the obsolete or regenerated object layouts as necessary. As a result, Regen accelerates access to objects by providing a flexible object layout. We implement Regen as a framework on top of Proactive Data Container (PDC) and evaluate it on Cori supercomputer, a production-scale HPC system, by using realistic HPC I/O benchmarks. The experimental results show that Regen improves the I/O performance by up to 16.92× compared with an existing system. •Object storage system is gaining interest and being adopted by the HPC community.•Many HPC applications perform I/O in various access patterns.•Regen, object layout regenerator, regenerates object layouts based on frequent access patterns on original objects.•Regen regenerates or removes object layouts based on read patterns of applications.•Regen can improve the I/O performance by up to 16.92× compared with an existing object storage system.
Bibliographie:USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
AC02-05CH11231
ISSN:0167-739X
DOI:10.1016/j.future.2025.107830