Regen: An object layout regenerator on large-scale production HPC systems

This article proposes an object layout regenerator called Regen which regenerates and removes the object layout dynamically to improve the read performance of applications. Regen first detects frequent access patterns from the I/O requests of the applications. Second, Regen reorganizes the objects a...

Full description

Saved in:
Bibliographic Details
Published in:Future generation computer systems Vol. 171; no. C; p. 107830
Main Authors: Sung, Dong Kyu, Kim, Sunggon, Lee, Sangjin, Tang, Houjun, Sim, Alex, Wu, Kesheng, Byna, Suren, Son, Yongseok
Format: Journal Article
Language:English
Published: Netherlands Elsevier B.V 01.10.2025
Elsevier
Subjects:
Distributed file system
High-performance computing
Object storage
Pattern detection
Pattern detection
Object storage
High-performance computing
Distributed file system
ISSN:0167-739X
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This article proposes an object layout regenerator called Regen which regenerates and removes the object layout dynamically to improve the read performance of applications. Regen first detects frequent access patterns from the I/O requests of the applications. Second, Regen reorganizes the objects and regenerates or preallocates new object layouts according to the identified access patterns. Finally, Regen removes or reuses the obsolete or regenerated object layouts as necessary. As a result, Regen accelerates access to objects by providing a flexible object layout. We implement Regen as a framework on top of Proactive Data Container (PDC) and evaluate it on Cori supercomputer, a production-scale HPC system, by using realistic HPC I/O benchmarks. The experimental results show that Regen improves the I/O performance by up to 16.92× compared with an existing system. •Object storage system is gaining interest and being adopted by the HPC community.•Many HPC applications perform I/O in various access patterns.•Regen, object layout regenerator, regenerates object layouts based on frequent access patterns on original objects.•Regen regenerates or removes object layouts based on read patterns of applications.•Regen can improve the I/O performance by up to 16.92× compared with an existing object storage system.
Bibliography:USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
AC02-05CH11231
ISSN:0167-739X
DOI:10.1016/j.future.2025.107830