Regen: An object layout regenerator on large-scale production HPC systems - Sung, Dong Kyu; Kim, Sunggon; Lee, Sangjin; Tang, Houjun; Sim, Alex; Wu, Kesheng; Byna, Suren; Son, Yongseok (Future generation computer systems, 2025)
This article proposes an object layout regenerator called Regen which regenerates and removes the object layout dynamically to improve the read performance of applications. Regen first detects frequent access patterns from the I/O requests of the applications. Second, Regen reorganizes the objects and regenerates or preallocates new object layouts according to the identified access patterns. Finally, Regen removes or reuses the obsolete or regenerated object layouts as necessary. As a result, Regen accelerates access to objects by providing a flexible object layout. We implement Regen as a framework on top of Proactive Data Container (PDC) and evaluate it on Cori supercomputer, a production-scale HPC system, by using realistic HPC I/O benchmarks. The experimental results show that Regen improves the I/O performance by up to 16.92× compared with an existing system. •Object storage system is gaining interest and being adopted by the HPC community.•Many HPC applications perform I/O in various access patterns.•Regen, object layout regenerator, regenerates object layouts based on frequent access patterns on original objects.•Regen regenerates or removes object layouts based on read patterns of applications.•Regen can improve the I/O performance by up to 16.92× compared with an existing object storage system.