Regen: An object layout regenerator on large-scale production HPC systems

This article proposes an object layout regenerator called Regen which regenerates and removes the object layout dynamically to improve the read performance of applications. Regen first detects frequent access patterns from the I/O requests of the applications. Second, Regen reorganizes the objects a...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Future generation computer systems Ročník 171; číslo C; s. 107830
Hlavní autoři: Sung, Dong Kyu, Kim, Sunggon, Lee, Sangjin, Tang, Houjun, Sim, Alex, Wu, Kesheng, Byna, Suren, Son, Yongseok
Médium: Journal Article
Jazyk:angličtina
Vydáno: Netherlands Elsevier B.V 01.10.2025
Elsevier
Témata:
Distributed file system
High-performance computing
Object storage
Pattern detection
Pattern detection
Object storage
High-performance computing
Distributed file system
ISSN:0167-739X
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:This article proposes an object layout regenerator called Regen which regenerates and removes the object layout dynamically to improve the read performance of applications. Regen first detects frequent access patterns from the I/O requests of the applications. Second, Regen reorganizes the objects and regenerates or preallocates new object layouts according to the identified access patterns. Finally, Regen removes or reuses the obsolete or regenerated object layouts as necessary. As a result, Regen accelerates access to objects by providing a flexible object layout. We implement Regen as a framework on top of Proactive Data Container (PDC) and evaluate it on Cori supercomputer, a production-scale HPC system, by using realistic HPC I/O benchmarks. The experimental results show that Regen improves the I/O performance by up to 16.92× compared with an existing system. •Object storage system is gaining interest and being adopted by the HPC community.•Many HPC applications perform I/O in various access patterns.•Regen, object layout regenerator, regenerates object layouts based on frequent access patterns on original objects.•Regen regenerates or removes object layouts based on read patterns of applications.•Regen can improve the I/O performance by up to 16.92× compared with an existing object storage system.
Bibliografie:USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
AC02-05CH11231
ISSN:0167-739X
DOI:10.1016/j.future.2025.107830