Improving I/O performance in HPC environments using the Expand Ad-Hoc file system

This work introduces an exhaustive evaluation, including both benchmarks and real-world data-intensive applications, performed in MareNostrum 4 and HPC4AI Laboratory supercomputers using the Expand Ad-Hoc file system. Expand Ad-Hoc is an ad-hoc file system that dynamically virtualizes the local stor...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:The Journal of supercomputing Ročník 81; číslo 16; s. 1541
Hlavní autori: Camarmas-Alonso, Diego, Garcia-Carballeira, Felix, Calderon-Mateos, Alejandro, Carretero, Jesus
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: New York Springer US 08.11.2025
Springer Nature B.V
Predmet:
Benchmarks
Bottlenecks
Compilers
Computer Science
Design
High performance computing
Interpreters
Metadata
Performance evaluation
Processor Architectures
Programming Languages
Remote sensing
Software
Storage area networks
Supercomputers
Data-intensive application
Parallel system evaluation
Ad hoc file system
High-Performance Computing (HPC)
Distributed file system
ISSN:1573-0484, 0920-8542, 1573-0484
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:This work introduces an exhaustive evaluation, including both benchmarks and real-world data-intensive applications, performed in MareNostrum 4 and HPC4AI Laboratory supercomputers using the Expand Ad-Hoc file system. Expand Ad-Hoc is an ad-hoc file system that dynamically virtualizes the local storage available on compute nodes (i.e., SSD, SHM, etc.) into a fast storage volume to reduce congestion on parallel file systems used as backends in High-Performance Computing (HPC) environments. The main contributions of this work include the design of a new ad-hoc parallel file system, called Expand Ad-Hoc , compatible with POSIX and MPI-IO, and an exhaustive and comprehensive evaluation of Expand. The evaluation compares the performance obtained using IOR and DLIO benchmarks and Nek5000 and Remote Sensing real-world applications on Expand Ad-Hoc , GekkoFS, GPFS, and BeeGFS, showing satisfactory results proving that Expand Ad-Hoc can be used in HPC environments to improve the I/O performance of data-intensive applications transparently.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1573-0484
0920-8542
1573-0484
DOI:10.1007/s11227-025-08015-x