Enabling Low-Overhead HT-HPC Workflows at Extreme Scale using GNU Parallel

GNU Parallel is a versatile and powerful tool for process parallelization widely used in scientific computing. This paper demonstrates its effective application in high-performance computing (HPC) environments, particularly focusing on its scalability and efficiency in executing large-scale high-thr...

Full description

Bibliographic Details

Published in:	SC24-W: Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis pp. 2056 - 2063
Main Authors:	Maheshwari, Ketan, Arndt, William, Karimi, Ahmad Maroof, Yin, Junqi, Suter, Frederic, Johnson, Seth, Da Silva, Rafael Ferreira
Format:	Conference Proceeding
Language:	English
Published:	IEEE 17.11.2024
Subjects:	Data transfer Distributed Computing GNU Parallel Graphics processing units High performance computing High-Throughput Computing Python Resource management Scalability Scientific computing Scientific Workflows Supercomputers Transforms Workflow management software
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!