Performance Characterization of Python Runtimes for Multi-device Task Parallel Programming

Modern Python programs in high-performance computing call into compiled libraries and kernels for performance-critical tasks. However, effectively parallelizing these finer-grained, and often dynamic, kernels across modern heterogeneous platforms remains a challenge. This paper designs and optimizes...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:International journal of parallel programming Ročník 53; číslo 2; s. 16
Hlavní autori: Ruys, William, Lee, Hochan, You, Bozhi, Talati, Shreya, Park, Jaeyoung, Almgren-Bell, James, Yan, Yineng, Fernando, Milinda, Erez, Mattan, Gligoric, Milos, Burtscher, Martin, Rossbach, Christopher J., Pingali, Keshav, Biros, George
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: New York Springer US 01.04.2025
Springer Nature B.V
Predmet:
Computer Science
Graphs
Interpreters
Libraries
Parallel programming
Processor Architectures
Python
Run time (computers)
Software Engineering/Programming and Operating Systems
Theory of Computation
Workloads
HPC in Python
Task parallel programming
GPU tasking systems
GPU programming in Python
Global Interpreter Lock
ISSN:0885-7458, 1573-7640
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Modern Python programs in high-performance computing call into compiled libraries and kernels for performance-critical tasks. However, effectively parallelizing these finer-grained, and often dynamic, kernels across modern heterogeneous platforms remains a challenge. This paper designs and optimizes a multi-threaded runtime for Python tasks on single-node multi-GPU systems, including tasks that use resources across multiple devices. We perform an experimental study which examines the impact of Python’s Global Interpreter Lock (GIL) on runtime performance and the potential gains under a GIL-less PEP703 future. This work explores tasks with variants for different different device sets, introducing new programming abstractions and runtime mechanisms to simplify their management and enhance portability. Our experimental analysis, using tasks graphs from synthetic and real applications, shows at least a 3 × (and up to 6 × ) performance improvement over its predecessor in scenarios with high GIL contention. Our implementation of multi-device tasks achieves 8 × less overhead per task relative to a multi-process alternative using Ray.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0885-7458
1573-7640
DOI:10.1007/s10766-025-00788-1