Performance Characterization of Python Runtimes for Multi-device Task Parallel Programming

Modern Python programs in high-performance computing call into compiled libraries and kernels for performance-critical tasks. However, effectively parallelizing these finer-grained, and often dynamic, kernels across modern heterogeneous platforms remains a challenge. This paper designs and optimizes...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:International journal of parallel programming Ročník 53; číslo 2; s. 16
Hlavní autoři: Ruys, William, Lee, Hochan, You, Bozhi, Talati, Shreya, Park, Jaeyoung, Almgren-Bell, James, Yan, Yineng, Fernando, Milinda, Erez, Mattan, Gligoric, Milos, Burtscher, Martin, Rossbach, Christopher J., Pingali, Keshav, Biros, George
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York Springer US 01.04.2025
Springer Nature B.V
Témata:
Computer Science
Graphs
Interpreters
Libraries
Parallel programming
Processor Architectures
Python
Run time (computers)
Software Engineering/Programming and Operating Systems
Theory of Computation
Workloads
HPC in Python
Task parallel programming
GPU tasking systems
GPU programming in Python
Global Interpreter Lock
ISSN:0885-7458, 1573-7640
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Modern Python programs in high-performance computing call into compiled libraries and kernels for performance-critical tasks. However, effectively parallelizing these finer-grained, and often dynamic, kernels across modern heterogeneous platforms remains a challenge. This paper designs and optimizes a multi-threaded runtime for Python tasks on single-node multi-GPU systems, including tasks that use resources across multiple devices. We perform an experimental study which examines the impact of Python’s Global Interpreter Lock (GIL) on runtime performance and the potential gains under a GIL-less PEP703 future. This work explores tasks with variants for different different device sets, introducing new programming abstractions and runtime mechanisms to simplify their management and enhance portability. Our experimental analysis, using tasks graphs from synthetic and real applications, shows at least a 3 × (and up to 6 × ) performance improvement over its predecessor in scenarios with high GIL contention. Our implementation of multi-device tasks achieves 8 × less overhead per task relative to a multi-process alternative using Ray.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0885-7458
1573-7640
DOI:10.1007/s10766-025-00788-1