Performance Characterization of Python Runtimes for Multi-device Task Parallel Programming

Modern Python programs in high-performance computing call into compiled libraries and kernels for performance-critical tasks. However, effectively parallelizing these finer-grained, and often dynamic, kernels across modern heterogeneous platforms remains a challenge. This paper designs and optimizes...

Full description

Saved in:
Bibliographic Details
Published in:International journal of parallel programming Vol. 53; no. 2; p. 16
Main Authors: Ruys, William, Lee, Hochan, You, Bozhi, Talati, Shreya, Park, Jaeyoung, Almgren-Bell, James, Yan, Yineng, Fernando, Milinda, Erez, Mattan, Gligoric, Milos, Burtscher, Martin, Rossbach, Christopher J., Pingali, Keshav, Biros, George
Format: Journal Article
Language:English
Published: New York Springer US 01.04.2025
Springer Nature B.V
Subjects:
Computer Science
Graphs
Interpreters
Libraries
Parallel programming
Processor Architectures
Python
Run time (computers)
Software Engineering/Programming and Operating Systems
Theory of Computation
Workloads
HPC in Python
Task parallel programming
GPU tasking systems
GPU programming in Python
Global Interpreter Lock
ISSN:0885-7458, 1573-7640
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Modern Python programs in high-performance computing call into compiled libraries and kernels for performance-critical tasks. However, effectively parallelizing these finer-grained, and often dynamic, kernels across modern heterogeneous platforms remains a challenge. This paper designs and optimizes a multi-threaded runtime for Python tasks on single-node multi-GPU systems, including tasks that use resources across multiple devices. We perform an experimental study which examines the impact of Python’s Global Interpreter Lock (GIL) on runtime performance and the potential gains under a GIL-less PEP703 future. This work explores tasks with variants for different different device sets, introducing new programming abstractions and runtime mechanisms to simplify their management and enhance portability. Our experimental analysis, using tasks graphs from synthetic and real applications, shows at least a 3 × (and up to 6 × ) performance improvement over its predecessor in scenarios with high GIL contention. Our implementation of multi-device tasks achieves 8 × less overhead per task relative to a multi-process alternative using Ray.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0885-7458
1573-7640
DOI:10.1007/s10766-025-00788-1