A Deep Dive into Task-Based Parallelism in Python

Modern Python programs in high-performance computing call into compiled libraries and kernels for performance-critical tasks. However, effectively parallelizing these finer-grained, and often dynamic, kernels across modern heterogeneous platforms remains a challenge. First, we perform an experimenta...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	2024 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) S. 1147 - 1149
Hauptverfasser:	Ruys, William, Lee, Hochan, You, Bozhi, Talati, Shreya, Park, Jaeyoung, Almgren-Bell, James, Yan, Yineng, Fernando, Milinda, Biros, George, Erez, Mattan, Burtscher, Martin, Rossbach, Christopher J., Pingali, Keshav, Gligoric, Milos
Format:	Tagungsbericht
Sprache:	Englisch
Veröffentlicht:	IEEE 27.05.2024
Schlagworte:	heterogeneous computing High performance computing Libraries Parallel processing parallel programming Performance evaluation Processor scheduling Programming Python Runtime task runtime
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Beschreibung
Zusammenfassung:	Modern Python programs in high-performance computing call into compiled libraries and kernels for performance-critical tasks. However, effectively parallelizing these finer-grained, and often dynamic, kernels across modern heterogeneous platforms remains a challenge. First, we perform an experimental study to examine the impact of Python's Global Interpreter Lock (GIL), and potential speedups under a GIL-less PEP703 future, to guide runtime design. Using our optimized runtime, we explore scheduling tasks with constraints that require resources across multiple, potentially diverse, devices through the introduction of new programming abstractions and runtime mechanisms. We extend an existing Python tasking library, Parla, to augment its performance and add support for such multi-device tasks. Our experimental analysis, using tasks graphs from synthetic and real applications, shows at least a 3 ×(and up to 6 ×) performance improvement over its predecessor in scenarios with high GIL contention. When scheduling multi-GPU tasks, we observe an 8x reduction in per-task launching overhead compared to a multi-process system.
DOI:	10.1109/IPDPSW63119.2024.00187