A Deep Dive into Task-Based Parallelism in Python

Modern Python programs in high-performance computing call into compiled libraries and kernels for performance-critical tasks. However, effectively parallelizing these finer-grained, and often dynamic, kernels across modern heterogeneous platforms remains a challenge. First, we perform an experimenta...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:2024 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) s. 1147 - 1149
Hlavní autori: Ruys, William, Lee, Hochan, You, Bozhi, Talati, Shreya, Park, Jaeyoung, Almgren-Bell, James, Yan, Yineng, Fernando, Milinda, Biros, George, Erez, Mattan, Burtscher, Martin, Rossbach, Christopher J., Pingali, Keshav, Gligoric, Milos
Médium: Konferenčný príspevok..
Jazyk:English
Vydavateľské údaje: IEEE 27.05.2024
Predmet:
heterogeneous computing
High performance computing
Libraries
Parallel processing
parallel programming
Performance evaluation
Processor scheduling
Programming
Python
Runtime
task runtime
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Modern Python programs in high-performance computing call into compiled libraries and kernels for performance-critical tasks. However, effectively parallelizing these finer-grained, and often dynamic, kernels across modern heterogeneous platforms remains a challenge. First, we perform an experimental study to examine the impact of Python's Global Interpreter Lock (GIL), and potential speedups under a GIL-less PEP703 future, to guide runtime design. Using our optimized runtime, we explore scheduling tasks with constraints that require resources across multiple, potentially diverse, devices through the introduction of new programming abstractions and runtime mechanisms. We extend an existing Python tasking library, Parla, to augment its performance and add support for such multi-device tasks. Our experimental analysis, using tasks graphs from synthetic and real applications, shows at least a 3 ×(and up to 6 ×) performance improvement over its predecessor in scenarios with high GIL contention. When scheduling multi-GPU tasks, we observe an 8x reduction in per-task launching overhead compared to a multi-process system.
DOI:10.1109/IPDPSW63119.2024.00187