Inspector-Executor Load Balancing Algorithms for Block-Sparse Tensor Contractions

Developing effective yet scalable load-balancing methods for irregular computations is critical to the successful application of simulations in a variety of disciplines at petascale and beyond. This paper explores a set of static and dynamic scheduling algorithms for block-sparse tensor contractions...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Proceedings of the International Conference on Parallel Processing s. 30 - 39
Hlavní autoři: Ozog, David, Hammond, Jeff R., Dinan, James, Balaji, Pavan, Shende, Sameer, Malony, Allen
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 01.10.2013
Témata:
Computational efficiency
Computational modeling
Dynamic Load Balancing
Global Arrays
Load management
Load modeling
Quantum Chemistry
Radiation detectors
Static Partitioning
Tensile stress
Tensor Contractions
Tiles
ISSN:0190-3918
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Developing effective yet scalable load-balancing methods for irregular computations is critical to the successful application of simulations in a variety of disciplines at petascale and beyond. This paper explores a set of static and dynamic scheduling algorithms for block-sparse tensor contractions within the NWChem computational chemistry code for different degrees of sparsity (and therefore load imbalance). In this particular application, a relatively large amount of task information can be obtained at minimal cost, which enables the use of static partitioning techniques that take the entire task list as input. However, fully static partitioning is incapable of dealing with dynamic variation of task costs, such as from transient network contention or operating system noise, so we also consider hybrid schemes that utilize dynamic scheduling within subgroups. These two schemes, which have not been previously implemented in NWChem or its proxies (i.e. quantum chemistry mini-apps) are compared to the original centralized dynamic load-balancing algorithm as well as improved centralized scheme. In all cases, we separate the scheduling of tasks from the execution of tasks into an inspector phase and an executor phase. The impact of these methods upon the application is substantial on a large InfiniBand cluster: execution time is reduced by as much as 50% at scale. The technique is applicable to any scientific application requiring load balance where performance models or estimations of kernel execution times are available.
ISSN:0190-3918
DOI:10.1109/ICPP.2013.12