MixHeter: A global scheduler for mixed workloads in heterogeneous environments
As data centers and applications grow more heterogeneous, allocating the proper resources to various applications increasingly depends on understanding the tradeoffs between different allocations, because mixed workloads may benefit from different resources e.g. GPU, Solid State Drives(as SSD). Howe...
Uloženo v:
| Vydáno v: | Journal of parallel and distributed computing Ročník 111; s. 93 - 103 |
|---|---|
| Hlavní autoři: | , , , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
Elsevier Inc
01.01.2018
|
| Témata: | |
| ISSN: | 0743-7315, 1096-0848 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Shrnutí: | As data centers and applications grow more heterogeneous, allocating the proper resources to various applications increasingly depends on understanding the tradeoffs between different allocations, because mixed workloads may benefit from different resources e.g. GPU, Solid State Drives(as SSD). However, traditional distributed programming models are designed and improved for homogeneous environments and have poor performances in current heterogeneous environments. Thus we reconsider the problem in this paper and make three contributions: (1) Through analysis of experimental results, we summarize the main reasons of poor performances are unreasonable allocation of tasks between heterogeneous nodes and improper allocation of resources to mixed workloads; (2) To resolve them, we propose a global scheduler MixHeter based on or-constraints. Or-constraints imbibes advantages of no-constraints and hard-constraints, which satisfy applications’ resource preferences when all the resources are available and do not waste the non-preferred resources when the preferred resources are occupied. The model of or-constraints is based on utility function, which can associate utility with different resource requests to represent resource preferences and maximize overall utility to improve system efficiency. (3) Finally, we prove MixHeter can greatly decrease execution time than capacity scheduler of Hadoop 2.7.3 and capacity scheduler with label-based scheduling up to 15%–60% in heterogeneous environments, especially in the condition of mixed workloads with different resource preferences.
•Traditional schedulers cannot benefit from heterogeneity of the cluster.•To improve them, we propose a global scheduler MixHeter based on or-constraints.•Or-constraints can satisfy resource preference and does not waste other resources.•Or-constraints employs efficient expression of utility function for complex tradeoffs.•MixHeter can maximize overall utility to improve efficiency and avoid sub-optimal. |
|---|---|
| ISSN: | 0743-7315 1096-0848 |
| DOI: | 10.1016/j.jpdc.2017.07.007 |