An adaptive hierarchical master–worker (AHMW) framework for grids—Application to B&B algorithms

Well-suited to embarrassingly parallel applications, the master–worker (MW) paradigm has largely and successfully used in parallel distributed computing. Nevertheless, such a paradigm is very limited in scalability in large computational grids. A natural way to improve the scalability is to add a la...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Journal of parallel and distributed computing Ročník 72; číslo 2; s. 120 - 131
Hlavní autori: Bendjoudi, A., Melab, N., Talbi, E-G.
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Elsevier Inc 01.02.2012
Predmet:
Algorithms
Branch and bound
Computer networks
Decomposition
Distributed processing
Dynamics
Exploration
Grid computing
Hierarchical master–worker
Hierarchies
Large scale experiments
Master–worker
Tasks
Grid computing
Hierarchical master–worker
Large scale experiments
Branch and bound
Master–worker
ISSN:0743-7315, 1096-0848
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Well-suited to embarrassingly parallel applications, the master–worker (MW) paradigm has largely and successfully used in parallel distributed computing. Nevertheless, such a paradigm is very limited in scalability in large computational grids. A natural way to improve the scalability is to add a layer of masters between the master and the workers making a hierarchical MW (HMW). In most existing HMW frameworks and algorithms, only a single layer of masters is used, the hierarchy is statically built and the granularity of tasks is fixed. Such frameworks and algorithms are not adapted to grids which are volatile, heterogeneous and large scale environments. In this paper, we revisit the HMW paradigm to match such characteristics of grids. We propose a new dynamic adaptive multi-layer hierarchical MW ( AHMW) dealing with the scalability, volatility and heterogeneity issues. The construction and deployment of the hierarchy and the task management (deployment, decomposition of work, distribution of tasks, … ) are performed in a dynamic collaborative distributed way. The framework has been applied to the parallel Branch and Bound algorithm and experimented on the Flow-Shop scheduling problem. The implementation has been performed using the ProActive grid middleware and the large experiments have been conducted using about 2000 processors from the Grid’5000 French nation-wide grid infrastructure. The results demonstrate the high scalability of the proposed approach and its efficiency in terms of deployment cost, decomposition and distribution of work and exploration time. The results show that AHMW outperforms HMW and MW in scalability and efficiency in terms of deployment and exploration time. ► The hierarchical master–worker (HMW) paradigm is rethought to make it efficiently exploitable in large scale grids. ► HMW is built and deployed in a dynamic adaptive way to take into account the volatility of resources. ► The task management, including task decomposition and distribution, for MHW-based applications is dynamic and distributed.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:0743-7315
1096-0848
DOI:10.1016/j.jpdc.2011.10.002