Transparent Orchestration of Task-based Parallel Applications in Containers Platforms

This paper presents a framework to easily build and execute parallel applications in container-based distributed computing platforms in a user-transparent way. The proposed framework is a combination of the COMP Superscalar (COMPSs) programming model and runtime, which provides a straightforward way...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Journal of grid computing Jg. 16; H. 1; S. 137 - 160
Hauptverfasser: Ramon-Cortes, Cristian, Serven, Albert, Ejarque, Jorge, Lezzi, Daniele, Badia, Rosa M.
Format: Journal Article Verlag
Sprache:Englisch
Veröffentlicht: Dordrecht Springer Netherlands 01.03.2018
Springer Nature B.V
Springer Verlag
Schlagworte:
Cloud computing
Clusters
Computer networks
Computer Science
Containers
Containers Orchestration
Distributed processing
Distributed Systems
Informàtica
Linux Containers
Management of Computing and Information Systems
Parallel programming (Computer science)
Parallel Programming Models
Processor Architectures
Programació en paral·lel (Informàtica)
User Interfaces and Human Computer Interaction
Àrees temàtiques de la UPC
Cloud computing
Linux containers
Parallel programming models
Containers orchestration
Distributed systems
ISSN:1570-7873, 1572-9184
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:This paper presents a framework to easily build and execute parallel applications in container-based distributed computing platforms in a user-transparent way. The proposed framework is a combination of the COMP Superscalar (COMPSs) programming model and runtime, which provides a straightforward way to develop task-based parallel applications from sequential codes, and containers management platforms that ease the deployment of applications in computing environments (as Docker, Mesos or Singularity). This framework provides scientists and developers with an easy way to implement parallel distributed applications and deploy them in a one-click fashion. We have built a prototype which integrates COMPSs with different containers engines in different scenarios: i) a Docker cluster, ii) a Mesos cluster, and iii) Singularity in an HPC cluster. We have evaluated the overhead in the building phase, deployment and execution of two benchmark applications compared to a Cloud testbed based on KVM and OpenStack and to the usage of bare metal nodes. We have observed an important gain in comparison to cloud environments during the building and deployment phases. This enables better adaptation of resources with respect to the computational load. In contrast, we detected an extra overhead during the execution, which is mainly due to the multi-host Docker networking.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1570-7873
1572-9184
DOI:10.1007/s10723-017-9425-z