Distributed scheduling and data sharing in late-binding overlays

Pull-based late-binding overlays are used in some of today's largest computational grids. Job agents are submitted to resources with the duty of retrieving real workload from a central queue at runtime. This helps overcome the problems of these complex environments: heterogeneity, imprecise sta...

Full description

Saved in:
Bibliographic Details
Published in:2014 International Conference on High Performance Computing & Simulation (HPCS) pp. 129 - 136
Main Authors: Delgado Peris, Antonio, Hernandez, Jose M., Huedo, Eduardo
Format: Conference Proceeding
Language:English
Published: IEEE 01.07.2014
Subjects:
Distributed algorithms
Grid and Cluster Computing
Measurement
Peer-to-Peer Architectures and Networks
Peer-to-peer computing
Reliable Parallel and Distributed Algorithms
Scalable Computing
ISBN:9781479953127, 1479953121
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Pull-based late-binding overlays are used in some of today's largest computational grids. Job agents are submitted to resources with the duty of retrieving real workload from a central queue at runtime. This helps overcome the problems of these complex environments: heterogeneity, imprecise status information and relatively high failure rates. In addition, the late job assignment allows dynamic adaptation to changes in grid conditions or user priorities. However, as the scale grows, the central assignment queue may become a bottleneck for the whole system. This article presents a distributed scheduling architecture for late-binding overlays, which addresses this issue by letting execution nodes build a distributed hash table and delegating job matching and assignment to them. This reduces the load on the central server and makes the system much more scalable and robust. Scalability makes fine-grained scheduling possible and enables new functionalities, like the implementation of a distributed data cache on the execution nodes, which helps alleviate the commonly congested grid storage services.
AbstractList Pull-based late-binding overlays are used in some of today's largest computational grids. Job agents are submitted to resources with the duty of retrieving real workload from a central queue at runtime. This helps overcome the problems of these complex environments: heterogeneity, imprecise status information and relatively high failure rates. In addition, the late job assignment allows dynamic adaptation to changes in grid conditions or user priorities. However, as the scale grows, the central assignment queue may become a bottleneck for the whole system. This article presents a distributed scheduling architecture for late-binding overlays, which addresses this issue by letting execution nodes build a distributed hash table and delegating job matching and assignment to them. This reduces the load on the central server and makes the system much more scalable and robust. Scalability makes fine-grained scheduling possible and enables new functionalities, like the implementation of a distributed data cache on the execution nodes, which helps alleviate the commonly congested grid storage services.
Author Delgado Peris, Antonio
Hernandez, Jose M.
Huedo, Eduardo
Author_xml – sequence: 1
  givenname: Antonio
  surname: Delgado Peris
  fullname: Delgado Peris, Antonio
  email: antonio.delgadoperis@ciemat.es
  organization: Sci. Comput. Unit, CIEMAT, Madrid, Spain
– sequence: 2
  givenname: Jose M.
  surname: Hernandez
  fullname: Hernandez, Jose M.
  email: jose.hernandez@ciemat.es
  organization: Sci. Comput. Unit, CIEMAT, Madrid, Spain
– sequence: 3
  givenname: Eduardo
  surname: Huedo
  fullname: Huedo, Eduardo
  email: ehuedo@fdi.ucm.es
  organization: Fac. de Inf., Univ. Complutense de Madrid (UCM), Madrid, Spain
BookMark eNo1j8FKAzEURSPqwtZ-QTfzAzPm5WWSyU4ZtRUKCuq6vDRvbGCaSmYq9O-lWFeHcxYX7kRcpX1iIeYgKwDp7pZv7XvcVUqCroyTaGxzISagrXM1AuClmDnb_LuyN-L-MQ5jjv4wciiGzZbDoY_pq6AUikAjFcOW8inEVPQ0culjCiff_3Du6TjciuuO-oFnZ07F5_PTR7ssV6-Ll_ZhVUZQOJYNEJrOhsZ51xnrENkxQbBaQaglkNJMiMYz1rXxqjNSMWvcWK-xA49TMf_bjcy8_s5xR_m4Pp_EXxisSaU
ContentType Conference Proceeding
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1109/HPCSim.2014.6903678
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Proceedings Order Plan All Online (POP All Online) 1998-present by volume
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
EISBN 1479953113
9781479953134
147995313X
9781479953110
EndPage 136
ExternalDocumentID 6903678
Genre orig-research
GroupedDBID 6IE
6IL
CBEJK
RIE
RIL
ID FETCH-LOGICAL-i123t-81a36f7d89b9f67933e9ea1d7421d501a24ea336be3556b2f602ee43c7b43f1b3
IEDL.DBID RIE
ISBN 9781479953127
1479953121
IngestDate Wed Jun 26 19:23:49 EDT 2024
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-i123t-81a36f7d89b9f67933e9ea1d7421d501a24ea336be3556b2f602ee43c7b43f1b3
PageCount 8
ParticipantIDs ieee_primary_6903678
PublicationCentury 2000
PublicationDate 20140701
PublicationDateYYYYMMDD 2014-07-01
PublicationDate_xml – month: 07
  year: 2014
  text: 20140701
  day: 01
PublicationDecade 2010
PublicationTitle 2014 International Conference on High Performance Computing & Simulation (HPCS)
PublicationTitleAbbrev HPCSim
PublicationYear 2014
Publisher IEEE
Publisher_xml – name: IEEE
Score 1.5603955
Snippet Pull-based late-binding overlays are used in some of today's largest computational grids. Job agents are submitted to resources with the duty of retrieving...
SourceID ieee
SourceType Publisher
StartPage 129
SubjectTerms Distributed algorithms
Grid and Cluster Computing
Measurement
Peer-to-Peer Architectures and Networks
Peer-to-peer computing
Reliable Parallel and Distributed Algorithms
Scalable Computing
Title Distributed scheduling and data sharing in late-binding overlays
URI https://ieeexplore.ieee.org/document/6903678
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3LSgMxFA1tceFKpRXfZOHSaSePJpOdUC1dSCn4oLuSTG5woEylD8G_N3c6VgU37mYCGSYZkptz55xzCblm6JnOciQ82TSRTPrExB0yQTMxyNPQD1Vq4OVBj8fZdGomDXKz08IAQEU-gy5eVv_y_SLfYKqsF5GciJtrkzS11j-0WmhqJhhnXxZO9b2uXYZYanqjyeCxQOU5k936Mb_qqVThZHjwvxc5JJ1vXR6d7CLOEWlA2Sa3d-h9i2WrwNOIVWPsQIk5taWnyP-kq1eLyTtalHQeT5aIhbE7RfLm3H6sOuR5eP80GCV1XYSkiHFmnWTMChW0z4wzQcUFJsCAZT6iXOb7cdK5BCuEchAPE8rxoFIOIEWunRSBOXFMWuWihBNCufOxKfS5VBGHMZeZjPvUKiOMdA74KWnj6GdvW-uLWT3ws7-bz8k-TvCWzXpBWuvlBi7JXv6-LlbLq-p7fQJrEZMM
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1dS8MwFA1zCvqksonf5sFHuzUfTZs3YSoT5xg4ZW8jaW6xMDrZh-C_N7erU8EX39pASpOS3Jzbc84l5JKhZzpLkfBkwkAy6QLtd8gAzcQgDbMoK1MDL724309GIz2okau1FgYASvIZtPCy_JfvpukSU2Vtj-SE31w3yGYkJWc_1FpoayYYZ18mTtV9XPkMsVC3u4POU47acyZb1YN-VVQpA8rd7v9eZY80v5V5dLCOOfukBkWDXN-g-y0WrgJHPVr10QNF5tQUjiIDlM5fDabvaF7QiT9bIhrG7hTpmxPzMW-S57vbYacbVJURgtxHmkWQMCNUFrtEW50pv8QEaDDMeZzLXOSnnUswQigL_jihLM9UyAGkSGMrRcasOCD1YlrAIaHcOt-URVwqj8SYTXTCXWiUFlpaC_yINHD047eV-cW4Gvjx380XZLs7fOyNe_f9hxOyg5O94raekvpitoQzspW-L_L57Lz8dp9FC5ZT
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2014+International+Conference+on+High+Performance+Computing+%26+Simulation+%28HPCS%29&rft.atitle=Distributed+scheduling+and+data+sharing+in+late-binding+overlays&rft.au=Delgado+Peris%2C+Antonio&rft.au=Hernandez%2C+Jose+M.&rft.au=Huedo%2C+Eduardo&rft.date=2014-07-01&rft.pub=IEEE&rft.isbn=9781479953127&rft.spage=129&rft.epage=136&rft_id=info:doi/10.1109%2FHPCSim.2014.6903678&rft.externalDocID=6903678
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=9781479953127/lc.gif&client=summon&freeimage=true
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=9781479953127/mc.gif&client=summon&freeimage=true
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=9781479953127/sc.gif&client=summon&freeimage=true