Algorithms for Integrated Routing and Scheduling for Aggregating Data from Distributed Resources on a Lambda Grid

In many e-science applications, there exists an important need to aggregate information from data repositories distributed around the world. In an effort to better link these resources in a unified manner, many lambda-grid networks, which provide end-to-end dedicated optical-circuit-switched connect...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on parallel and distributed systems Vol. 19; no. 1; pp. 24 - 34
Main Authors: Banerjee, A., Wu-chun Feng, Ghosal, D., Mukherjee, B.
Format: Journal Article
Language:English
Published: New York IEEE 01.01.2008
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Aggregates
Algorithms
circuit switching
Circuits
Computer networks
Distributed computing
Distributed databases
Grid computing
lambda grid
large scale data transfers
Network topologies
Network topology
Networks
On-line systems
Online
Optical fiber networks
Processor scheduling
Routing
Routing (telecommunications)
Schedules
Scheduling
Scheduling algorithm
Studies
scheduling
lambda grid
routing
circuit switching
large scale data transfers
ISSN:1045-9219, 1558-2183
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract In many e-science applications, there exists an important need to aggregate information from data repositories distributed around the world. In an effort to better link these resources in a unified manner, many lambda-grid networks, which provide end-to-end dedicated optical-circuit-switched connections, have been investigated. In this context, we consider the problem of aggregating files from distributed databases at a (grid) computing node over a lambda grid. The challenge is (1) to identify routes (that is, circuits) in the lambda-grid network, along which files should be transmitted, and (2) to schedule the transfers of these files over their respective circuits. To address this challenge, we propose a hybrid approach that combines offline and online scheduling. We define the Time-Path Scheduling Problem (TPSP) for offline scheduling. We prove that TPSP is NP-complete, develop a Mixed Integer Linear Program (MILP) formulation for TPSP, and then propose a greedy approach to solve TPSP because the MILP does not scale well. We compare the performance of the greedy approach on a few representative lambda-grid network topologies. One key input to the offline schedule is the file transfer time. Due to dynamics at the receiving end host, which is hard to model precisely, the actual file transfer time may vary. We first propose a model for estimating the file transfer time. Then, we propose online reconfiguration algorithms so that as files are transferred, the offline schedule may be modified online, depending on the amount of time that it actually took to transfer the file. This helps in reducing the total time to transfer all the files, which is an important metric. To demonstrate the effectiveness of our approach, we present results on an emulated lambda-grid network testbed.
AbstractList In many e-science applications, there exists an important need to aggregate information from data repositories distributed around the world. In an effort to better link these resources in a unified manner, many lambda- grid networks, which provide [abstract truncated by publisher].
In many e-science applications, there exists an important need to aggregate information from data repositories distributed around the world. In an effort to better link these resources in a unified manner, many lambda-grid networks, which provide end-to-end dedicated optical-circuit-switched connections, have been investigated. In this context, we consider the problem of aggregating files from distributed databases at a (grid) computing node over a lambda grid. The challenge is (1) to identify routes (that is, circuits) in the lambda-grid network, along which files should be transmitted, and (2) to schedule the transfers of these files over their respective circuits. To address this challenge, we propose a hybrid approach that combines offline and online scheduling. We define the Time-Path Scheduling Problem (TPSP) for offline scheduling. We prove that TPSP is NP-complete, develop a Mixed Integer Linear Program (MILP) formulation for TPSP, and then propose a greedy approach to solve TPSP because the MILP does not scale well. We compare the performance of the greedy approach on a few representative lambda-grid network topologies. One key input to the offline schedule is the file transfer time. Due to dynamics at the receiving end host, which is hard to model precisely, the actual file transfer time may vary. We first propose a model for estimating the file transfer time. Then, we propose online reconfiguration algorithms so that as files are transferred, the offline schedule may be modified online, depending on the amount of time that it actually took to transfer the file. This helps in reducing the total time to transfer all the files, which is an important metric. To demonstrate the effectiveness of our approach, we present results on an emulated lambda-grid network testbed.
[...] we propose online reconfiguration algorithms so that as files are transferred, the offline schedule may be modified online, depending on the amount of time that it actually took to transfer the file.
Author Ghosal, D.
Wu-chun Feng
Banerjee, A.
Mukherjee, B.
Author_xml – sequence: 1
  givenname: A.
  surname: Banerjee
  fullname: Banerjee, A.
  organization: California Davis Univ., Davis
– sequence: 2
  surname: Wu-chun Feng
  fullname: Wu-chun Feng
– sequence: 3
  givenname: D.
  surname: Ghosal
  fullname: Ghosal, D.
– sequence: 4
  givenname: B.
  surname: Mukherjee
  fullname: Mukherjee, B.
BookMark eNqFkT1v3SAUhq0qkZqkXbt0QR3aybd8GDDjVW6bRrpSoySdLQzHDpENCeAh_744N-oQqe0ERzzvAc5zWh354KGqPhC8IQSrr7dXu5sNxViWktA31QnhvK0padlR2eOG14oS9bY6TekeY9Jw3JxUj9tpDNHluzmhIUR06TOMUWew6Dos2fkRaW_RjbkDu0xruVLbcYww6ufjnc4aDTHMaOdSjq5fnsOQwhINJBQ80miv595qdBGdfVcdD3pK8P5lPat-ff92e_6j3v-8uDzf7mvDWpLrXjEpNB3kwEhLtLFS4MEIwSVRTDRGW8AgZd8KrTnhfUOtYNYMSjFlASQ7q74c-j7E8LhAyt3skoFp0h7CkjqFmaBYNM1_yVZyzFQZaiE__5NkpRuXvC3gp1fgfRmHL__tFKFUtpSzAm0OkIkhpQhD9xDdrONTR3C3Ku1Wpd2qtFuVlkDzKmBcLhaCz1G76e-xj4eYA4A_d5SXqgZj9huMMq8p
CODEN ITDSEO
CitedBy_id crossref_primary_10_1364_JOCN_9_000658
crossref_primary_10_1109_TNET_2009_2026576
crossref_primary_10_1016_j_future_2013_01_003
crossref_primary_10_1364_JOCN_465019
crossref_primary_10_1364_OE_17_001023
crossref_primary_10_1007_s11227_010_0509_9
crossref_primary_10_1109_JLT_2009_2021485
crossref_primary_10_1007_s11107_008_0155_9
crossref_primary_10_1364_JOCN_2_000241
Cites_doi 10.1137/0214054
10.1109/CCGrid.2004.1336572
10.1109/CLUSTR.2002.1137760
10.1109/ICC.2004.1312774
10.1287/opre.44.3.510
10.1023/B:GRID.0000037556.83278.70
10.1109/IPDPS.2002.1015510
10.1109/INFOCOM.2006.35
10.1145/251007.251012
10.1109/INFOCOM.2006.20
10.1109/IPDPS.2006.1639329
10.1109/FTCS.1995.466999
10.1109/SC.2004.24
10.1109/MCOM.2005.1541694
ContentType Journal Article
Copyright Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2008
Copyright_xml – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2008
DBID 97E
RIA
RIE
AAYXX
CITATION
7SC
7SP
8FD
JQ2
L7M
L~C
L~D
F28
FR3
DOI 10.1109/TPDS.2007.1112
DatabaseName IEEE All-Society Periodicals Package (ASPP) 2005–Present
IEEE All-Society Periodicals Package (ASPP) 1998–Present
IEEE Electronic Library (IEL)
CrossRef
Computer and Information Systems Abstracts
Electronics & Communications Abstracts
Technology Research Database
ProQuest Computer Science Collection
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts – Academic
Computer and Information Systems Abstracts Professional
ANTE: Abstracts in New Technology & Engineering
Engineering Research Database
DatabaseTitle CrossRef
Technology Research Database
Computer and Information Systems Abstracts – Academic
Electronics & Communications Abstracts
ProQuest Computer Science Collection
Computer and Information Systems Abstracts
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts Professional
Engineering Research Database
ANTE: Abstracts in New Technology & Engineering
DatabaseTitleList Technology Research Database

Technology Research Database
Technology Research Database
Technology Research Database
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Engineering
Computer Science
EISSN 1558-2183
EndPage 34
ExternalDocumentID 2545085421
10_1109_TPDS_2007_1112
4359400
Genre orig-research
GroupedDBID --Z
-~X
.DC
0R~
29I
4.4
5GY
5VS
6IK
97E
AAJGR
AARMG
AASAJ
AAWTH
ABAZT
ABFSI
ABQJQ
ABVLG
ACGFO
ACIWK
AENEX
AETIX
AGQYO
AGSQL
AHBIQ
AI.
AIBXA
AKJIK
AKQYR
ALLEH
ALMA_UNASSIGNED_HOLDINGS
ASUFR
ATWAV
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CS3
DU5
E.L
EBS
EJD
HZ~
H~9
ICLAB
IEDLZ
IFIPE
IFJZH
IPLJI
JAVBF
LAI
M43
MS~
O9-
OCL
P2P
PQQKQ
RIA
RIE
RNI
RNS
RZB
TN5
TWZ
UHB
VH1
AAYXX
CITATION
7SC
7SP
8FD
JQ2
L7M
L~C
L~D
F28
FR3
ID FETCH-LOGICAL-c381t-b9376a2f7f3181acd760fc665719364cade0e77b86aa515b42d63dcf9939dee73
IEDL.DBID RIE
ISICitedReferencesCount 19
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000251876300004&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 1045-9219
IngestDate Mon Sep 29 06:35:01 EDT 2025
Thu Oct 02 07:36:44 EDT 2025
Sun Sep 28 08:42:08 EDT 2025
Sun Nov 30 04:39:26 EST 2025
Tue Nov 18 21:45:11 EST 2025
Sat Nov 29 08:08:27 EST 2025
Wed Aug 27 02:52:31 EDT 2025
IsPeerReviewed true
IsScholarly true
Issue 1
Keywords scheduling
lambda grid
routing
circuit switching
large scale data transfers
Language English
License https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c381t-b9376a2f7f3181acd760fc665719364cade0e77b86aa515b42d63dcf9939dee73
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ObjectType-Article-2
ObjectType-Feature-1
content type line 23
PQID 912278253
PQPubID 23500
PageCount 11
ParticipantIDs proquest_miscellaneous_875039111
ieee_primary_4359400
crossref_citationtrail_10_1109_TPDS_2007_1112
proquest_miscellaneous_34435758
proquest_journals_912278253
proquest_miscellaneous_903620644
crossref_primary_10_1109_TPDS_2007_1112
PublicationCentury 2000
PublicationDate 2008-Jan.
2008-01-00
20080101
PublicationDateYYYYMMDD 2008-01-01
PublicationDate_xml – month: 01
  year: 2008
  text: 2008-Jan.
PublicationDecade 2000
PublicationPlace New York
PublicationPlace_xml – name: New York
PublicationTitle IEEE transactions on parallel and distributed systems
PublicationTitleAbbrev TPDS
PublicationYear 2008
Publisher IEEE
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml – name: IEEE
– name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References Cormen (ref12) 2001
ref15
ref14
ref11
ref10
(ref1) 2007
ref17
ref16
(ref6) 2007
Garey (ref13) 1979
(ref3) 2003
(ref8) 2007
ref24
ref23
ref26
ref25
ref20
(ref5) 2007
ref22
(ref7) 2007
Pinedo (ref19) 2002
ref27
Mukherjee (ref18) 2006
(ref2) 2007
ref9
Rao (ref21) 2004; E87-B
(ref4) 2007
References_xml – volume-title: DoE UltraScience Net Testbed
  year: 2007
  ident: ref2
– volume-title: US Dept. of Energy (DoE) Workshop
  year: 2003
  ident: ref3
  article-title: Genomes to Life Requires New Life from Networks
– ident: ref11
  doi: 10.1137/0214054
– volume-title: ILOG CPLEX version 9.0
  year: 2007
  ident: ref4
– volume-title: Computers and Intractability: A Guide to the Theory of NP-Completeness
  year: 1979
  ident: ref13
– ident: ref27
  doi: 10.1109/CCGrid.2004.1336572
– ident: ref17
  doi: 10.1109/CLUSTR.2002.1137760
– volume-title: Nat’l LambdaRail Inc.
  year: 2007
  ident: ref5
– volume-title: Introduction to Algorithms
  year: 2001
  ident: ref12
– volume-title: CANARIE CA*net4
  year: 2007
  ident: ref1
– volume: E87-B
  issue: 4
  year: 2004
  ident: ref21
  article-title: On Algorithms for Quickest Paths under Different Routing Modes
  publication-title: IEICE Trans. Comm.
– ident: ref9
  doi: 10.1109/ICC.2004.1312774
– volume-title: Optiputer
  year: 2007
  ident: ref6
– ident: ref16
  doi: 10.1287/opre.44.3.510
– ident: ref25
  doi: 10.1023/B:GRID.0000037556.83278.70
– ident: ref24
  doi: 10.1109/IPDPS.2002.1015510
– volume-title: TeraGrid
  year: 2007
  ident: ref8
– volume-title: Optical Communication Networks
  year: 2006
  ident: ref18
– ident: ref22
  doi: 10.1109/INFOCOM.2006.35
– ident: ref23
  doi: 10.1145/251007.251012
– volume-title: Scheduling: Theory, Algorithms, and Systems
  year: 2002
  ident: ref19
– ident: ref14
  doi: 10.1109/INFOCOM.2006.20
– ident: ref10
  doi: 10.1109/IPDPS.2006.1639329
– volume-title: QUANTA 1.0, EVL
  year: 2007
  ident: ref7
– ident: ref26
  doi: 10.1109/FTCS.1995.466999
– ident: ref15
  doi: 10.1109/SC.2004.24
– ident: ref20
  doi: 10.1109/MCOM.2005.1541694
SSID ssj0014504
Score 1.9850799
Snippet In many e-science applications, there exists an important need to aggregate information from data repositories distributed around the world. In an effort to...
[...] we propose online reconfiguration algorithms so that as files are transferred, the offline schedule may be modified online, depending on the amount of...
SourceID proquest
crossref
ieee
SourceType Aggregation Database
Enrichment Source
Index Database
Publisher
StartPage 24
SubjectTerms Aggregates
Algorithms
circuit switching
Circuits
Computer networks
Distributed computing
Distributed databases
Grid computing
lambda grid
large scale data transfers
Network topologies
Network topology
Networks
On-line systems
Online
Optical fiber networks
Processor scheduling
Routing
Routing (telecommunications)
Schedules
Scheduling
Scheduling algorithm
Studies
Title Algorithms for Integrated Routing and Scheduling for Aggregating Data from Distributed Resources on a Lambda Grid
URI https://ieeexplore.ieee.org/document/4359400
https://www.proquest.com/docview/912278253
https://www.proquest.com/docview/34435758
https://www.proquest.com/docview/875039111
https://www.proquest.com/docview/903620644
Volume 19
WOSCitedRecordID wos000251876300004&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVIEE
  databaseName: IEEE Electronic Library (IEL)
  customDbUrl:
  eissn: 1558-2183
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0014504
  issn: 1045-9219
  databaseCode: RIE
  dateStart: 19900101
  isFulltext: true
  titleUrlDefault: https://ieeexplore.ieee.org/
  providerName: IEEE
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1RT9wwDLYA7WF7AAabuAFbHibxsoxemybN44mDbRJCSAPEW5UmKZwELbv29vtnp70KtN3D3lrVqSK5rj_H9meAz1KUOjWZ50ZkJRdpIngWO8O1dNonSmUunOnenKuLi-z2Vl-uwZehF8Z7H4rP_Fe6DLl8V9sFHZUdo2unOd7rsK6U6nq1hoyBSMOoQIwuUq7RDHuCxnGkj68upz87skI07PiFAwoTVf76DQffcrb1f7vahs0eQ7JJp_S3sOarHdhazmdgvbnuwJtnZIO78GvycFfPZ-39Y8MQqbIfS6IIx6gsCGWYqRyuvkfvQ03qQWpyhwE5sXDg_dS0hlE_CpsS3S5NyqLFfQKgYXXFDDs3j4Uz7Nt85t7B9dnp1cl33g9c4BYdd8sLxCrSxKUq0dLHxjolo9JSbgZhnhRUsB95pYpMGoM4qBCxk4mzJWIc7bxXyXvYqOrK7wGzMvKIPD1GQ7EoVGkMBnulSRNpI7xNRsCXWshtz0ZOQzEe8hCVRDonrdGQTEVBSjyCo0H-qePhWCm5SzoapHr1jGB_qeS8N9Mm12PqBI5T3M6n4SnaFyVNTOXrRZMnAl-AQdUI2AqJjFLB5DNWi2jCCQj-xId_b24fXnd1KHS0cwAb7XzhD-GV_d3OmvnH8KH_AQ8T_KU
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1Rb9MwED6NgQQ8MNiGKAPmByReMEsTx44fK8rYRKkmUdDeIsd2tkpbAk3K7-fOSSMQ9IG3RDlHli6X-8539x3AaylKnZrMcyOykos0ETyLneFaOu0TpTIXznS_zdR8nl1e6osdeDv0wnjvQ_GZf0eXIZfvarumo7ITdO00x_sO3E2FiMddt9aQMxBpGBaI8UXKNRpiT9E4jvTJ4mL6paMrRNOO_3BBYabKXz_i4F1O9_5vX4_hUY8i2aRT-xPY8dU-7G0mNLDeYPfh4W90gwfwY3JzVa-W7fVtwxCrsvMNVYRjVBiEMsxUDldfo_-hNvUgNbnCkJx4OPB-alrDqCOFTYlwl2Zl0eI-BdCwumKGzcxt4Qz7uFq6Q_h6-mHx_oz3Ixe4Rdfd8gLRijRxqUq09bGxTsmotJSdQaAnBZXsR16pIpPGIBIqROxk4myJKEc771XyFHaruvLPgFkZecSeHuOhWBSqNAbDvdKkibQR3iYj4Bst5LbnI6exGDd5iEsinZPWaEymojAlHsGbQf57x8SxVfKAdDRI9eoZwdFGyXlvqE2ux9QLHKe4nePhKVoYpU1M5et1kycCX4Bh1QjYFomMksHkNbaLaEIKCP_E839v7hjuny0-z_LZ-fzTETzoqlLooOcF7LartX8J9-zPdtmsXoWP_hdP_f_s
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Algorithms+for+Integrated+Routing+and+Scheduling+for+Aggregating+Data+from+Distributed+Resources+on+a+Lambda+Grid&rft.jtitle=IEEE+transactions+on+parallel+and+distributed+systems&rft.au=Banerjee%2C+A&rft.au=Feng%2C+Wu-chun&rft.au=Ghosal%2C+D&rft.au=Mukherjee%2C+B&rft.date=2008-01-01&rft.pub=The+Institute+of+Electrical+and+Electronics+Engineers%2C+Inc.+%28IEEE%29&rft.issn=1045-9219&rft.eissn=1558-2183&rft.volume=19&rft.issue=1&rft.spage=24&rft_id=info:doi/10.1109%2FTPDS.2007.1112&rft.externalDBID=NO_FULL_TEXT&rft.externalDocID=2545085421
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1045-9219&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1045-9219&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1045-9219&client=summon