Algorithms for Integrated Routing and Scheduling for Aggregating Data from Distributed Resources on a Lambda Grid
In many e-science applications, there exists an important need to aggregate information from data repositories distributed around the world. In an effort to better link these resources in a unified manner, many lambda-grid networks, which provide end-to-end dedicated optical-circuit-switched connect...
Uloženo v:
| Vydáno v: | IEEE transactions on parallel and distributed systems Ročník 19; číslo 1; s. 24 - 34 |
|---|---|
| Hlavní autoři: | , , , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
New York
IEEE
01.01.2008
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Témata: | |
| ISSN: | 1045-9219, 1558-2183 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | In many e-science applications, there exists an important need to aggregate information from data repositories distributed around the world. In an effort to better link these resources in a unified manner, many lambda-grid networks, which provide end-to-end dedicated optical-circuit-switched connections, have been investigated. In this context, we consider the problem of aggregating files from distributed databases at a (grid) computing node over a lambda grid. The challenge is (1) to identify routes (that is, circuits) in the lambda-grid network, along which files should be transmitted, and (2) to schedule the transfers of these files over their respective circuits. To address this challenge, we propose a hybrid approach that combines offline and online scheduling. We define the Time-Path Scheduling Problem (TPSP) for offline scheduling. We prove that TPSP is NP-complete, develop a Mixed Integer Linear Program (MILP) formulation for TPSP, and then propose a greedy approach to solve TPSP because the MILP does not scale well. We compare the performance of the greedy approach on a few representative lambda-grid network topologies. One key input to the offline schedule is the file transfer time. Due to dynamics at the receiving end host, which is hard to model precisely, the actual file transfer time may vary. We first propose a model for estimating the file transfer time. Then, we propose online reconfiguration algorithms so that as files are transferred, the offline schedule may be modified online, depending on the amount of time that it actually took to transfer the file. This helps in reducing the total time to transfer all the files, which is an important metric. To demonstrate the effectiveness of our approach, we present results on an emulated lambda-grid network testbed. |
|---|---|
| AbstractList | In many e-science applications, there exists an important need to aggregate information from data repositories distributed around the world. In an effort to better link these resources in a unified manner, many lambda-grid networks, which provide end-to-end dedicated optical-circuit-switched connections, have been investigated. In this context, we consider the problem of aggregating files from distributed databases at a (grid) computing node over a lambda grid. The challenge is (1) to identify routes (that is, circuits) in the lambda-grid network, along which files should be transmitted, and (2) to schedule the transfers of these files over their respective circuits. To address this challenge, we propose a hybrid approach that combines offline and online scheduling. We define the Time-Path Scheduling Problem (TPSP) for offline scheduling. We prove that TPSP is NP-complete, develop a Mixed Integer Linear Program (MILP) formulation for TPSP, and then propose a greedy approach to solve TPSP because the MILP does not scale well. We compare the performance of the greedy approach on a few representative lambda-grid network topologies. One key input to the offline schedule is the file transfer time. Due to dynamics at the receiving end host, which is hard to model precisely, the actual file transfer time may vary. We first propose a model for estimating the file transfer time. Then, we propose online reconfiguration algorithms so that as files are transferred, the offline schedule may be modified online, depending on the amount of time that it actually took to transfer the file. This helps in reducing the total time to transfer all the files, which is an important metric. To demonstrate the effectiveness of our approach, we present results on an emulated lambda-grid network testbed. In many e-science applications, there exists an important need to aggregate information from data repositories distributed around the world. In an effort to better link these resources in a unified manner, many lambda- grid networks, which provide [abstract truncated by publisher]. [...] we propose online reconfiguration algorithms so that as files are transferred, the offline schedule may be modified online, depending on the amount of time that it actually took to transfer the file. |
| Author | Ghosal, D. Wu-chun Feng Banerjee, A. Mukherjee, B. |
| Author_xml | – sequence: 1 givenname: A. surname: Banerjee fullname: Banerjee, A. organization: California Davis Univ., Davis – sequence: 2 surname: Wu-chun Feng fullname: Wu-chun Feng – sequence: 3 givenname: D. surname: Ghosal fullname: Ghosal, D. – sequence: 4 givenname: B. surname: Mukherjee fullname: Mukherjee, B. |
| BookMark | eNqFkT1v3SAUhq0qkZqkXbt0QR3aybd8GDDjVW6bRrpSoySdLQzHDpENCeAh_744N-oQqe0ERzzvAc5zWh354KGqPhC8IQSrr7dXu5sNxViWktA31QnhvK0padlR2eOG14oS9bY6TekeY9Jw3JxUj9tpDNHluzmhIUR06TOMUWew6Dos2fkRaW_RjbkDu0xruVLbcYww6ufjnc4aDTHMaOdSjq5fnsOQwhINJBQ80miv595qdBGdfVcdD3pK8P5lPat-ff92e_6j3v-8uDzf7mvDWpLrXjEpNB3kwEhLtLFS4MEIwSVRTDRGW8AgZd8KrTnhfUOtYNYMSjFlASQ7q74c-j7E8LhAyt3skoFp0h7CkjqFmaBYNM1_yVZyzFQZaiE__5NkpRuXvC3gp1fgfRmHL__tFKFUtpSzAm0OkIkhpQhD9xDdrONTR3C3Ku1Wpd2qtFuVlkDzKmBcLhaCz1G76e-xj4eYA4A_d5SXqgZj9huMMq8p |
| CODEN | ITDSEO |
| CitedBy_id | crossref_primary_10_1364_JOCN_9_000658 crossref_primary_10_1109_TNET_2009_2026576 crossref_primary_10_1016_j_future_2013_01_003 crossref_primary_10_1364_JOCN_465019 crossref_primary_10_1364_OE_17_001023 crossref_primary_10_1007_s11227_010_0509_9 crossref_primary_10_1109_JLT_2009_2021485 crossref_primary_10_1007_s11107_008_0155_9 crossref_primary_10_1364_JOCN_2_000241 |
| Cites_doi | 10.1137/0214054 10.1109/CCGrid.2004.1336572 10.1109/CLUSTR.2002.1137760 10.1109/ICC.2004.1312774 10.1287/opre.44.3.510 10.1023/B:GRID.0000037556.83278.70 10.1109/IPDPS.2002.1015510 10.1109/INFOCOM.2006.35 10.1145/251007.251012 10.1109/INFOCOM.2006.20 10.1109/IPDPS.2006.1639329 10.1109/FTCS.1995.466999 10.1109/SC.2004.24 10.1109/MCOM.2005.1541694 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2008 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2008 |
| DBID | 97E RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D F28 FR3 |
| DOI | 10.1109/TPDS.2007.1112 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional ANTE: Abstracts in New Technology & Engineering Engineering Research Database |
| DatabaseTitle | CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional Engineering Research Database ANTE: Abstracts in New Technology & Engineering |
| DatabaseTitleList | Technology Research Database Technology Research Database Technology Research Database Technology Research Database |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering Computer Science |
| EISSN | 1558-2183 |
| EndPage | 34 |
| ExternalDocumentID | 2545085421 10_1109_TPDS_2007_1112 4359400 |
| Genre | orig-research |
| GroupedDBID | --Z -~X .DC 0R~ 29I 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABFSI ABQJQ ABVLG ACGFO ACIWK AENEX AETIX AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 E.L EBS EJD HZ~ H~9 ICLAB IEDLZ IFIPE IFJZH IPLJI JAVBF LAI M43 MS~ O9- OCL P2P PQQKQ RIA RIE RNI RNS RZB TN5 TWZ UHB VH1 AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D F28 FR3 |
| ID | FETCH-LOGICAL-c381t-b9376a2f7f3181acd760fc665719364cade0e77b86aa515b42d63dcf9939dee73 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 19 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000251876300004&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1045-9219 |
| IngestDate | Mon Sep 29 06:35:01 EDT 2025 Thu Oct 02 07:36:44 EDT 2025 Sun Sep 28 08:42:08 EDT 2025 Sun Nov 30 04:39:26 EST 2025 Tue Nov 18 21:45:11 EST 2025 Sat Nov 29 08:08:27 EST 2025 Wed Aug 27 02:52:31 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 1 |
| Keywords | scheduling lambda grid routing circuit switching large scale data transfers |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c381t-b9376a2f7f3181acd760fc665719364cade0e77b86aa515b42d63dcf9939dee73 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 ObjectType-Article-2 ObjectType-Feature-1 content type line 23 |
| PQID | 912278253 |
| PQPubID | 23500 |
| PageCount | 11 |
| ParticipantIDs | proquest_miscellaneous_875039111 ieee_primary_4359400 crossref_citationtrail_10_1109_TPDS_2007_1112 proquest_miscellaneous_34435758 proquest_journals_912278253 proquest_miscellaneous_903620644 crossref_primary_10_1109_TPDS_2007_1112 |
| PublicationCentury | 2000 |
| PublicationDate | 2008-Jan. 2008-01-00 20080101 |
| PublicationDateYYYYMMDD | 2008-01-01 |
| PublicationDate_xml | – month: 01 year: 2008 text: 2008-Jan. |
| PublicationDecade | 2000 |
| PublicationPlace | New York |
| PublicationPlace_xml | – name: New York |
| PublicationTitle | IEEE transactions on parallel and distributed systems |
| PublicationTitleAbbrev | TPDS |
| PublicationYear | 2008 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | Cormen (ref12) 2001 ref15 ref14 ref11 ref10 (ref1) 2007 ref17 ref16 (ref6) 2007 Garey (ref13) 1979 (ref3) 2003 (ref8) 2007 ref24 ref23 ref26 ref25 ref20 (ref5) 2007 ref22 (ref7) 2007 Pinedo (ref19) 2002 ref27 Mukherjee (ref18) 2006 (ref2) 2007 ref9 Rao (ref21) 2004; E87-B (ref4) 2007 |
| References_xml | – volume-title: DoE UltraScience Net Testbed year: 2007 ident: ref2 – volume-title: US Dept. of Energy (DoE) Workshop year: 2003 ident: ref3 article-title: Genomes to Life Requires New Life from Networks – ident: ref11 doi: 10.1137/0214054 – volume-title: ILOG CPLEX version 9.0 year: 2007 ident: ref4 – volume-title: Computers and Intractability: A Guide to the Theory of NP-Completeness year: 1979 ident: ref13 – ident: ref27 doi: 10.1109/CCGrid.2004.1336572 – ident: ref17 doi: 10.1109/CLUSTR.2002.1137760 – volume-title: Nat’l LambdaRail Inc. year: 2007 ident: ref5 – volume-title: Introduction to Algorithms year: 2001 ident: ref12 – volume-title: CANARIE CA*net4 year: 2007 ident: ref1 – volume: E87-B issue: 4 year: 2004 ident: ref21 article-title: On Algorithms for Quickest Paths under Different Routing Modes publication-title: IEICE Trans. Comm. – ident: ref9 doi: 10.1109/ICC.2004.1312774 – volume-title: Optiputer year: 2007 ident: ref6 – ident: ref16 doi: 10.1287/opre.44.3.510 – ident: ref25 doi: 10.1023/B:GRID.0000037556.83278.70 – ident: ref24 doi: 10.1109/IPDPS.2002.1015510 – volume-title: TeraGrid year: 2007 ident: ref8 – volume-title: Optical Communication Networks year: 2006 ident: ref18 – ident: ref22 doi: 10.1109/INFOCOM.2006.35 – ident: ref23 doi: 10.1145/251007.251012 – volume-title: Scheduling: Theory, Algorithms, and Systems year: 2002 ident: ref19 – ident: ref14 doi: 10.1109/INFOCOM.2006.20 – ident: ref10 doi: 10.1109/IPDPS.2006.1639329 – volume-title: QUANTA 1.0, EVL year: 2007 ident: ref7 – ident: ref26 doi: 10.1109/FTCS.1995.466999 – ident: ref15 doi: 10.1109/SC.2004.24 – ident: ref20 doi: 10.1109/MCOM.2005.1541694 |
| SSID | ssj0014504 |
| Score | 1.9850799 |
| Snippet | In many e-science applications, there exists an important need to aggregate information from data repositories distributed around the world. In an effort to... [...] we propose online reconfiguration algorithms so that as files are transferred, the offline schedule may be modified online, depending on the amount of... |
| SourceID | proquest crossref ieee |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 24 |
| SubjectTerms | Aggregates Algorithms circuit switching Circuits Computer networks Distributed computing Distributed databases Grid computing lambda grid large scale data transfers Network topologies Network topology Networks On-line systems Online Optical fiber networks Processor scheduling Routing Routing (telecommunications) Schedules Scheduling Scheduling algorithm Studies |
| Title | Algorithms for Integrated Routing and Scheduling for Aggregating Data from Distributed Resources on a Lambda Grid |
| URI | https://ieeexplore.ieee.org/document/4359400 https://www.proquest.com/docview/912278253 https://www.proquest.com/docview/34435758 https://www.proquest.com/docview/875039111 https://www.proquest.com/docview/903620644 |
| Volume | 19 |
| WOSCitedRecordID | wos000251876300004&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1558-2183 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014504 issn: 1045-9219 databaseCode: RIE dateStart: 19900101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8QwEB5UPOjBt1ifOQherHb7SnNcXF8gIqjgraRJqgva6rbr73cm7RZF9-CtpZMSmE7nm8zjAzjUGGJkCQ_cMLJHN5q7IjPc9cI8UD46kEQpSzbBb2-TpydxNwPHXS-MMcYWn5kTurS5fF2qMR2VnaJrJx7vWZjlnDe9Wl3GIIwsVSBGF5Er0AzbAY09T5w-3A3um2GFaNj-DwdkGVV-_Yatb7lY_t-uVmCpxZCs3yh9FWZMsQbLE34G1prrGix-Gza4Dh_91-dyNKxf3iqGSJVdTwZFaEZlQSjDZKFx9Qt6H2pSt1L9ZwzIaQoH3g9kLRn1o7ABjdslpixa3CYAKlYWTLIb-ZZpyS5HQ70BjxfnD2dXbku44Cp03LWbIVaJpZ_zHC29J5XmsZcrys0gzItDKtj3DOdZEkuJOCgLfR0HWuWIcYQ2hgebMFeUhdkCxoX0NUafgkeIeYzAQBvFfZ3FEfo_LhxwJ1pIVTuNnEgxXlMblXgiJa0RSSanIMV34KiTf2_mcEyVXCcddVKtehzYmSg5bc20SkWPOoH9KHDgoHuK9kVJE1mYclylQYgvwKDKATZFIqFUMPmM6SKCcAKCv3D7783twEJTh0JHO7swV4_GZg_m1Wc9rEb79kP_AvO5-20 |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1Nb9QwEB2VgkQ5UGhBDS3UByQuhGYdJ46PK5bSimVViUXqLXJsp12pTdpNlt_PjJONQGUP3BJlHFmaTOaN5-MBvLcYYhSZjEOR-KMbK0NVOBlGoowNRweSGePJJuRsll1eqost-Dj0wjjnfPGZ-0SXPpdva7Oio7ITdO3E4_0IHidC8FHXrTXkDETiyQIxvkhChYbYj2gcRepkfjH50Y0rRNPmf7kgz6ny4Efsvcvp7v_t6wU871EkG3dqfwlbrtqD3TVDA-sNdg-e_TFucB_uxzdX9XLRXt82DLEqO1-PirCMCoNQhunK4upr9D_Upu6lxlcYktMcDryf6FYz6khhExq4S1xZtLhPATSsrphmU31bWM2-Lhf2Ffw8_TL_fBb2lAuhQdfdhgWilVTzUpZo6yNtrEyj0lB2BoFeKqhkP3JSFlmqNSKhQnCbxtaUiHKUdU7Gr2G7qit3AEwqzS3Gn0omiHqcwlAbxbkt0gQ9oFQBhGst5KafR060GDe5j0silZPWiCZTUpjCA_gwyN91kzg2Su6TjgapXj0BHK6VnPeG2uRqRL3APIkDOB6eooVR2kRXrl41eSzwBRhWBcA2SGSUDCavsVlEEVJA-Cfe_Htzx_D0bP59mk_PZ98OYaerSqGDniPYbpcr9xaemF_tolm-8x_9byWC_rQ |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Algorithms+for+Integrated+Routing+and+Scheduling+for+Aggregating+Data+from+Distributed+Resources+on+a+Lambda+Grid&rft.jtitle=IEEE+transactions+on+parallel+and+distributed+systems&rft.au=Banerjee%2C+A&rft.au=Feng%2C+Wu-chun&rft.au=Ghosal%2C+D&rft.au=Mukherjee%2C+B&rft.date=2008-01-01&rft.issn=1045-9219&rft.volume=19&rft.issue=1&rft_id=info:doi/10.1109%2FTPDS.2007.1112&rft.externalDBID=NO_FULL_TEXT |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1045-9219&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1045-9219&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1045-9219&client=summon |