Knowledge-Engineered Multi-Cloud Resource Brokering for Application Workflow Optimization

Data-intensive application workflows benefit by leveraging cloud services to decrease execution times and increase data sharing. Cloud service providers (CSPs) have distinct capabilities and policies, and performance/cost of the cloud services are amongst the prime factors for CSP selection. However...

Full description

Saved in:
Bibliographic Details
Published in:IEEE eTransactions on network and service management Vol. 20; no. 3; p. 1
Main Authors: Pandey, Ashish, Calyam, Prasad, Lyu, Zhen, Wang, Songjie, Chemodanov, Dmitrii, Joshi, Trupti
Format: Journal Article
Language:English
Published: New York IEEE 01.09.2023
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Bioinformatics
Bioinformatics workflows
Cloud computing
Cloud interoperability
Cloud resource recommendation
Costs
Custom templates
Infrastructure agility
Integer programming
Knowledge based systems
Knowledge engineering
Linear programming
Machine learning
Optimization
Performance optimization
Quality of service
Resource management
User requirements
Workflow
ISSN:1932-4537, 1932-4537
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Data-intensive application workflows benefit by leveraging cloud services to decrease execution times and increase data sharing. Cloud service providers (CSPs) have distinct capabilities and policies, and performance/cost of the cloud services are amongst the prime factors for CSP selection. However, workflow users who need brokering of cloud resources often lack expert guidance to handle the problem of overwhelming choice in CSP selection, and optimization to compensate for service dynamics. In this paper, we address the optimal resource selection problem using a multi-cloud resource broker viz., OnTimeURB that uses knowledge-engineering of user requirements and service capabilities across multiple CSPs. OnTimeURB is powered by integer linear programming and a Naive Bayes classifier to recommend optimal cloud template solutions by weighting performance, agility, cost, and security (PACS) factors. We evaluate the OnTimeURB recommendations with a catalog of bioinformatics application workflows using four CSP resources featuring more than 300 different instance configurations. Our evaluation results show the efficacy of OnTimeURB in creating consistently cost-effective and agile solutions compared to a state-of-the-art k-nearest neighbors (k-NN) approach. We also show that OnTimeURB has 91% success rate improvement in workflow execution times via cloud template recommendations over approaches that do not use knowledge-engineered multi-CSP resource brokering.
AbstractList Data-intensive application workflows benefit by leveraging cloud services to decrease execution times and increase data sharing. Cloud service providers (CSPs) have distinct capabilities and policies, and performance/cost of the cloud services are amongst the prime factors for CSP selection. However, workflow users who need brokering of cloud resources often lack expert guidance to handle the problem of overwhelming choice in CSP selection, and optimization to compensate for service dynamics. In this paper, we address the optimal resource selection problem using a multi-cloud resource broker viz., OnTimeURB that uses knowledge-engineering of user requirements and service capabilities across multiple CSPs. OnTimeURB is powered by integer linear programming and a Naive Bayes classifier to recommend optimal cloud template solutions by weighting performance, agility, cost, and security (PACS) factors. We evaluate the OnTimeURB recommendations with a catalog of bioinformatics application workflows using four CSP resources featuring more than 300 different instance configurations. Our evaluation results show the efficacy of OnTimeURB in creating consistently cost-effective and agile solutions compared to a state-of-the-art k-nearest neighbors (k-NN) approach. We also show that OnTimeURB has 91% success rate improvement in workflow execution times via cloud template recommendations over approaches that do not use knowledge-engineered multi-CSP resource brokering.
Author Joshi, Trupti
Lyu, Zhen
Pandey, Ashish
Chemodanov, Dmitrii
Wang, Songjie
Calyam, Prasad
Author_xml – sequence: 1
  givenname: Ashish
  surname: Pandey
  fullname: Pandey, Ashish
  organization: University of Missouri, Columbia, USA
– sequence: 2
  givenname: Prasad
  orcidid: 0000-0002-7666-5389
  surname: Calyam
  fullname: Calyam, Prasad
  organization: University of Missouri, Columbia, USA
– sequence: 3
  givenname: Zhen
  surname: Lyu
  fullname: Lyu, Zhen
  organization: University of Missouri, Columbia, USA
– sequence: 4
  givenname: Songjie
  orcidid: 0000-0002-6967-579X
  surname: Wang
  fullname: Wang, Songjie
  organization: University of Missouri, Columbia, USA
– sequence: 5
  givenname: Dmitrii
  orcidid: 0000-0002-8948-2370
  surname: Chemodanov
  fullname: Chemodanov, Dmitrii
  organization: University of Missouri, Columbia, USA
– sequence: 6
  givenname: Trupti
  orcidid: 0000-0001-8944-4924
  surname: Joshi
  fullname: Joshi, Trupti
  organization: University of Missouri, Columbia, USA
BookMark eNp9kE1Lw0AQhhepYFv9AeIl4Dl1P7rZ7LGW-oGtglbE05JuJmXbdDduEor-epO2iHjwNMPM-8y8vD3Usc4CQucEDwjB8mr--DIbUEzpgFEqRCSOUJdIRsMhZ6Lzqz9BvbJcYcxjImkXvT9Yt80hXUI4sUtjATykwazOKxOOc1enwTOUrvYagmvv1uCNXQaZ88GoKHKjk8o4G7w5v85ytw2eispszNdueoqOsyQv4exQ--j1ZjIf34XTp9v78WgaaipZFeqFJDrJQALNOCNpFGeacp2SBWk2mAsQcYIhGmYxbafRQnKZUqKHmCSSEtZHl_u7hXcfNZSVWjV-bfNS0VhwLhiLWpXYq7R3ZekhU9pUO5-VT0yuCFZtjqrNUbU5qkOODUn-kIU3m8R__stc7BkDAD96KQWPGWHfE4-BkQ
CODEN ITNSC4
CitedBy_id crossref_primary_10_1155_int_1009713
crossref_primary_10_1109_TNSM_2025_3559610
crossref_primary_10_1007_s13198_024_02686_0
crossref_primary_10_1109_TNSM_2025_3566771
crossref_primary_10_3390_sym17040503
Cites_doi 10.1002/9781118255674.ch9
10.1109/ECOWS.2006.20
10.1016/j.future.2019.04.049
10.1109/TCC.2017.2659728
10.1186/s12864-019-6287-8
10.1109/RTSS.2005.19
10.1109/GLOCOM.2015.7417230
10.5176/978-981-08-5837-7_166
10.1016/j.future.2014.10.008
10.1109/TCC.2017.2732344
10.1016/j.bjp.2013.12.037
10.1109/BIBM.2018.8621369
10.1109/CLOUD.2011.106
10.1145/3138808.3138810
10.1016/j.compeleceng.2014.11.002
10.1109/ISADS.2015.42
10.1002/spe.2803
10.1007/978-3-031-33669-0_7
10.1016/j.eswa.2014.09.014
10.1109/SAI.2014.6918213
10.1371/journal.pbio.1002342
10.1007/978-3-662-62185-1_2
10.1109/ICNP.2019.8888119
10.1109/E2EMON.2005.1564471
10.1109/TSMC.1985.6313384
10.1109/ICCAC.2014.26
10.1177/1094342019852127
10.1088/1742-6596/898/9/092018
10.1109/BIBM47256.2019.8983386
10.1109/TNSM.2017.2738026
10.1007/s11227-014-1376-6
10.1145/3341302.3342080
10.2139/ssrn.4121705
10.1145/3127479.3131614
ContentType Journal Article
Copyright Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023
Copyright_xml – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023
DBID 97E
RIA
RIE
AAYXX
CITATION
DOI 10.1109/TNSM.2022.3227767
DatabaseName IEEE All-Society Periodicals Package (ASPP) 2005–Present
IEEE All-Society Periodicals Package (ASPP) 1998–Present
IEEE Electronic Library (IEL)
CrossRef
DatabaseTitle CrossRef
DatabaseTitleList

Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Xplore
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Engineering
EISSN 1932-4537
EndPage 1
ExternalDocumentID 10_1109_TNSM_2022_3227767
9975831
Genre orig-research
GrantInformation_xml – fundername: NSF
  grantid: OAC-1827177
GroupedDBID 0R~
5VS
6IK
97E
AAJGR
AARMG
AASAJ
AAWTH
ABAZT
ABJNI
ABQJQ
ABVLG
ACGFO
ACIWK
AENEX
AGQYO
AHBIQ
AKJIK
AKQYR
ALMA_UNASSIGNED_HOLDINGS
ATWAV
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CS3
EBS
HZ~
IES
IFIPE
IPLJI
JAVBF
LAI
M43
O9-
OCL
P2P
RIA
RIE
4.4
AAYXX
AETIX
AGSQL
AIBXA
CITATION
EJD
ID FETCH-LOGICAL-c293t-cb91cafe9e2f531d68fc25cd1b1b91057e78a0e64f82cd1b6b959d21c401a9213
IEDL.DBID RIE
ISICitedReferencesCount 6
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001119505800068&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 1932-4537
IngestDate Mon Jun 30 08:46:57 EDT 2025
Sat Nov 29 03:13:17 EST 2025
Tue Nov 18 21:39:53 EST 2025
Mon Aug 11 03:35:39 EDT 2025
IsPeerReviewed true
IsScholarly true
Issue 3
Language English
License https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
https://doi.org/10.15223/policy-029
https://doi.org/10.15223/policy-037
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c293t-cb91cafe9e2f531d68fc25cd1b1b91057e78a0e64f82cd1b6b959d21c401a9213
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ORCID 0000-0002-8948-2370
0000-0001-8944-4924
0000-0002-7666-5389
0000-0002-6967-579X
PQID 2875573361
PQPubID 85504
PageCount 1
ParticipantIDs crossref_citationtrail_10_1109_TNSM_2022_3227767
ieee_primary_9975831
crossref_primary_10_1109_TNSM_2022_3227767
proquest_journals_2875573361
PublicationCentury 2000
PublicationDate 2023-09-01
PublicationDateYYYYMMDD 2023-09-01
PublicationDate_xml – month: 09
  year: 2023
  text: 2023-09-01
  day: 01
PublicationDecade 2020
PublicationPlace New York
PublicationPlace_xml – name: New York
PublicationTitle IEEE eTransactions on network and service management
PublicationTitleAbbrev T-NSM
PublicationYear 2023
Publisher IEEE
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml – name: IEEE
– name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References ref13
ref12
ref34
ref15
ref37
ref14
ref36
ref31
ref30
ref11
ref10
ref32
ref2
(ref41) 2021
ref1
ref17
ref39
ref16
ref19
(ref45) 2022
ref18
Venkataraman (ref35)
(ref8) 2020
(ref27) 2020
Samak (ref38)
(ref5) 2021
ref24
ref23
ref26
ref25
ref20
ref42
ref22
ref44
ref21
(ref40) 2021
ref28
ref29
ref7
ref9
ref4
ref3
(ref46) 2021
(ref33) 2021
(ref43) 2021
Calyam (ref6)
References_xml – ident: ref1
  doi: 10.1002/9781118255674.ch9
– start-page: 311
  volume-title: Proc. IFIP/IEEE Int. Symp. Integr. Netw. Manag.
  ident: ref6
  article-title: Leveraging OpenFlow for resource placement of virtual desktop cloud applications
– volume-title: Cloud computing comparison engine
  year: 2021
  ident: ref40
– ident: ref30
  doi: 10.1109/ECOWS.2006.20
– ident: ref12
  doi: 10.1016/j.future.2019.04.049
– ident: ref21
  doi: 10.1109/TCC.2017.2659728
– ident: ref9
  doi: 10.1186/s12864-019-6287-8
– ident: ref31
  doi: 10.1109/RTSS.2005.19
– ident: ref13
  doi: 10.1109/GLOCOM.2015.7417230
– ident: ref29
  doi: 10.5176/978-981-08-5837-7_166
– start-page: 46
  volume-title: Proc. 8th Int. Conf. Netw. Serv. Manag. (CNSM) Workshop Syst. Virtualiztion Manag. (SVM)
  ident: ref38
  article-title: Failure analysis of distributed scientific workflows executing in the cloud
– volume-title: Open-source ImageMagick package
  year: 2022
  ident: ref45
– ident: ref16
  doi: 10.1016/j.future.2014.10.008
– ident: ref20
  doi: 10.1109/TCC.2017.2732344
– start-page: 363
  volume-title: Proc. 13th USENIX Symp. Netw. Syst. Des. Implement. (NSDI)
  ident: ref35
  article-title: Ernest: Efficient performance prediction for large-scale advanced analytics
– ident: ref2
  doi: 10.1016/j.bjp.2013.12.037
– ident: ref10
  doi: 10.1109/BIBM.2018.8621369
– ident: ref23
  doi: 10.1109/CLOUD.2011.106
– volume-title: Amazon OpsWorks
  year: 2021
  ident: ref5
– ident: ref7
  doi: 10.1145/3138808.3138810
– ident: ref18
  doi: 10.1016/j.compeleceng.2014.11.002
– ident: ref22
  doi: 10.1109/ISADS.2015.42
– ident: ref24
  doi: 10.1002/spe.2803
– ident: ref4
  doi: 10.1007/978-3-031-33669-0_7
– ident: ref37
  doi: 10.1016/j.eswa.2014.09.014
– ident: ref42
  doi: 10.1109/SAI.2014.6918213
– ident: ref26
  doi: 10.1371/journal.pbio.1002342
– volume-title: CyNeuro science gateway
  year: 2021
  ident: ref33
– ident: ref34
  doi: 10.1007/978-3-662-62185-1_2
– ident: ref44
  doi: 10.1109/ICNP.2019.8888119
– ident: ref32
  doi: 10.1109/E2EMON.2005.1564471
– ident: ref39
  doi: 10.1109/TSMC.1985.6313384
– ident: ref11
  doi: 10.1109/ICCAC.2014.26
– ident: ref36
  doi: 10.1177/1094342019852127
– ident: ref28
  doi: 10.1088/1742-6596/898/9/092018
– volume-title: Knowledge base commons
  year: 2020
  ident: ref8
– ident: ref15
  doi: 10.1109/BIBM47256.2019.8983386
– volume-title: University of Missouri data center
  year: 2021
  ident: ref43
– volume-title: Open-source OnTimeURB package
  year: 2021
  ident: ref46
– ident: ref14
  doi: 10.1109/TNSM.2017.2738026
– ident: ref19
  doi: 10.1007/s11227-014-1376-6
– ident: ref25
  doi: 10.1145/3341302.3342080
– ident: ref3
  doi: 10.2139/ssrn.4121705
– ident: ref17
  doi: 10.1145/3127479.3131614
– volume-title: HTCondor
  year: 2020
  ident: ref27
– volume-title: Public cloud services comparison
  year: 2021
  ident: ref41
SSID ssj0058192
Score 2.3274093
Snippet Data-intensive application workflows benefit by leveraging cloud services to decrease execution times and increase data sharing. Cloud service providers (CSPs)...
SourceID proquest
crossref
ieee
SourceType Aggregation Database
Enrichment Source
Index Database
Publisher
StartPage 1
SubjectTerms Bioinformatics
Bioinformatics workflows
Cloud computing
Cloud interoperability
Cloud resource recommendation
Costs
Custom templates
Infrastructure agility
Integer programming
Knowledge based systems
Knowledge engineering
Linear programming
Machine learning
Optimization
Performance optimization
Quality of service
Resource management
User requirements
Workflow
Title Knowledge-Engineered Multi-Cloud Resource Brokering for Application Workflow Optimization
URI https://ieeexplore.ieee.org/document/9975831
https://www.proquest.com/docview/2875573361
Volume 20
WOSCitedRecordID wos001119505800068&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVIEE
  databaseName: IEEE Xplore
  customDbUrl:
  eissn: 1932-4537
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0058192
  issn: 1932-4537
  databaseCode: RIE
  dateStart: 20040101
  isFulltext: true
  titleUrlDefault: https://ieeexplore.ieee.org/
  providerName: IEEE
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA61eNCDrypWq-TgSUy7SXc3ybEUiyBWwQr1tGTzALF2pQ_9-ybZbFUUwduSzUDIJDPfzGRmADjTGAtFmUGpyBWKJUkQ7yqKJE2NjLXgERO-2QQdDtl4zO9q4GKVC6O19o_PdNt9-li-KuTSuco6nFt065Km1yilZa5WJXUTV9grRC1xxDuj4f2Ntf4IadsT6yrWfNM7vpHKD-nrVcpg-3-L2QFbATrCXsnrXVDT0z2w-aWgYAM8XlcuMlSNawV9ki3qT4qlgpW7Hlr7-9lTQQtbYe8zjg2d_9xMind4a8XJS8jT3AcPg8tR_wqF5glIWg2-QDLnWAqjuSbG3jOVMmMZIRXOsf1jUZqmTEQ6jQ0jbjTNecIVwdIaXIIT3D0A9Wkx1YcAUoeScGS6jPE4oULE2uLMWFKLDYhOoyaIqq3NZKgs7hpcTDJvYUQ8c9zIHDeywI0mOF-RvJZlNf6a3HDbv5oYdr4JWhX_snDx5pk1ABNf4hEf_U51DDZcx_jymVgL1BezpT4B6_Jt8TSfnfoz9QFcI8wa
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEB5KFdSDrypWq-bgSUy7SfeVYymWSttVsEI9LWmSBbF2pQ_9-ybZ3aoogrclm4GQSWa-mcnMAFwoQrgMwgT7fCyxK6iHWVMGWAR-IlzFmRNy22wiiKJwNGJ3Jbha5cIopezjM1U3nzaWL1OxNK6yBmMa3Zqk6TXPdSnJsrUKueuZ0l553JI4rDGM7gfa_qO0rs-sqVnzTfPYVio_5K9VKp2d_y1nF7Zz8IhaGbf3oKSm-7D1paRgBR57hZMMF-NKIptmi9uTdClR4bBH2gJ_tlRIA1fU-oxkI-NBTybpO7rVAuUlz9Q8gIfO9bDdxXn7BCy0Dl9gMWZE8EQxRRN906QfJpoVQpIx0X80TlNByB3lu0lIzag_Zh6TlAhtcnFGSfMQytN0qo4ABQYnESdphiFzvYBzV2mk6YpAowOqfKcKTrG1schri5sWF5PY2hgOiw03YsONOOdGFS5XJK9ZYY2_JlfM9q8m5jtfhVrBvzi_evNYm4CeLfJIjn-nOoeN7nDQj_s3Ue8ENk3_-OzRWA3Ki9lSncK6eFs8zWdn9nx9ACHWz2E
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Knowledge-Engineered+Multi-Cloud+Resource+Brokering+for+Application+Workflow+Optimization&rft.jtitle=IEEE+eTransactions+on+network+and+service+management&rft.au=Pandey%2C+Ashish&rft.au=Calyam%2C+Prasad&rft.au=Lyu%2C+Zhen&rft.au=Wang%2C+Songjie&rft.date=2023-09-01&rft.pub=IEEE&rft.eissn=1932-4537&rft.spage=1&rft.epage=1&rft_id=info:doi/10.1109%2FTNSM.2022.3227767&rft.externalDocID=9975831
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1932-4537&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1932-4537&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1932-4537&client=summon