An Efficient Cross-Platform Workflow Optimization Method

To manage complex data analysis tasks, cross-platform data processing systems combining multiple platforms are being developed.The platform selection of operators in the cross-platform workflow of the system is critical to the system performance, because the implementation of operators on different...

Full description

Saved in:
Bibliographic Details
Published in:Ji suan ji gong cheng Vol. 48; no. 7; pp. 13 - 21,28
Main Author: DU Qinghua, ZHANG Kai
Format: Journal Article
Language:Chinese
English
Published: Editorial Office of Computer Engineering 01.07.2022
Subjects:
cross-platform workflow|ggfn model|graph attention mechanism|gated recurrent unit(gru)|enumeration algorithm
ISSN:1000-3428
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract To manage complex data analysis tasks, cross-platform data processing systems combining multiple platforms are being developed.The platform selection of operators in the cross-platform workflow of the system is critical to the system performance, because the implementation of operators on different platforms will result in significantly different performances.Currently, cost-based optimization methods are primarily applied in cross-platform workflow optimization to achieve platform selection;however, the existing cost models cannot mine the potential information of cross-platform workflows, thus resulting in inaccurate cost estimation.Hence, a more efficient cross-platform workflow optimization method is proposed herein.This method uses the GAT-BiGRU-FC Network(GGFN) model as the cost model, which uses both operator and workflow features as model inputs.The model uses a graph attention mechanism to capture the structure information of the Directed Acyclic Graph(DAG)-type cross-platform workflow and the information of the neighbor nodes of the operator.The gated recurrent unit is used to memorize the operation timing information of operators to achieve accurate cost estimations.Subsequently, the enumeration algorithm of the operator implementation platform is designed and implemented based on the characteristics of the cross-platform workflow.The algorithm utilizes the GGFN-based cost model and delay-greedy pruning method to perform enumeration and selects the appropriate implementation platform for each operator.Experiments show that this method can improve the execution performance of cross-platform workflows by 3x and reduce the runtime by more than 60%.
AbstractList To manage complex data analysis tasks, cross-platform data processing systems combining multiple platforms are being developed.The platform selection of operators in the cross-platform workflow of the system is critical to the system performance, because the implementation of operators on different platforms will result in significantly different performances.Currently, cost-based optimization methods are primarily applied in cross-platform workflow optimization to achieve platform selection;however, the existing cost models cannot mine the potential information of cross-platform workflows, thus resulting in inaccurate cost estimation.Hence, a more efficient cross-platform workflow optimization method is proposed herein.This method uses the GAT-BiGRU-FC Network(GGFN) model as the cost model, which uses both operator and workflow features as model inputs.The model uses a graph attention mechanism to capture the structure information of the Directed Acyclic Graph(DAG)-type cross-platform workflow and the information of the neighbor nodes of the operator.The gated recurrent unit is used to memorize the operation timing information of operators to achieve accurate cost estimations.Subsequently, the enumeration algorithm of the operator implementation platform is designed and implemented based on the characteristics of the cross-platform workflow.The algorithm utilizes the GGFN-based cost model and delay-greedy pruning method to perform enumeration and selects the appropriate implementation platform for each operator.Experiments show that this method can improve the execution performance of cross-platform workflows by 3x and reduce the runtime by more than 60%.
Author DU Qinghua, ZHANG Kai
Author_xml – sequence: 1
  fullname: DU Qinghua, ZHANG Kai
  organization: 1. School of Software, Fudan University, Shanghai 200438, China;2. School of Computer Science, Fudan University, Shanghai 200438, China
BookMark eNo9zNFKwzAUgOFcTHCbvkN9gNaT5jRNL8eYOpjMC8XLkiYnmtk1IwmIPr2gw6sfvot_wWZTmIixGw4V72Srbg-VT2mqOACUAmtVAUjkUszY_N8u2SKlAwDWNcCcqdVUbJzzxtOUi3UMKZVPo84uxGPxGuKHG8NnsT9lf_TfOvswFY-U34O9YhdOj4muz12yl7vN8_qh3O3vt-vVrrRcyFw2IC2KThmrmwY7KRoHph4syA4Hx01TA0psneAOhTHWtk5JBy2iEyQbEku2_fvaoA_9Kfqjjl990L7_hRDfeh2zNyP1VJtBAFlCAmw73pE2LToauGqlIiV-AO5-V-0
ContentType Journal Article
DBID DOA
DOI 10.19678/j.issn.1000-3428.0064163
DatabaseName DOAJ Directory of Open Access Journals
DatabaseTitleList
Database_xml – sequence: 1
  dbid: DOA
  name: DOAJ Directory of Open Access Journals
  url: https://www.doaj.org/
  sourceTypes: Open Website
DeliveryMethod fulltext_linktorsrc
Discipline Engineering
EndPage 21,28
ExternalDocumentID oai_doaj_org_article_e2cb30ede4e047919eac74feb18768e8
GroupedDBID -0Y
5XA
5XJ
92H
92I
ABJNI
ACGFS
ALMA_UNASSIGNED_HOLDINGS
CCEZO
CUBFJ
CW9
GROUPED_DOAJ
TCJ
TGT
U1G
U5S
ID FETCH-LOGICAL-d136t-506d4398cda5549635f0c2bd0694bf1c5204647f31f43ccdd7f86f0744f3e65e3
IEDL.DBID DOA
ISSN 1000-3428
IngestDate Mon Nov 03 22:00:19 EST 2025
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Issue 7
Language Chinese
English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-d136t-506d4398cda5549635f0c2bd0694bf1c5204647f31f43ccdd7f86f0744f3e65e3
OpenAccessLink https://doaj.org/article/e2cb30ede4e047919eac74feb18768e8
ParticipantIDs doaj_primary_oai_doaj_org_article_e2cb30ede4e047919eac74feb18768e8
PublicationCentury 2000
PublicationDate 2022-07-01
PublicationDateYYYYMMDD 2022-07-01
PublicationDate_xml – month: 07
  year: 2022
  text: 2022-07-01
  day: 01
PublicationDecade 2020
PublicationTitle Ji suan ji gong cheng
PublicationYear 2022
Publisher Editorial Office of Computer Engineering
Publisher_xml – name: Editorial Office of Computer Engineering
SSID ssj0042200
Score 2.1861403
Snippet To manage complex data analysis tasks, cross-platform data processing systems combining multiple platforms are being developed.The platform selection of...
SourceID doaj
SourceType Open Website
StartPage 13
SubjectTerms cross-platform workflow|ggfn model|graph attention mechanism|gated recurrent unit(gru)|enumeration algorithm
Title An Efficient Cross-Platform Workflow Optimization Method
URI https://doaj.org/article/e2cb30ede4e047919eac74feb18768e8
Volume 48
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVAON
  databaseName: DOAJ Directory of Open Access Journals
  issn: 1000-3428
  databaseCode: DOA
  dateStart: 20160101
  customDbUrl:
  isFulltext: true
  dateEnd: 99991231
  titleUrlDefault: https://www.doaj.org/
  omitProxy: false
  ssIdentifier: ssj0042200
  providerName: Directory of Open Access Journals
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1NS8QwEA2yiOhB_MRvIniNmzRp0hxX2cWDrntQ2FtpvkBZu7JW_ftO0irryYvXFkqaYd68tPPeIHRhnKd5wQKJXntEGOOINkES5lh0MFPO8yQUvlXjcTGd6snSqK_YE9baA7cb1_eZNZx654WPbuhMA1IoEQBiII8Ln2S-VOnvw1SLwSLLaOtDQAFlgGGvofOkzANo7j-n9Lr8uRdbuyIt-eXan8rLaAttdrwQD9r1bKMVX--gjSW3wF1UDGo8TI4PUCjwdSxvZDKrmkg7cfzoHWbzT3wPGPDSiSvxXZoPvYceR8OH6xvSDT4gjnHZkJxKB0ShsK6Cag8pkgdqM-OiSNUEZvMs_pBUgbMguLXOqVDIAGRABO5l7vk-6tXz2h8gHBjTusqpkZURGdVGm0rCoQi20XOhxCG6ii9dvrbeFmV0m04XIAZlF4Pyrxgc_cdDjtF6FqUFqRX2BPWaxbs_Rav2o3l6W5yl8H4BJ6Corw
linkProvider Directory of Open Access Journals
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=An+Efficient+Cross-Platform+Workflow+Optimization+Method&rft.jtitle=Ji+suan+ji+gong+cheng&rft.au=DU+Qinghua%2C+ZHANG+Kai&rft.date=2022-07-01&rft.pub=Editorial+Office+of+Computer+Engineering&rft.issn=1000-3428&rft.volume=48&rft.issue=7&rft.spage=13&rft.epage=21%2C28&rft_id=info:doi/10.19678%2Fj.issn.1000-3428.0064163&rft.externalDBID=DOA&rft.externalDocID=oai_doaj_org_article_e2cb30ede4e047919eac74feb18768e8
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1000-3428&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1000-3428&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1000-3428&client=summon