MF-Storm: a maximum flow-based job scheduler for stream processing engines on computational clusters to increase throughput
A scheduling algorithm tries to schedule multiple computational tasks on a cluster of multiple computing nodes to maximize throughput with optimal utilization of computational and communicational resources. A Stream Processing Engine (SPE) is deployed to run streaming applications (computational tas...
Saved in:
| Published in: | PeerJ. Computer science Vol. 8; p. e1077 |
|---|---|
| Main Authors: | , |
| Format: | Journal Article |
| Language: | English |
| Published: |
San Diego
PeerJ. Ltd
26.09.2022
PeerJ, Inc PeerJ Inc |
| Subjects: | |
| ISSN: | 2376-5992, 2376-5992 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | A scheduling algorithm tries to schedule multiple computational tasks on a cluster of multiple computing nodes to maximize throughput with optimal utilization of computational and communicational resources. A Stream Processing Engine (SPE) is deployed to run streaming applications (computational tasks) on a computational cluster which helps execution and coordination of these applications. It is observed that there is a gap in the optimal mapping of a computational and communicational load of a streaming application on the underlying computational and communication power of the resources (cluster). Frequently communicated tasks are scheduled at different processing nodes with relatively slow communicating links. This increases network latency with a decrease in resource utilization. Hence, reduces the achieved throughput of the cluster significantly. MF-Storm, a max-flow min-cut based job scheduler is presented to achieve a near-optimum schedule to maximize throughput. It schedules a streaming application by considering the processing, communication demands, available computational and communicational resources in a heterogeneous cluster, dynamically with minimized scheduling cost. To keep the scheduling cost minimum, the scheduler is built in a pipeline with two major stages: in the first stage, the application's tasks graph is partitioned using the max-flow min-cut algorithm to minimize inter-partition traffic, and in the second stage, these partitions are assigned to computing nodes according to the computational power of the cluster's nodes. Extensive experiments were done to evaluate the performance of MF-Storm using different topologies with multiple scenarios on a physical cluster implementation. Results showed on average 148% improvement in throughput with 30% less computational resources as compared to different state-of-the-art schedulers. |
|---|---|
| AbstractList | Background A scheduling algorithm tries to schedule multiple computational tasks on a cluster of multiple computing nodes to maximize throughput with optimal utilization of computational and communicational resources. A Stream Processing Engine (SPE) is deployed to run streaming applications (computational tasks) on a computational cluster which helps execution and coordination of these applications. It is observed that there is a gap in the optimal mapping of a computational and communicational load of a streaming application on the underlying computational and communication power of the resources (cluster). Frequently communicated tasks are scheduled at different processing nodes with relatively slow communicating links. This increases network latency with a decrease in resource utilization. Hence, reduces the achieved throughput of the cluster significantly. Methods MF-Storm, a max-flow min-cut based job scheduler is presented to achieve a near-optimum schedule to maximize throughput. It schedules a streaming application by considering the processing, communication demands, available computational and communicational resources in a heterogeneous cluster, dynamically with minimized scheduling cost. To keep the scheduling cost minimum, the scheduler is built in a pipeline with two major stages: in the first stage, the application’s tasks graph is partitioned using the max-flow min-cut algorithm to minimize inter-partition traffic, and in the second stage, these partitions are assigned to computing nodes according to the computational power of the cluster’s nodes. Results Extensive experiments were done to evaluate the performance of MF-Storm using different topologies with multiple scenarios on a physical cluster implementation. Results showed on average 148% improvement in throughput with 30% less computational resources as compared to different state-of-the-art schedulers. A scheduling algorithm tries to schedule multiple computational tasks on a cluster of multiple computing nodes to maximize throughput with optimal utilization of computational and communicational resources. A Stream Processing Engine (SPE) is deployed to run streaming applications (computational tasks) on a computational cluster which helps execution and coordination of these applications. It is observed that there is a gap in the optimal mapping of a computational and communicational load of a streaming application on the underlying computational and communication power of the resources (cluster). Frequently communicated tasks are scheduled at different processing nodes with relatively slow communicating links. This increases network latency with a decrease in resource utilization. Hence, reduces the achieved throughput of the cluster significantly.BackgroundA scheduling algorithm tries to schedule multiple computational tasks on a cluster of multiple computing nodes to maximize throughput with optimal utilization of computational and communicational resources. A Stream Processing Engine (SPE) is deployed to run streaming applications (computational tasks) on a computational cluster which helps execution and coordination of these applications. It is observed that there is a gap in the optimal mapping of a computational and communicational load of a streaming application on the underlying computational and communication power of the resources (cluster). Frequently communicated tasks are scheduled at different processing nodes with relatively slow communicating links. This increases network latency with a decrease in resource utilization. Hence, reduces the achieved throughput of the cluster significantly.MF-Storm, a max-flow min-cut based job scheduler is presented to achieve a near-optimum schedule to maximize throughput. It schedules a streaming application by considering the processing, communication demands, available computational and communicational resources in a heterogeneous cluster, dynamically with minimized scheduling cost. To keep the scheduling cost minimum, the scheduler is built in a pipeline with two major stages: in the first stage, the application's tasks graph is partitioned using the max-flow min-cut algorithm to minimize inter-partition traffic, and in the second stage, these partitions are assigned to computing nodes according to the computational power of the cluster's nodes.MethodsMF-Storm, a max-flow min-cut based job scheduler is presented to achieve a near-optimum schedule to maximize throughput. It schedules a streaming application by considering the processing, communication demands, available computational and communicational resources in a heterogeneous cluster, dynamically with minimized scheduling cost. To keep the scheduling cost minimum, the scheduler is built in a pipeline with two major stages: in the first stage, the application's tasks graph is partitioned using the max-flow min-cut algorithm to minimize inter-partition traffic, and in the second stage, these partitions are assigned to computing nodes according to the computational power of the cluster's nodes.Extensive experiments were done to evaluate the performance of MF-Storm using different topologies with multiple scenarios on a physical cluster implementation. Results showed on average 148% improvement in throughput with 30% less computational resources as compared to different state-of-the-art schedulers.ResultsExtensive experiments were done to evaluate the performance of MF-Storm using different topologies with multiple scenarios on a physical cluster implementation. Results showed on average 148% improvement in throughput with 30% less computational resources as compared to different state-of-the-art schedulers. A scheduling algorithm tries to schedule multiple computational tasks on a cluster of multiple computing nodes to maximize throughput with optimal utilization of computational and communicational resources. A Stream Processing Engine (SPE) is deployed to run streaming applications (computational tasks) on a computational cluster which helps execution and coordination of these applications. It is observed that there is a gap in the optimal mapping of a computational and communicational load of a streaming application on the underlying computational and communication power of the resources (cluster). Frequently communicated tasks are scheduled at different processing nodes with relatively slow communicating links. This increases network latency with a decrease in resource utilization. Hence, reduces the achieved throughput of the cluster significantly. MF-Storm, a max-flow min-cut based job scheduler is presented to achieve a near-optimum schedule to maximize throughput. It schedules a streaming application by considering the processing, communication demands, available computational and communicational resources in a heterogeneous cluster, dynamically with minimized scheduling cost. To keep the scheduling cost minimum, the scheduler is built in a pipeline with two major stages: in the first stage, the application's tasks graph is partitioned using the max-flow min-cut algorithm to minimize inter-partition traffic, and in the second stage, these partitions are assigned to computing nodes according to the computational power of the cluster's nodes. Extensive experiments were done to evaluate the performance of MF-Storm using different topologies with multiple scenarios on a physical cluster implementation. Results showed on average 148% improvement in throughput with 30% less computational resources as compared to different state-of-the-art schedulers. |
| ArticleNumber | e1077 |
| Audience | Academic |
| Author | Muhammad, Asif Abdul Qadir, Muhammad |
| Author_xml | – sequence: 1 givenname: Asif orcidid: 0000-0001-9215-4100 surname: Muhammad fullname: Muhammad, Asif – sequence: 2 givenname: Muhammad surname: Abdul Qadir fullname: Abdul Qadir, Muhammad |
| BookMark | eNptks1vFCEYxiemJtbao3cSL3qYOswHDB5MmsbqJjUmVs_kHXiZZTMDKzC1xn9etluj2xQOfP2e5w3wPC-OnHdYFC9pdcY55W-3iGFTqnhGK86fFMd1w1nZCVEf_Td_VpzGuKmqinY0N3Fc_P58WV4nH-Z3BMgMt3ZeZmIm_7McIKImGz-QqNaolwkDMT6QmALCTLbBK4zRupGgG63DSLwjys_bJUGy3sFE1LTEhCGS5Il1KusikrQOfhnXGXtRPDUwRTy9H0-K75cfvl18Kq--fFxdnF-VqutEKpVue8DKcK0G3RvaaVpDIwyjDdYDgkIteNspGDqW122Pipm2gaZijW503ZwUq72v9rCR22BnCL-kByvvNnwYJYRk1YRy0HXPOWto1_YtBwG9UWig5wYY01hlr_d7r-0yzKgVuhRgOjA9PHF2LUd_I0XHu75l2eD1vUHwPxaMSc42KpwmcOiXKGteM1FTVnUZffUA3fgl5IfdUbTP3yj69h81Qr6AdcbnumpnKs85FaxuRb0re_YIlbvG2aocJWPz_oHgzYEgMwlv0whLjHJ1_fWQbfasCj7GgEYquw9BLmInSSu5y6i8y6hUUe4ymlXlA9XfV3yc_wPwl-8Y |
| CitedBy_id | crossref_primary_10_7717_peerj_cs_2767 crossref_primary_10_1016_j_future_2024_107673 crossref_primary_10_1016_j_future_2025_108036 |
| Cites_doi | 10.1016/j.future.2018.07.011 10.14778/3184470.3184474 10.1145/2904080 10.1080/17445760.2019.1585848 10.1016/j.jnca.2017.03.007 10.1016/j.ins.2015.03.027 10.1016/j.procs.2017.05.249 10.1007/s11227-019-03060-9 10.1007/s10586-020-03117-y 10.1109/ACCESS.2019.2930652 10.1007/s10922-021-09632-6 10.1007/s11227-020-03223-z 10.1002/cpe.4334 10.7717/peerj-cs.932 10.1145/3355399 10.1007/s11227-018-2435-1 10.1007/s11227-020-03289-9 |
| ContentType | Journal Article |
| Copyright | COPYRIGHT 2022 PeerJ. Ltd. 2022 Muhammad and Abdul Qadir. This is an open access article distributed under the terms of the Creative Commons Attribution License: https://creativecommons.org/licenses/by/4.0/ (the “License”), which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. For attribution, the original author(s), title, publication source (PeerJ Computer Science) and either DOI or URL of the article must be cited. Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. 2022 Muhammad and Abdul Qadir. 2022 Muhammad and Abdul Qadir 2022 Muhammad and Abdul Qadir |
| Copyright_xml | – notice: COPYRIGHT 2022 PeerJ. Ltd. – notice: 2022 Muhammad and Abdul Qadir. This is an open access article distributed under the terms of the Creative Commons Attribution License: https://creativecommons.org/licenses/by/4.0/ (the “License”), which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. For attribution, the original author(s), title, publication source (PeerJ Computer Science) and either DOI or URL of the article must be cited. Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. – notice: 2022 Muhammad and Abdul Qadir. – notice: 2022 Muhammad and Abdul Qadir 2022 Muhammad and Abdul Qadir |
| DBID | AAYXX CITATION ISR 3V. 7XB 8AL 8FE 8FG 8FK ABUWG AFKRA ARAPS AZQEC BENPR BGLVJ CCPQU DWQXO GNUQQ HCIFZ JQ2 K7- M0N P5Z P62 PHGZM PHGZT PIMPY PKEHL PQEST PQGLB PQQKQ PQUKI PRINS Q9U 7X8 5PM DOA |
| DOI | 10.7717/peerj-cs.1077 |
| DatabaseName | CrossRef Gale In Context: Science ProQuest Central (Corporate) ProQuest Central (purchase pre-March 2016) Computing Database (Alumni Edition) ProQuest SciTech Collection ProQuest Technology Collection ProQuest Central (Alumni) (purchase pre-March 2016) ProQuest Central (Alumni) ProQuest Central UK/Ireland Advanced Technologies & Computer Science Collection ProQuest Central Essentials - QC ProQuest Central ProQuest Technology Collection ProQuest One Community College ProQuest Central Korea ProQuest Central Student SciTech Collection (ProQuest) ProQuest Computer Science Collection Computer Science Database Computing Database Advanced Technologies & Aerospace Database ProQuest Advanced Technologies & Aerospace Collection ProQuest Central Premium ProQuest One Academic (New) Publicly Available Content Database ProQuest One Academic Middle East (New) ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Applied & Life Sciences ProQuest One Academic (retired) ProQuest One Academic UKI Edition ProQuest Central China ProQuest Central Basic MEDLINE - Academic PubMed Central (Full Participant titles) Directory of Open Access Journals |
| DatabaseTitle | CrossRef Publicly Available Content Database Computer Science Database ProQuest Central Student Technology Collection ProQuest One Academic Middle East (New) ProQuest Advanced Technologies & Aerospace Collection ProQuest Central Essentials ProQuest Computer Science Collection ProQuest Central (Alumni Edition) SciTech Premium Collection ProQuest One Community College ProQuest Central China ProQuest Central ProQuest One Applied & Life Sciences ProQuest Central Korea ProQuest Central (New) Advanced Technologies & Aerospace Collection ProQuest Computing ProQuest Central Basic ProQuest Computing (Alumni Edition) ProQuest One Academic Eastern Edition ProQuest Technology Collection ProQuest SciTech Collection Advanced Technologies & Aerospace Database ProQuest One Academic UKI Edition ProQuest One Academic ProQuest One Academic (New) ProQuest Central (Alumni) MEDLINE - Academic |
| DatabaseTitleList | Publicly Available Content Database MEDLINE - Academic |
| Database_xml | – sequence: 1 dbid: DOA name: Open Access资源_DOAJ url: https://www.doaj.org/ sourceTypes: Open Website – sequence: 2 dbid: PIMPY name: ProQuest Publicly Available Content Database url: http://search.proquest.com/publiccontent sourceTypes: Aggregation Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISSN | 2376-5992 |
| ExternalDocumentID | oai_doaj_org_article_bd287763154847a9a8fcefa87fa66de0 PMC9575846 A719624926 10_7717_peerj_cs_1077 |
| GroupedDBID | 53G 5VS 8FE 8FG AAFWJ AAYXX ABUWG ADBBV AFFHD AFKRA AFPKN ALMA_UNASSIGNED_HOLDINGS ARAPS AZQEC BCNDV BENPR BGLVJ BPHCQ CCPQU CITATION DWQXO FRP GNUQQ GROUPED_DOAJ H13 HCIFZ IAO ICD IEA ISR ITC K6V K7- M~E OK1 P62 PHGZM PHGZT PIMPY PQGLB PQQKQ PROAC RPM 3V. 7XB 8AL 8FK JQ2 M0N PKEHL PQEST PQUKI PRINS Q9U 7X8 PUEGO 5PM |
| ID | FETCH-LOGICAL-c559t-cd48ae0f7dcbd8f15d12a39f613e2beaced9745cab562be48ec6f43a3063d3d23 |
| IEDL.DBID | P5Z |
| ISICitedReferencesCount | 3 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000863117100002&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 2376-5992 |
| IngestDate | Fri Oct 03 12:46:17 EDT 2025 Tue Nov 04 02:07:07 EST 2025 Fri Sep 05 11:06:55 EDT 2025 Mon Jul 14 08:18:44 EDT 2025 Tue Nov 11 10:04:11 EST 2025 Tue Nov 04 17:25:43 EST 2025 Thu Nov 13 15:52:40 EST 2025 Tue Nov 18 22:27:07 EST 2025 Sat Nov 29 03:45:18 EST 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Language | English |
| License | https://creativecommons.org/licenses/by/4.0 This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. For attribution, the original author(s), title, publication source (PeerJ Computer Science) and either DOI or URL of the article must be cited. |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c559t-cd48ae0f7dcbd8f15d12a39f613e2beaced9745cab562be48ec6f43a3063d3d23 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 content type line 23 |
| ORCID | 0000-0001-9215-4100 |
| OpenAccessLink | https://www.proquest.com/docview/2718000984?pq-origsite=%requestingapplication% |
| PQID | 2718000984 |
| PQPubID | 2045934 |
| PageCount | e1077 |
| ParticipantIDs | doaj_primary_oai_doaj_org_article_bd287763154847a9a8fcefa87fa66de0 pubmedcentral_primary_oai_pubmedcentral_nih_gov_9575846 proquest_miscellaneous_2726921605 proquest_journals_2718000984 gale_infotracmisc_A719624926 gale_infotracacademiconefile_A719624926 gale_incontextgauss_ISR_A719624926 crossref_citationtrail_10_7717_peerj_cs_1077 crossref_primary_10_7717_peerj_cs_1077 |
| PublicationCentury | 2000 |
| PublicationDate | 2022-09-26 |
| PublicationDateYYYYMMDD | 2022-09-26 |
| PublicationDate_xml | – month: 09 year: 2022 text: 2022-09-26 day: 26 |
| PublicationDecade | 2020 |
| PublicationPlace | San Diego |
| PublicationPlace_xml | – name: San Diego – name: San Diego, USA |
| PublicationTitle | PeerJ. Computer science |
| PublicationYear | 2022 |
| Publisher | PeerJ. Ltd PeerJ, Inc PeerJ Inc |
| Publisher_xml | – name: PeerJ. Ltd – name: PeerJ, Inc – name: PeerJ Inc |
| References | Twitter (10.7717/peerj-cs.1077/ref-38) 2014 Farooq (10.7717/peerj-cs.1077/ref-10) 2022; 30 Storm A (10.7717/peerj-cs.1077/ref-34) 2014 Aniello (10.7717/peerj-cs.1077/ref-2) 2013 Karanth (10.7717/peerj-cs.1077/ref-19) 2014 Khalid (10.7717/peerj-cs.1077/ref-20) 2018; 74 Zhang (10.7717/peerj-cs.1077/ref-44) 2017 Elahi (10.7717/peerj-cs.1077/ref-4) 2022; 8 Falt (10.7717/peerj-cs.1077/ref-8) 2011 GitHub (10.7717/peerj-cs.1077/ref-15) 2014 Eskandari (10.7717/peerj-cs.1077/ref-7) 2018b; 89 Weng (10.7717/peerj-cs.1077/ref-41) 2017 Qian (10.7717/peerj-cs.1077/ref-32) 2017; 2016 Muhammad (10.7717/peerj-cs.1077/ref-28) 2021; 77 Muhammad (10.7717/peerj-cs.1077/ref-29) 2021; 24 Foundation AS (10.7717/peerj-cs.1077/ref-12) 2018 Tahir (10.7717/peerj-cs.1077/ref-36) 2019; 2019 Fan (10.7717/peerj-cs.1077/ref-9) 2016 Madsen (10.7717/peerj-cs.1077/ref-27) 2015 Liu (10.7717/peerj-cs.1077/ref-24) 2018 Peng (10.7717/peerj-cs.1077/ref-31) 2015 GitHub (10.7717/peerj-cs.1077/ref-16) 2019 Sun (10.7717/peerj-cs.1077/ref-35) 2015; 319 Liu (10.7717/peerj-cs.1077/ref-26) 2017 Fischer (10.7717/peerj-cs.1077/ref-11) 2015; 2015 Cardellini (10.7717/peerj-cs.1077/ref-3) 2018; 30 Nasiri (10.7717/peerj-cs.1077/ref-30) 2020 Tantalaki (10.7717/peerj-cs.1077/ref-37) 2020; 35 Xu (10.7717/peerj-cs.1077/ref-42) 2014 Eskandari (10.7717/peerj-cs.1077/ref-5) 2016 Ghaderi (10.7717/peerj-cs.1077/ref-14) 2016; 1 Ullah (10.7717/peerj-cs.1077/ref-39) 2019; 7 Foundation AS, Apache Software Foundation (10.7717/peerj-cs.1077/ref-13) 2015 Kamburugamuve (10.7717/peerj-cs.1077/ref-18) 2016 Li (10.7717/peerj-cs.1077/ref-21) 2018; 11 Liu (10.7717/peerj-cs.1077/ref-23) 2017 Gulzar Ahmad (10.7717/peerj-cs.1077/ref-17) 2020; 76 Liu (10.7717/peerj-cs.1077/ref-25) 2020; 53 van der Veen (10.7717/peerj-cs.1077/ref-40) 2015 Li (10.7717/peerj-cs.1077/ref-22) 2017; 87 Smirnov (10.7717/peerj-cs.1077/ref-33) 2017; 108 Al-Sinayyid (10.7717/peerj-cs.1077/ref-1) 2020; 76 Eskandari (10.7717/peerj-cs.1077/ref-6) 2018a Xue (10.7717/peerj-cs.1077/ref-43) 2015; 2015 |
| References_xml | – year: 2019 ident: 10.7717/peerj-cs.1077/ref-16 article-title: Exclamation topology – start-page: 1 year: 2016 ident: 10.7717/peerj-cs.1077/ref-5 article-title: P-scheduler: adaptive hierarchical scheduling in Apache Storm – year: 2015 ident: 10.7717/peerj-cs.1077/ref-13 article-title: Apache Flink: stateful computations over data streams – year: 2011 ident: 10.7717/peerj-cs.1077/ref-8 article-title: Task scheduling in data stream processing – start-page: 149 year: 2015 ident: 10.7717/peerj-cs.1077/ref-31 article-title: R-storm: resource-aware scheduling in storm – start-page: 13 year: 2015 ident: 10.7717/peerj-cs.1077/ref-27 article-title: Dynamic resource management in a massively parallel stream processing engine – volume: 89 start-page: 617 issue: 8 year: 2018b ident: 10.7717/peerj-cs.1077/ref-7 article-title: T3-Scheduler: a topology and Traffic aware two-level Scheduler for stream processing systems in a heterogeneous cluster publication-title: Future Generation Computer Systems doi: 10.1016/j.future.2018.07.011 – volume: 11 start-page: 705 issue: 6 year: 2018 ident: 10.7717/peerj-cs.1077/ref-21 article-title: Model-free control for distributed stream data processing using deep reinforcement learning publication-title: Proceedings of the VLDB Endowment doi: 10.14778/3184470.3184474 – volume: 2016 start-page: 623 year: 2017 ident: 10.7717/peerj-cs.1077/ref-32 article-title: S-Storm: a slot-aware scheduling strategy for even scheduler in storm – volume: 1 start-page: 1 issue: 4 year: 2016 ident: 10.7717/peerj-cs.1077/ref-14 article-title: Scheduling storms and streams in the cloud publication-title: ACM Transactions on Modeling and Performance Evaluation of Computing Systems doi: 10.1145/2904080 – volume: 35 start-page: 571 issue: 5 year: 2020 ident: 10.7717/peerj-cs.1077/ref-37 article-title: A review on big data real-time stream processing and its scheduling techniques publication-title: International Journal of Parallel, Emergent and Distributed Systems doi: 10.1080/17445760.2019.1585848 – volume: 87 start-page: 100 issue: 12 year: 2017 ident: 10.7717/peerj-cs.1077/ref-22 article-title: Real-time scheduling based on optimized topology and communication traffic in distributed real-time computation platform of storm publication-title: Journal of Network and Computer Applications doi: 10.1016/j.jnca.2017.03.007 – volume: 319 start-page: 92 issue: 1 year: 2015 ident: 10.7717/peerj-cs.1077/ref-35 article-title: Re-Stream: real-time and energy-efficient resource scheduling in big data stream computing environments publication-title: Information Sciences doi: 10.1016/j.ins.2015.03.027 – volume: 2015 start-page: 154 year: 2015 ident: 10.7717/peerj-cs.1077/ref-43 article-title: When computing meets heterogeneous cluster: workload assignment in graph computation – year: 2014 ident: 10.7717/peerj-cs.1077/ref-19 article-title: Mastering hadoop – start-page: 1363 year: 2017 ident: 10.7717/peerj-cs.1077/ref-41 article-title: AdaStorm: Resource efficient storm with adaptive configuration – start-page: 571 year: 2017 ident: 10.7717/peerj-cs.1077/ref-26 article-title: E-Storm: replication-based state management in distributed stream processing systems – volume: 108 start-page: 2240 year: 2017 ident: 10.7717/peerj-cs.1077/ref-33 article-title: Performance-aware scheduling of streaming applications using genetic algorithm publication-title: Procedia Computer Science doi: 10.1016/j.procs.2017.05.249 – year: 2018 ident: 10.7717/peerj-cs.1077/ref-12 article-title: Apache Spark™ – Unified analytics engine for big data – volume: 76 start-page: 708 issue: 1 year: 2020 ident: 10.7717/peerj-cs.1077/ref-17 article-title: Use case-based evaluation of workflow optimization strategy in real-time computation system publication-title: Journal of Supercomputing doi: 10.1007/s11227-019-03060-9 – start-page: 485 year: 2018 ident: 10.7717/peerj-cs.1077/ref-24 article-title: D-Storm: dynamic resource-efficient scheduling of stream processing applications – year: 2014 ident: 10.7717/peerj-cs.1077/ref-34 article-title: Storm documentation – start-page: 372 year: 2017 ident: 10.7717/peerj-cs.1077/ref-44 article-title: The real-time scheduling strategy based on traffic and load balancing in storm – volume: 2019 start-page: 160 year: 2019 ident: 10.7717/peerj-cs.1077/ref-36 article-title: MCD: mutually connected community detection using clustering coefficient approach in social networks – volume: 24 start-page: 417 issue: 1 year: 2021 ident: 10.7717/peerj-cs.1077/ref-29 article-title: TOP-Storm: a topology-based resource-aware scheduler for stream processing engine publication-title: Cluster Computing doi: 10.1007/s10586-020-03117-y – volume: 7 year: 2019 ident: 10.7717/peerj-cs.1077/ref-39 article-title: Energy-efficient harvested-aware clustering and cooperative routing protocol for WBAN (E-HARP) publication-title: IEEE Access doi: 10.1109/ACCESS.2019.2930652 – start-page: 207 year: 2013 ident: 10.7717/peerj-cs.1077/ref-2 article-title: Adaptive online scheduling in storm – volume: 2015 start-page: 124 year: 2015 ident: 10.7717/peerj-cs.1077/ref-11 article-title: Workload scheduling in distributed stream processors using graph partitioning – year: 2020 ident: 10.7717/peerj-cs.1077/ref-30 article-title: A scheduling algorithm to maximize storm throughput in heterogeneous cluster – volume: 30 start-page: 183 issue: 1 year: 2022 ident: 10.7717/peerj-cs.1077/ref-10 article-title: A novel cooperative micro-caching algorithm based on fuzzy inference through NFV in ultra-dense IoT networks publication-title: Journal of Network and Systems Management doi: 10.1007/s10922-021-09632-6 – volume: 76 start-page: 9609 issue: 12 year: 2020 ident: 10.7717/peerj-cs.1077/ref-1 article-title: Job scheduler for streaming applications in heterogeneous distributed processing systems publication-title: Journal of Supercomputing doi: 10.1007/s11227-020-03223-z – year: 2017 ident: 10.7717/peerj-cs.1077/ref-23 article-title: Energy usage proling and topology-based scheduling for clusters – year: 2014 ident: 10.7717/peerj-cs.1077/ref-38 article-title: Apache Heron – start-page: 535 year: 2014 ident: 10.7717/peerj-cs.1077/ref-42 article-title: T-storm: traffic-aware online scheduling in storm – start-page: 154 year: 2015 ident: 10.7717/peerj-cs.1077/ref-40 article-title: Dynamically scaling apache storm for the analysis of streaming data – year: 2014 ident: 10.7717/peerj-cs.1077/ref-15 article-title: Storm isolation scheduler – start-page: 234 year: 2018a ident: 10.7717/peerj-cs.1077/ref-6 article-title: Poster: iterative scheduling for distributed stream processing systems – volume: 30 start-page: e4334 issue: 9 year: 2018 ident: 10.7717/peerj-cs.1077/ref-3 article-title: Optimal operator deployment and replication for elastic distributed data stream processing publication-title: Concurrency Computation doi: 10.1002/cpe.4334 – volume-title: Survey of distributed stream processing year: 2016 ident: 10.7717/peerj-cs.1077/ref-18 – volume: 8 start-page: e932 issue: 9 year: 2022 ident: 10.7717/peerj-cs.1077/ref-4 article-title: An evolutionary algorithm for multi-objective optimization of freshwater consumption in textile dyeing industry publication-title: PeerJ Computer Science doi: 10.7717/peerj-cs.932 – volume: 53 start-page: 1 issue: 3 year: 2020 ident: 10.7717/peerj-cs.1077/ref-25 article-title: Resource management and scheduling in distributed stream processing systems: a taxonomy, review, and future directions publication-title: ACM Computing Surveys doi: 10.1145/3355399 – start-page: 309 year: 2016 ident: 10.7717/peerj-cs.1077/ref-9 article-title: Adaptive task scheduling in storm – volume: 74 start-page: 5399 issue: 10 year: 2018 ident: 10.7717/peerj-cs.1077/ref-20 article-title: E-OSched: a load balancing scheduler for heterogeneous multicores publication-title: Journal of Supercomputing doi: 10.1007/s11227-018-2435-1 – volume: 77 start-page: 1059 issue: 2 year: 2021 ident: 10.7717/peerj-cs.1077/ref-28 article-title: A3-Storm: topology-, traffic-, and resource-aware storm scheduler for heterogeneous clusters publication-title: Journal of Supercomputing doi: 10.1007/s11227-020-03289-9 |
| SSID | ssj0001511119 |
| Score | 2.2241437 |
| Snippet | A scheduling algorithm tries to schedule multiple computational tasks on a cluster of multiple computing nodes to maximize throughput with optimal utilization... Background A scheduling algorithm tries to schedule multiple computational tasks on a cluster of multiple computing nodes to maximize throughput with optimal... |
| SourceID | doaj pubmedcentral proquest gale crossref |
| SourceType | Open Website Open Access Repository Aggregation Database Enrichment Source Index Database |
| StartPage | e1077 |
| SubjectTerms | Algorithms Algorithms and Analysis of Algorithms APACHE storm Artificial Intelligence Bandwidths Clusters Communication Computation Distributed and Parallel Computing Experiments Heterogeneous cluster Job scheduler Literature reviews Mutual funds Network latency Nodes Optimization Resource utilization Resource-aware Schedules Scheduling Social networks Software Stream processing engines Task scheduling Topology Workloads |
| SummonAdditionalLinks | – databaseName: Directory of Open Access Journals dbid: DOA link: http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1Lb9NAEF6hqgcuvBGGghaE4IJVv3fdW0FEcKBCFKTeVvukqWI7imNaqX--M-tNFIMQF47JjhN7Z3bn-5Kdbwh5zcqCwbKpYxjWQFByHUOeL-NMJSbJAeCqUvlmE-zkhJ-d1V93Wn3hmbBRHnicuENlANPDIkBoDR9bS-60dZIzJ6vKWM_WE1bvkKmxPhi3gnoU1WRAWQ6X1q4uYt0DXWVskoS8Vv-fO_LvpyR30s7sHrkT8CI9Hu_zPrll2wfk7qYXAw1L8yG5_jKLT4E_N0dU0kZezZuhoW7RXcaYpwy96BQFIguJZQGXAVKlWCUiG7ocKwUgg1HrtQl72rVU-y8IPxRSvRhQT6Gn647OW8SZvaWhww-YPSI_Zh-_f_gUh8YKsQYCsY61Kbi0iWNGK8NdWpo0k3ntILXbTMFWbA3QjFJLBehI2YJbXbkil0AvcpObLH9M9tqutU8I5ZlNuXXcpJUsUmVUIZNC6zw1rmLwIiLvNjMtdFAdx-YXCwHsAx0jvGOE7gU6JiJvtubLUW7jb4bv0W1bI1TJ9m9A7IgQO-JfsRORV-h0gToYLR60-SmHvhefT7-JYwZbk1dTjMjbYOQ6uHMtQ90CPD9KZ00sDyaWsFD1dHgTWyJsFL3IABsgzuVFRF5uh_FKPPzW2m5Am6yqsxSIZ0TYJCYnjz8daefnXiy8BjwOGPPp_5ivZ-R2htUf-KdcdUD21qvBPif7-td63q9e-BV4AyVQPj0 priority: 102 providerName: Directory of Open Access Journals |
| Title | MF-Storm: a maximum flow-based job scheduler for stream processing engines on computational clusters to increase throughput |
| URI | https://www.proquest.com/docview/2718000984 https://www.proquest.com/docview/2726921605 https://pubmed.ncbi.nlm.nih.gov/PMC9575846 https://doaj.org/article/bd287763154847a9a8fcefa87fa66de0 |
| Volume | 8 |
| WOSCitedRecordID | wos000863117100002&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVAON databaseName: Open Access资源_DOAJ customDbUrl: eissn: 2376-5992 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001511119 issn: 2376-5992 databaseCode: DOA dateStart: 20150101 isFulltext: true titleUrlDefault: https://www.doaj.org/ providerName: Directory of Open Access Journals – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources (ISSN International Center) customDbUrl: eissn: 2376-5992 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001511119 issn: 2376-5992 databaseCode: M~E dateStart: 20150101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre – providerCode: PRVPQU databaseName: Advanced Technologies & Aerospace Database customDbUrl: eissn: 2376-5992 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001511119 issn: 2376-5992 databaseCode: P5Z dateStart: 20150527 isFulltext: true titleUrlDefault: https://search.proquest.com/hightechjournals providerName: ProQuest – providerCode: PRVPQU databaseName: Computer Science Database customDbUrl: eissn: 2376-5992 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001511119 issn: 2376-5992 databaseCode: K7- dateStart: 20150527 isFulltext: true titleUrlDefault: http://search.proquest.com/compscijour providerName: ProQuest – providerCode: PRVPQU databaseName: ProQuest Central customDbUrl: eissn: 2376-5992 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001511119 issn: 2376-5992 databaseCode: BENPR dateStart: 20150527 isFulltext: true titleUrlDefault: https://www.proquest.com/central providerName: ProQuest – providerCode: PRVPQU databaseName: ProQuest Publicly Available Content Database customDbUrl: eissn: 2376-5992 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001511119 issn: 2376-5992 databaseCode: PIMPY dateStart: 20150527 isFulltext: true titleUrlDefault: http://search.proquest.com/publiccontent providerName: ProQuest |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1bb9MwFLbYxgMvjKsWGJVBCF6w1lyd8II21IoJtao2kAovluPL6NQmpWkBCYnfzjmu2xEQvPASKfGJkuhcP8f-DiFPeZpwcJuCwbACgBIrBnk-ZVHZ1d0YCtwyLV2zCT4c5uNxMfITbo1fVrmJiS5Q61rhHPlRBEEUC4I8eTX_zLBrFP5d9S00dsgesiRg64ZR-vFqjiXFgFCsqTU5AJejuTGLS6YaAK2ct1KRY-z_My7_vlbyl-TT3__f175Fbvqykx6v7eQ2uWaqO2R_09KBeg-_S74P-uwcYPjsJZV0Jr9NZqsZtdP6K8N0p-llXVLAw5CfpnAbFLwUN5vIGZ2vNxxAIqTGURw2tK6ocg_w841UTVdIy9DQZU0nFZarjaG-URCI3SPv-713r98w35-BKcAhS6Z0kkvTtVyrUuc2THUYybiwUCGYqISIbjSglVTJEoqs0iS5UZlNYgkoJdaxjuL7ZLeqK3NAaB6ZMDc212Emk7DUZSK7iVJxqG3G4SQgLzaqEsqTl2MPjakAEIOaFU6zQjUCNRuQZ1vx-Zq142-CJ6j3rRCSbbsL9eJCeN8VpQZYCXEY0R1YdiFzq4yVObcyy7TpBuQJWo1AOo0K1-tcyFXTiNPzM3HMIcI5UsaAPPdCtoY3V9Jvf4DvRwauluRhSxL8XbWHN5YmfLxpxJWZBeTxdhjvxDV0lalXKBNlRRQCfg0Ibxl16_PbI9Xkk-McL6Csh1L1wb8f_pDciHB7CP61yw7J7nKxMo_IdfVlOWkWHbLDx3mH7J30hqOzjpv9gONbzuA4-NHrOOeF8dHpYPThJxUOUuA |
| linkProvider | ProQuest |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1Zb9NAEF5VBQleKKcwFFgQxwur-t41EkLliBqljRAtUt-2e7mkSuwQJxTEf-I3MuPYKQbBWx94dHYsH5n55pv17jeEPOZJzCFsMgbDBgqUyDDI8wkLtW_9CAiuTnTdbIIPh-LwMHu_Rn60e2FwWWWLiTVQ29LgHPlWCCCKhEDEr6afGXaNwq-rbQuNpVsM3LdTKNmql_238P8-CcPeu4M3O6zpKsAMsOc5MzYWyvk5t0ZbkQeJDUIVZTnkNRdqwCFngWMnRmmgBtrFwpk0jyMF3DqykUWhA4D8C3EkOMbVgLOzOZ0EAShbSnlyKJS2ps7NTpipoEjmvJP66g4Bf-aB39dm_pLsehv_22u6Sq40tJpuL-PgGllzxXWy0basoA2C3SDf93psfw48_QVVdKK-jiaLCc3H5SnDdG7pSakp1PuQf8dwGhB6iptp1IROlxsqINFTV0s4VrQsqKkv0MynUjNeoOxEReclHRVIxytHm0ZIYHaTfDyXV3CLrBdl4W4TKkIXCJcLG6QqDrTVsfJjY6LA5imHA488b11DmkacHXuEjCUUaehJsvYkaSqJnuSRpyvz6VKV5G-Gr9HPVkYoJl7_UM6OZYNNUlsomyHPYPUKkZspkRuXK8FzlabW-R55hF4qUS6kwPVIx2pRVbK__0Fuc0DwWnTSI88ao7yEOzeq2d4Bz48KYx3LzY4l4JnpDreeLRs8reSZW3vk4WoYz8Q1goUrF2gTplkYQH3uEd4Jos7jd0eK0adaUz2DsgWo-J1_X_wBubRzsLcrd_vDwV1yOcStMPiFMt0k6_PZwt0jF82X-aia3a-BgZKj8w6xnxwXq-M |
| linkToPdf | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1Lb9NAEF5VBSEulKcwFFgQjwtW4ufaSAgVSkRUiCIKUsVl2WdJldghjimIf8avY8ZZpxgEtx44OjuWH5n55pv17jeE3GdJzCBsch-GFRQokfIhzyd-KPu6HwHBlYlsmk2w0Sg7OMjHG-RHuxcGl1W2mNgAtS4VzpH3QgBRJARZ3LNuWcR4d_Bs_tnHDlL4pbVtp7FykT3z7RjKt-rpcBf-6wdhOHj57sUr33UY8BUw6aWvdJwJ07dMK6kzGyQ6CEWUW8hxJpSASUYD306UkEATpIkzo1IbRwJ4dqQjjaIHAP9nGNSYuJxwnHw4md9JEIzylawng6KpNzdmceSrCgpmxjppsOkW8GdO-H2d5i-Jb7D1P7-yi-SCo9t0ZxUfl8iGKS6TrbaVBXXIdoV8fzPw95fA359QQWfi62RWz6idlsc-pnlNj0pJK_BvXU_hNCD6FDfZiBmdrzZaAAGgppF2rGhZUNVcwM2zUjWtUY6iosuSTgqk6ZWhrkESmF0l70_lFVwjm0VZmOuEZqEJMmMzHaQiDqSWsejHSkWBtimDA488bt2EKyfajr1DphyKN_Qq3ngVVxVHr_LIw7X5fKVW8jfD5-hzayMUGW9-KBeH3GEWlxrKacg_WNVCROcis8pYkTEr0lSbvkfuocdylBEp0LsORV1VfLj_lu8wQPZGjNIjj5yRLeHOlXDbPuD5UXmsY7ndsQScU93h1su5w9mKn7i4R-6uh_FMXDtYmLJGmzDNwwDqdo-wTkB1Hr87Ukw-NVrrOZQzQNFv_Pvid8g5iCz-ejjau0nOh7hDBj9cpttkc7mozS1yVn1ZTqrF7QYjKPl42hH2ExlztQc |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=MF-Storm%3A+a+maximum+flow-based+job+scheduler+for+stream+processing+engines+on+computational+clusters+to+increase+throughput&rft.jtitle=PeerJ.+Computer+science&rft.au=Muhammad%2C+Asif&rft.au=Abdul+Qadir%2C+Muhammad&rft.date=2022-09-26&rft.issn=2376-5992&rft.eissn=2376-5992&rft.volume=8&rft.spage=e1077&rft_id=info:doi/10.7717%2Fpeerj-cs.1077&rft.externalDBID=NO_FULL_TEXT |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2376-5992&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2376-5992&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2376-5992&client=summon |