In EDS ansehen

Real-Time Distribution Algorithm for Fully Comparison Data Based on Storm.

Gespeichert in:
Bibliographische Detailangaben
Titel: Real-Time Distribution Algorithm for Fully Comparison Data Based on Storm.
Autoren: Dong, Chang-qing, Chen, Chen, Ren, Nver, Cai, Jian-jun
Quelle: Mobile Networks & Applications; Apr2022, Vol. 27 Issue 2, p588-597, 10p
Schlagwörter: DISTRIBUTION (Probability theory), REDUNDANCY in engineering, TRANSACTION costs, STATISTICS, COST allocation, SETTLEMENT costs, SPANNING trees
Abstract: Current data allocation algorithms neglect the problems of unsatisfactory allocation results and long execution time caused by the redundancy of full comparative data and the complexity of data types. To solve these problems, a real-time allocation algorithm of full comparison data based on storm is proposed. Firstly, the phase unwrapping algorithm of minimum spanning tree is used to remove redundant data in full comparison data; then, the distributed data clustering algorithm and storm framework are used to realize the full comparison data clustering after redundancy removal. Several main factors affecting the selection of statistical information are summarized according to the clustering results. Then the communication cost of data loading and transaction processing is determined, and the trade-off between read-only transaction and update transaction cost is achieved. By judging whether the total cost of read-only transaction and update transaction is reduced or not, the replica is eliminated, and a full comparison data allocation algorithm with minimum total cost of read-only transaction and update transaction is proposed to realize real-time allocation of full-comparative data. The example analysis shows that the proposed algorithm can meet the user's needs in terms of execution time, acceleration ratio, storage efficiency and cost. Compared with the reference algorithm, the proposed algorithm has the lowest execution time, the highest acceleration ratio and the closest allocation cost to the ideal overhead. [ABSTRACT FROM AUTHOR]
Copyright of Mobile Networks & Applications is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Datenbank: Complementary Index
Beschreibung
Abstract:Current data allocation algorithms neglect the problems of unsatisfactory allocation results and long execution time caused by the redundancy of full comparative data and the complexity of data types. To solve these problems, a real-time allocation algorithm of full comparison data based on storm is proposed. Firstly, the phase unwrapping algorithm of minimum spanning tree is used to remove redundant data in full comparison data; then, the distributed data clustering algorithm and storm framework are used to realize the full comparison data clustering after redundancy removal. Several main factors affecting the selection of statistical information are summarized according to the clustering results. Then the communication cost of data loading and transaction processing is determined, and the trade-off between read-only transaction and update transaction cost is achieved. By judging whether the total cost of read-only transaction and update transaction is reduced or not, the replica is eliminated, and a full comparison data allocation algorithm with minimum total cost of read-only transaction and update transaction is proposed to realize real-time allocation of full-comparative data. The example analysis shows that the proposed algorithm can meet the user's needs in terms of execution time, acceleration ratio, storage efficiency and cost. Compared with the reference algorithm, the proposed algorithm has the lowest execution time, the highest acceleration ratio and the closest allocation cost to the ideal overhead. [ABSTRACT FROM AUTHOR]
ISSN:1383469X
DOI:10.1007/s11036-021-01824-3