A Distributed Algorithm for IDC Snapshot Data - Sun, ZhiChao; Meng, WeiYe; Wei, Sen (2023 International Conference on Mobile Internet, Cloud Computing and Information Security (MICCIS), 2023)
With the AI technology used in IDC energy saving, The whole room snapshot data are used in some scenarios, The room snapshot data is composed of the snapshot data of various devices in the room. These devices can only collect data regularly at a certain frequency due to settings, performance and other reasons, and can not achieve time synchronization and when used in AI algorithm the value from different collect time will cause inaccurate. Therefore, in the data processing and modeling phase, it is necessary to predict the device snapshot value at certain time, in practice , AI needs a large amount of training data to get good training results, and China Telecom has a large number of computer rooms that provide a large amount of training data. Generally, interpolation algorithms are used to unify data from multiple data sources, but data processing is only one step in the AI computing process. Traditional stand-alone interpolation algorithms cannot meet the time requirements. This paper proposes a Spark based distributed interpolation algorithm, Experiments show that this algorithm can reduce the running time of the algorithm in equal proportion by increasing resources.