Enhancing Throughput of Hadoop Distributed File System for Interaction-Intensive Tasks - Xiayu Hua; Hao Wu; Shangping Ren (Proceedings - Euromicro Workshop on Parallel and Distributed Processing, 2014)
The performance of the Hadoop Distributed File System (HDFS)decreases dramatically when handling interaction-intensive files, i.e., files that have relatively small size but are accessed frequently. The paper analyzes the cause of throughput degradation issue when accessing interaction-intensive files and presents an enhanced HDFS architecture along with an associated storage allocation algorithm that overcomes the performance degradation problem. Experiments have shown that with the proposed architecture together with the associated storage allocation algorithm, the HDFS throughput for interaction-intensive files increase 300% in average with only a negligible performance decrease for large data set tasks.