Enhancing Throughput of Hadoop Distributed File System for Interaction-Intensive Tasks

The performance of the Hadoop Distributed File System (HDFS)decreases dramatically when handling interaction-intensive files, i.e., files that have relatively small size but are accessed frequently. The paper analyzes the cause of throughput degradation issue when accessing interaction-intensive fil...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Proceedings - Euromicro Workshop on Parallel and Distributed Processing s. 508 - 511
Hlavní autori: Xiayu Hua, Hao Wu, Shangping Ren
Médium: Konferenčný príspevok..
Jazyk:English
Vydavateľské údaje: IEEE 01.02.2014
Predmet:
Arrays
Cache
HDFS
Hierarchical structure
Multi-layer neural network
Periodic structures
PSO
Resource management
Storage Allocation Algorithm
Throughput
Vectors
ISSN:1066-6192
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:The performance of the Hadoop Distributed File System (HDFS)decreases dramatically when handling interaction-intensive files, i.e., files that have relatively small size but are accessed frequently. The paper analyzes the cause of throughput degradation issue when accessing interaction-intensive files and presents an enhanced HDFS architecture along with an associated storage allocation algorithm that overcomes the performance degradation problem. Experiments have shown that with the proposed architecture together with the associated storage allocation algorithm, the HDFS throughput for interaction-intensive files increase 300% in average with only a negligible performance decrease for large data set tasks.
ISSN:1066-6192
DOI:10.1109/PDP.2014.110