Enhancing Throughput of Hadoop Distributed File System for Interaction-Intensive Tasks

The performance of the Hadoop Distributed File System (HDFS)decreases dramatically when handling interaction-intensive files, i.e., files that have relatively small size but are accessed frequently. The paper analyzes the cause of throughput degradation issue when accessing interaction-intensive fil...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Proceedings - Euromicro Workshop on Parallel and Distributed Processing s. 508 - 511
Hlavní autoři: Xiayu Hua, Hao Wu, Shangping Ren
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 01.02.2014
Témata:
Arrays
Cache
HDFS
Hierarchical structure
Multi-layer neural network
Periodic structures
PSO
Resource management
Storage Allocation Algorithm
Throughput
Vectors
ISSN:1066-6192
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:The performance of the Hadoop Distributed File System (HDFS)decreases dramatically when handling interaction-intensive files, i.e., files that have relatively small size but are accessed frequently. The paper analyzes the cause of throughput degradation issue when accessing interaction-intensive files and presents an enhanced HDFS architecture along with an associated storage allocation algorithm that overcomes the performance degradation problem. Experiments have shown that with the proposed architecture together with the associated storage allocation algorithm, the HDFS throughput for interaction-intensive files increase 300% in average with only a negligible performance decrease for large data set tasks.
ISSN:1066-6192
DOI:10.1109/PDP.2014.110