Enhancing Throughput of Hadoop Distributed File System for Interaction-Intensive Tasks

The performance of the Hadoop Distributed File System (HDFS)decreases dramatically when handling interaction-intensive files, i.e., files that have relatively small size but are accessed frequently. The paper analyzes the cause of throughput degradation issue when accessing interaction-intensive fil...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Proceedings - Euromicro Workshop on Parallel and Distributed Processing S. 508 - 511
Hauptverfasser: Xiayu Hua, Hao Wu, Shangping Ren
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: IEEE 01.02.2014
Schlagworte:
Arrays
Cache
HDFS
Hierarchical structure
Multi-layer neural network
Periodic structures
PSO
Resource management
Storage Allocation Algorithm
Throughput
Vectors
ISSN:1066-6192
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The performance of the Hadoop Distributed File System (HDFS)decreases dramatically when handling interaction-intensive files, i.e., files that have relatively small size but are accessed frequently. The paper analyzes the cause of throughput degradation issue when accessing interaction-intensive files and presents an enhanced HDFS architecture along with an associated storage allocation algorithm that overcomes the performance degradation problem. Experiments have shown that with the proposed architecture together with the associated storage allocation algorithm, the HDFS throughput for interaction-intensive files increase 300% in average with only a negligible performance decrease for large data set tasks.
ISSN:1066-6192
DOI:10.1109/PDP.2014.110