Enhancing Throughput of Hadoop Distributed File System for Interaction-Intensive Tasks

The performance of the Hadoop Distributed File System (HDFS)decreases dramatically when handling interaction-intensive files, i.e., files that have relatively small size but are accessed frequently. The paper analyzes the cause of throughput degradation issue when accessing interaction-intensive fil...

Full description

Saved in:

Bibliographic Details
Published in:	Proceedings - Euromicro Workshop on Parallel and Distributed Processing pp. 508 - 511
Main Authors:	Xiayu Hua, Hao Wu, Shangping Ren
Format:	Conference Proceeding
Language:	English
Published:	IEEE 01.02.2014
Subjects:	Arrays Cache HDFS Hierarchical structure Multi-layer neural network Periodic structures PSO Resource management Storage Allocation Algorithm Throughput Vectors
ISSN:	1066-6192
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Description
Summary:	The performance of the Hadoop Distributed File System (HDFS)decreases dramatically when handling interaction-intensive files, i.e., files that have relatively small size but are accessed frequently. The paper analyzes the cause of throughput degradation issue when accessing interaction-intensive files and presents an enhanced HDFS architecture along with an associated storage allocation algorithm that overcomes the performance degradation problem. Experiments have shown that with the proposed architecture together with the associated storage allocation algorithm, the HDFS throughput for interaction-intensive files increase 300% in average with only a negligible performance decrease for large data set tasks.
ISSN:	1066-6192
DOI:	10.1109/PDP.2014.110