Evaluation of distributed recovery in large-scale storage systems

Storage clusters consisting of thousands of disk drives are now being used both for their large capacity and high throughput. However, their reliability is far worse than that of smaller storage systems due to the increased number of storage nodes. RAID technology is no longer sufficient to guarante...

Full description

Saved in:
Bibliographic Details
Published in:Proceedings of the IEEE International Symposium on High Performance Distributed Computing pp. 172 - 181
Main Authors: Qin Xin, Miller, E.L., Schwarz, S.J.T.J.E.
Format: Conference Proceeding
Language:English
Published: IEEE 2004
Subjects:
Application software
Bandwidth
Buildings
Computational modeling
Costs
Disk drives
Internet
Large-scale systems
Reliability engineering
Throughput
ISBN:0769521754, 9780769521756, 0780321754
ISSN:1082-8907
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Storage clusters consisting of thousands of disk drives are now being used both for their large capacity and high throughput. However, their reliability is far worse than that of smaller storage systems due to the increased number of storage nodes. RAID technology is no longer sufficient to guarantee the necessary high data reliability for such systems, because disk rebuild time lengthens as disk capacity grows. We present fast recovery mechanism (FARM), a distributed recovery approach that exploits excess disk capacity and reduces data recovery time. FARM works in concert with replication and erasure-coding redundancy schemes to dramatically lower the probability of data loss in large-scale storage systems. We have examined essential factors that influence system reliability, performance, and costs, such as failure detections, disk bandwidth usage for recovery, disk space utilization, disk drive replacement, and system scales, by simulating system behavior under disk failures. Our results show the reliability improvement from FARM and demonstrate the impacts of various factors on system reliability. Using our techniques, system designers will be better able to build multipetabyte storage systems with much higher reliability at lower cost than previously possible.
Bibliography:SourceType-Conference Papers & Proceedings-1
ObjectType-Conference Paper-1
content type line 25
ISBN:0769521754
9780769521756
0780321754
ISSN:1082-8907
DOI:10.1109/HPDC.2004.1323523