Optimizing Small Files Operations in HDFS File Storage Mode.
Saved in:
| Title: | Optimizing Small Files Operations in HDFS File Storage Mode. |
|---|---|
| Authors: | Yi-Yang Chen, Rui-Jun Wang, Zhen Hong, Akhtar, Zahid, Siddique, Kamran |
| Source: | International Journal of Design, Analysis & Tools for Integrated Circuits & Systems; Nov2022, Vol. 11 Issue 1, p43-49, 7p |
| Subject Terms: | RECORDS management, STORAGE, BIG data |
| Abstract: | Hadoop Distributed File System (HDFS) is based on Google File System (GFS), a big data distributed file management system included in Hadoop. Nowadays, many HDFS and many other similar frameworks have the need to store small files in the system. In this aspect, HDFS affects its performance and Namenode memory management when dealing with a large number of small files. Therefore, researchers have proposed various solutions to address the shortcomings of HDFS for storing small and medium-sized files. This paper presents three HDFS schemes for merging small files and analyzes the importance of correlation and prefetching after merging small files. The efficiency of reading small files can be improved by correlated file prefetching. Finally, the small file storage architecture is obtained to stand superior to the NHAR architecture. [ABSTRACT FROM AUTHOR] |
| Copyright of International Journal of Design, Analysis & Tools for Integrated Circuits & Systems is the property of International Journal of Design, Analysis & Tools for Integrated Circuits & Systems and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Complementary Index |
Nájsť tento článok vo Web of Science Be the first to leave a comment!