A write-friendly approach to manage namespace of Hadoop distributed file system by utilizing nonvolatile memory

With the emergence of the big data era, various technologies have been proposed to cope with the exascale of data. For a considerably large volume of data, a single machine does not comprise enough resources to store the complete data. Hadoop distributed file system (HDFS) enables large datasets to...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:The Journal of supercomputing Ročník 75; číslo 10; s. 6632 - 6662
Hlavní autoři: Choi, Won Gi, Park, Sanghyun
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York Springer US 01.10.2019
Springer Nature B.V
Témata:
Big Data
Compilers
Computer Science
Data management
Data storage
Disk drives
Flash memory (computers)
Interpreters
Memory devices
Metadata
Processor Architectures
Programming Languages
Random access memory
Solid state devices
NameNode
Hadoop filesystem
Nonvolatile memory
Distributed computing
ISSN:0920-8542, 1573-0484
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:With the emergence of the big data era, various technologies have been proposed to cope with the exascale of data. For a considerably large volume of data, a single machine does not comprise enough resources to store the complete data. Hadoop distributed file system (HDFS) enables large datasets to be stored across the big data environment consisting of several machines. Although Hadoop has become a crucial part of the big data industry, because of its simple architecture which composed of master and slaves several problems such as scalability and performance bottleneck has been remained to solve. New storage technologies offer an opportunity to solve the problems and improve HDFS. We propose a novel management scheme for namespace metadata of HDFS by utilizing nonvolatile memory which has been mentioned as the next-generation device since flash memory devices. Nonvolatile memory, which can guarantee data persistence and high performance with byte-address access, alleviates Namenode bottlenecks resulting from journaling processes performed to preserve the file system’s metadata. Our proposed methods show significant improvement compared with block devices such as hard disk drive, solid-state drive in terms of NameNode performance.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0920-8542
1573-0484
DOI:10.1007/s11227-019-02876-9