The Hadoop Distributed File System
The Hadoop Distributed File System (HDFS) is designed to store very large data sets reliably, and to stream those data sets at high bandwidth to user applications. In a large cluster, thousands of servers both host directly attached storage and execute user application tasks. By distributing storage...
Gespeichert in:
| Veröffentlicht in: | 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST) S. 1 - 10 |
|---|---|
| Hauptverfasser: | , , , |
| Format: | Tagungsbericht |
| Sprache: | Englisch |
| Veröffentlicht: |
IEEE
01.05.2010
|
| Schlagworte: | |
| ISBN: | 1424471524, 9781424471522 |
| ISSN: | 2160-195X |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | The Hadoop Distributed File System (HDFS) is designed to store very large data sets reliably, and to stream those data sets at high bandwidth to user applications. In a large cluster, thousands of servers both host directly attached storage and execute user application tasks. By distributing storage and computation across many servers, the resource can grow with demand while remaining economical at every size. We describe the architecture of HDFS and report on experience using HDFS to manage 25 petabytes of enterprise data at Yahoo!. |
|---|---|
| AbstractList | The Hadoop Distributed File System (HDFS) is designed to store very large data sets reliably, and to stream those data sets at high bandwidth to user applications. In a large cluster, thousands of servers both host directly attached storage and execute user application tasks. By distributing storage and computation across many servers, the resource can grow with demand while remaining economical at every size. We describe the architecture of HDFS and report on experience using HDFS to manage 25 petabytes of enterprise data at Yahoo!. |
| Author | Hairong Kuang Chansler, R Shvachko, K Radia, S |
| Author_xml | – sequence: 1 givenname: K surname: Shvachko fullname: Shvachko, K email: Shv@Yahoo-Inc.com organization: Yahoo!, Sunnyvale, CA, USA – sequence: 2 surname: Hairong Kuang fullname: Hairong Kuang email: Hairong@Yahoo-Inc.com organization: Yahoo!, Sunnyvale, CA, USA – sequence: 3 givenname: S surname: Radia fullname: Radia, S email: SRadia@Yahoo-Inc.com organization: Yahoo!, Sunnyvale, CA, USA – sequence: 4 givenname: R surname: Chansler fullname: Chansler, R email: Chansler@Yahoo-Inc.com organization: Yahoo!, Sunnyvale, CA, USA |
| BookMark | eNo1j0FPwkAQhceIiYD9AcZL4724M7vbZY8GQUwgHNqDNzLNTuMaoKStB_69TcR3efkuX96bwOjUnATgEdUMUfmXbVGUM1IDWuNz7-gGEu_maMgYh1b7W5j8A5kRjAlzlaG3n_eQdN23GmLsINBjeC6_JF1zaJpz-ha7vo3VTy8hXcWDpMWl6-X4AHc1HzpJrj2FcrUsF-tss3v_WLxusojO9hk7K4zDQEMsRnJNrJCVZouObG6CcxwUBV3V6FlTZUKNta15TpWlSk_h6U8bRWR_buOR28v--lD_AkHuQhs |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IH CBEJK RIE RIO |
| DOI | 10.1109/MSST.2010.5496972 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP) 1998-present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISBN | 9781424471539 1424471532 |
| EndPage | 10 |
| ExternalDocumentID | 5496972 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IF 6IH 6IK 6IL 6IM 6IN AAJGR AAWTH ABLEC ACGFS ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IJVOP IPLJI M43 OCL RIE RIL RIO |
| ID | FETCH-LOGICAL-i175t-a75ea111042ae4e632a01a03a5172564d77ad02d3bf19a32b4df1f5fa82b52b3 |
| IEDL.DBID | RIE |
| ISBN | 1424471524 9781424471522 |
| ISICitedReferencesCount | 1347 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000287502800009&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 2160-195X |
| IngestDate | Wed Aug 27 02:35:55 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i175t-a75ea111042ae4e632a01a03a5172564d77ad02d3bf19a32b4df1f5fa82b52b3 |
| PageCount | 10 |
| ParticipantIDs | ieee_primary_5496972 |
| PublicationCentury | 2000 |
| PublicationDate | 2010-May |
| PublicationDateYYYYMMDD | 2010-05-01 |
| PublicationDate_xml | – month: 05 year: 2010 text: 2010-May |
| PublicationDecade | 2010 |
| PublicationTitle | 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST) |
| PublicationTitleAbbrev | MSST |
| PublicationYear | 2010 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0000452013 ssj0000942121 |
| Score | 2.088143 |
| Snippet | The Hadoop Distributed File System (HDFS) is designed to store very large data sets reliably, and to stream those data sets at high bandwidth to user... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 1 |
| SubjectTerms | Bandwidth Clustering algorithms Computer architecture Concurrent computing Distributed computing distributed file system File servers File systems Hadoop HDFS Protection Protocols |
| Title | The Hadoop Distributed File System |
| URI | https://ieeexplore.ieee.org/document/5496972 |
| WOSCitedRecordID | wos000287502800009&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwED6VigEWHi3irQgxYmo7cezMQMVCVakdulV-SmVoqtLy-znnUUBiYYuzJJbt7-7z3X0HcJ8JawtqFMm1tSRTQRHlDe7llBnnAppgaatmE3I0UrNZMe7Aw64WxntfJZ_5x_hYxfJdabfxqmyAXCYvJALunpR5Xau1u0-J0uAtVXivM-YQlSPf4iynhBVi1tZ1SbRZWSv31Ix5E_FktBi8TSbTOumr-eCvziuV4Rke_e-Xj6H_XcGXjHe26QQ6fnkKhz_EB3twhzskQeApy1XyHOVzY-cr75Ih4kRSC5n3YTp8mT69kqZjAlmgG7AhWgqvEb3wJGqf-TzlmjJNUy3QTxF55qTUjnKXmsAKnXKTucCCCFpxI7hJz6C7LJf-HBJho65OQH8EGRd3QhkvmY6JmxpJraAX0IuTna9qTYx5M8_Lv19fwUEbdafsGrqb9dbfwL793Cw-1rfVQn4BUO2UQw |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwED5VBQlYeLSINxFiJNR27DiZgaqItqrUDN0qvyKVoa1Ky-_nnEcBiYUtzpJYtr-7z3f3HcA9F8akRCdhrIwJeZInYeI07uWIamtzNMHSFM0m5HCYTCbpqAEP21oY51yRfOYe_WMRy7cLs_FXZR3kMnEqEXB3BOeMlNVa2xsVLw5ek4X3MmcOcdkzLkZjEtJUTOrKLolWi9eCT9WYVTFPStLOYDzOyrSv6pO_eq8Upqd7-L-fPoL2dw1fMNpap2NouPkJHPyQH2zBHe6RAKFnsVgGz15A1_e-cjboIlIEpZR5G7LuS_bUC6ueCeEMHYF1qKRwCvELz6Jy3MURU4QqEimBnoqIuZVSWcJspHOaqohpbnOai1wlTAumo1NozhdzdwaBMF5ZJ0ePBDkXsyLRTlLlUzcV0lpBzqHlJztdlqoY02qeF3-_voW9XjboT_uvw7dL2K9j8IReQXO92rhr2DWf69nH6qZY1C9F6JeK |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2010+IEEE+26th+Symposium+on+Mass+Storage+Systems+and+Technologies+%28MSST%29&rft.atitle=The+Hadoop+Distributed+File+System&rft.au=Shvachko%2C+K&rft.au=Hairong+Kuang&rft.au=Radia%2C+S&rft.au=Chansler%2C+R&rft.date=2010-05-01&rft.pub=IEEE&rft.isbn=9781424471522&rft.issn=2160-195X&rft.spage=1&rft.epage=10&rft_id=info:doi/10.1109%2FMSST.2010.5496972&rft.externalDocID=5496972 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2160-195X&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2160-195X&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2160-195X&client=summon |