Lessons learned from a year’s worth of benchmarks of large data clouds
Saved in:
| Title: | Lessons learned from a year’s worth of benchmarks of large data clouds |
|---|---|
| Authors: | Yunhong Gu, Robert L Grossman |
| Contributors: | The Pennsylvania State University CiteSeerX Archives |
| Source: | http://pubs.rgrossman.com/dl/proc-117.pdf. |
| Publication Year: | 2009 |
| Collection: | CiteSeerX |
| Subject Terms: | Performance, Experimentation Keywords Cloud Computing, Data Intensive Computing, High Performance Computing, Grid Computing, MapReduce, Multi-Task Computing |
| Description: | In this paper, we discuss some of the lessons that we have learned working with the Hadoop and Sector/Sphere systems. Both of these systems are cloud-based systems designed to support data intensive computing. Both include distributed file systems and closely coupled systems for processing data in parallel. Hadoop uses MapReduce, while Sphere supports the ability to execute an arbitrary user defined function over the data managed by Sector. We compare and contrast these systems and discuss some of the design trade-offs necessary in data intensive computing. In our experimental studies over the past year, Sector/Sphere has consistently performed about 2 – 4 times faster than Hadoop. We discuss some of the reasons that might be responsible for this difference in performance. |
| Document Type: | text |
| File Description: | application/pdf |
| Language: | English |
| Relation: | http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.617.9116 |
| Availability: | http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.617.9116 http://pubs.rgrossman.com/dl/proc-117.pdf |
| Rights: | Metadata may be used without restrictions as long as the oai identifier remains attached to it. |
| Accession Number: | edsbas.5EEB6BA4 |
| Database: | BASE |
Nájsť tento článok vo Web of Science | Abstract: | In this paper, we discuss some of the lessons that we have learned working with the Hadoop and Sector/Sphere systems. Both of these systems are cloud-based systems designed to support data intensive computing. Both include distributed file systems and closely coupled systems for processing data in parallel. Hadoop uses MapReduce, while Sphere supports the ability to execute an arbitrary user defined function over the data managed by Sector. We compare and contrast these systems and discuss some of the design trade-offs necessary in data intensive computing. In our experimental studies over the past year, Sector/Sphere has consistently performed about 2 – 4 times faster than Hadoop. We discuss some of the reasons that might be responsible for this difference in performance. |
|---|