Research Article
Survey on Improved Scheduling in Hadoop MapReduce in Cloud Environments
|
International Journal of Computer Applications
Foundation of Computer Science (FCS), NY, USA
|
| Volume 34 - Issue 9 |
| Published: November 2011 |
| Authors: B.Thirumala Rao, Dr. L.S.S.Reddy |
10.5120/4128-5964
|
B.Thirumala Rao, Dr. L.S.S.Reddy . Survey on Improved Scheduling in Hadoop MapReduce in Cloud Environments. International Journal of Computer Applications. 34, 9 (November 2011), 29-33. DOI=10.5120/4128-5964
@article{ 10.5120/4128-5964,
author = { B.Thirumala Rao,Dr. L.S.S.Reddy },
title = { Survey on Improved Scheduling in Hadoop MapReduce in Cloud Environments },
journal = { International Journal of Computer Applications },
year = { 2011 },
volume = { 34 },
number = { 9 },
pages = { 29-33 },
doi = { 10.5120/4128-5964 },
publisher = { Foundation of Computer Science (FCS), NY, USA }
}
%0 Journal Article
%D 2011
%A B.Thirumala Rao
%A Dr. L.S.S.Reddy
%T Survey on Improved Scheduling in Hadoop MapReduce in Cloud Environments%T
%J International Journal of Computer Applications
%V 34
%N 9
%P 29-33
%R 10.5120/4128-5964
%I Foundation of Computer Science (FCS), NY, USA
Abstract
Cloud Computing is emerging as a new computational paradigm shift. Hadoop-MapReduce has become a powerful Computation Model for processing large data on distributed commodity hardware clusters such as Clouds. In all Hadoop implementations, the default FIFO scheduler is available where jobs are scheduled in FIFO order with support for other priority based schedulers also. In this paper we study various scheduler improvements possible with Hadoop and also provided some guidelines on how to improve the scheduling in Hadoop in Cloud Environments.
References
- Cloud Computing on Wikipedia, en.wikipedia.org / wiki /Cloudcomputing,
- NIST Definition of Cloud Computing v15, csrc.nist.gov/groups/SNS/cloud-computing/cloud-def-v15.doc
- Apache Hadoop. http://hadoop.apache.org.
- J. Dean and S. Ghemawat. Mapreduce: Simplified data processing on large clusters. OSDI ’04, pages 137–150, 2004
- Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung.The Google file system. In 19th Symposium on Operating Systems Principles, pages 29–43, Lake George, New York, 2003.
- Hadoop Distributed File System, http://hadoop.apache.org/hdfs
- Hadoop’s Fair Scheduler http://hadoop.apache.org/common/docs/r0.20.2/fair_scheduler.html
- Hadoop’s Capacity Scheduler: http://hadoop.apache.org/core/docs/current/capacity_scheduler.html.
- Matei Zaharia, Dhruba Borthakur, Joydeep Sen Sarma, Khaled Elmeleegy, Scott Shenker, and Ion Stoica. Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduling. In EuroSys ’10: Proceedings of the 5th European conference on Computer systems, pages 265–278, New York, NY, USA, 2010. ACM.
- Thomas Sandholm and Kevin Lai. Dynamic proportional share scheduling in hadoop. In JSSPP ’10: 15th Workshop on Job Scheduling Strategies for Parallel Processing, April,2010
- K. Kc and K. Anyanwu, "Scheduling Hadoop Jobs to Meet Deadlines", in Proc. CloudCom, 2010, pp.388-392.
- Mark Yong, Nitin Garegrat, Shiwali Mohan: “Towards a Resource Aware Scheduler in Hadoop” in Proc. ICWS, 2009, pp:102-109
- M.Zaharia, A.Konwinski, A.Joseph, Y.zatz, and I.Stoica. Improving mapreduce performance in heterogeneous environments. In OSDI’08: 8th USENIX Symposium on Operating Systems Design and Implementation, October 2008
- B.Thirmala Rao, N.V.Sridevei, V. Krishna Reddy, LSS.Reddy. Performance Issues of Heterogeneous Hadoop Clusters in Cloud Computing. Global Journal Computer Science & Technology Vol. 11, no. 8, May 2011,pp.81-87
- V.Krishna Reddy, B.Thirumala Rao, LSS Reddy. Research issues in Cloud Computing. Global Journal Computer Science & Technology Vol. 11, no. 11, June 2011,pp.70-76
- K. Thirupathi Rao, P. Sai Kiran, Dr. L.S.S Reddy, V. Krishna Reddy, B. Thirumala Rao, “Genetic Algorithm For Energy Efficient Placement Of Virtual Machines In Cloud Environment”, in proc IEEE International Conference on Future Information Technology (IEEE ICFIT 2010), China, December 2010, pp: V2-213 to V2-217.
Index Terms
Keywords