Mobile Agent Based New Framework for Improving Big Data Analysis

The rising number of applications serving millions of users and dealing with terabytes of data need to a faster processing paradigms. Recently, there is growing enthusiasm for the notion of big data analysis. Big data analysis becomes a very important aspect for growth productivity, reliability and quality of services (QoS). Processing of big data using a powerful machine is not efficient solution. So, companies focused on using Hadoop software for big data analysis. This is because Hadoop designed to support parallel and distributed data processing. However, Hadoop has several drawbacks effect on its performance and reliability against big data analysis. In this paper, a new framework is proposed to improve big data analysis and overcome the drawbacks of Hadoop. The proposed framework is called MapReduce Agent Mobility (MRAM). MRAM is developed by using mobile agent and MapReduce paradigm under Java Agent Development Framework (JADE).

[1]  A. Kala Karun,et al.  A review on hadoop — HDFS infrastructure extensions , 2013, 2013 IEEE CONFERENCE ON INFORMATION AND COMMUNICATION TECHNOLOGIES.

[2]  GhemawatSanjay,et al.  The Google file system , 2003 .

[3]  Hairong Kuang,et al.  The Hadoop Distributed File System , 2010, 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST).

[4]  Danny B. Lange,et al.  Programming and Deploying Java¿ Mobile Agents with Aglets¿ , 1998 .

[5]  Jun Wang,et al.  Improving metadata management for small files in HDFS , 2009, 2009 IEEE International Conference on Cluster Computing and Workshops.

[6]  Weizhong Zhao,et al.  h-MapReduce: A Framework for Workload Balancing in MapReduce , 2013, 2013 IEEE 27th International Conference on Advanced Information Networking and Applications (AINA).

[7]  Alan L. Cox,et al.  The Hadoop distributed filesystem: Balancing portability and performance , 2010, 2010 IEEE International Symposium on Performance Analysis of Systems & Software (ISPASS).

[8]  Torsten Eymann,et al.  Technical opinion: The real challenges of mobile agents , 2000, CACM.

[9]  Peter Braun,et al.  Mobile Agents: Basic Concepts, Mobility Models, and the Tracy Toolkit , 2004 .

[10]  Chuck Lam,et al.  Hadoop in Action , 2010 .

[11]  Sanjay Ghemawat,et al.  MapReduce: a flexible data processing tool , 2010, CACM.

[12]  Guanyu Li,et al.  Researches on Performance Optimization of Distributed Integrated System Based on Mobile Agent , 2006, 2006 6th World Congress on Intelligent Control and Automation.

[13]  Xiaoqiao Meng,et al.  Coupling task progress for MapReduce resource-aware scheduling , 2013, 2013 Proceedings IEEE INFOCOM.