Big Data and MapReduce Challenges, Opportunities and Trends

Nowadays we all are surrounded by Big data. The term ‘Big Data’ itself indicates huge volume, high velocity, variety and veracity i.e. uncertainty of data which gave rise to new difficulties and challenges. Big data generated may be structured data, Semi Structured data or unstructured data. For existing database and systems lot of difficulties are there to process, analyze, store and manage such a Big Data.  The Big Data challenges are Protection, Curation, Capture, Analysis, Searching, Visualization, Storage, Transfer and sharing. Map Reduce is a framework using which we can write applications to process huge amount of data, in parallel, on large clusters of commodity hardware in a reliable manner. Lot of efforts have been put by different researchers to make it simple, easy, effective and efficient. In our survey paper we emphasized on the working of Map Reduce, challenges, opportunities and recent trends so that researchers can think on further improvement. Full Text: PDF DOI: http://dx.doi.org/10.11591/ijece.v6i6.10555

[1]  Zeba Khanam,et al.  Map Reduce: A Survey Paper on Recent Expansion , 2015 .

[2]  Raul Queiroz Feitosa,et al.  CLASSIFICATION ALGORITHMS FOR BIG DATA ANALYSIS, A MAP REDUCE APPROACH , 2015, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[3]  Jun Liu,et al.  An Efficient Job Scheduling for MapReduce Clusters , 2015 .

[4]  Hamid Bagheri,et al.  Big Data: Challenges, Opportunities and Cloud Based Solutions , 2015 .

[5]  Vimla Jethani,et al.  Distributed Metadata Management Scheme in HDFS , 2013 .

[6]  G. Vadivu,et al.  MapReduce: A Technical Review , 2016 .

[7]  Du Zhang,et al.  Inconsistencies in big data , 2013, 2013 IEEE 12th International Conference on Cognitive Informatics and Cognitive Computing.

[8]  Jie Wu,et al.  Dache: A data aware caching for big-data applications using the MapReduce framework , 2014 .

[9]  Khadija Ateya Almohsen,et al.  Recommender Systems in Light of Big Data , 2015 .

[10]  Gabriel Antoniu,et al.  Optimizing intermediate data management in MapReduce computations , 2011, CloudCP '11.

[11]  Sungyoung Lee,et al.  Context‐aware scheduling in MapReduce: a compact review , 2015, Concurr. Comput. Pract. Exp..

[12]  Wei Fan,et al.  Mining big data: current status, and forecast to the future , 2013, SKDD.

[13]  N. B. Anuar,et al.  The rise of "big data" on cloud computing: Review and open research issues , 2015, Inf. Syst..

[14]  Bo Zhang,et al.  Self-Configuration of the Number of Concurrently Running MapReduce Jobs in a Hadoop Cluster , 2015, 2015 IEEE International Conference on Autonomic Computing.

[15]  Miriam A. M. Capretz,et al.  Challenges for MapReduce in Big Data , 2014, 2014 IEEE World Congress on Services.

[16]  Priyaneet Bhatia,et al.  Correlated Appraisal of Big Data, Hadoop and MapReduce , 2015 .

[17]  Shukun Liu,et al.  The Optimization and Improvement of MapReduce in Web Data Mining , 2015, ICA3PP.

[18]  Atul Negi,et al.  Performance Improvement of MapReduce Framework in Heterogeneous Context using Reinforcement Learning , 2015 .

[19]  P. Kannan.  Review Paper on Data-aware Caching for Big Data Applications , 2015 .