Big data clustering using grid computing and ant-based algorithm

Big data has the power to dramatically change the way institutes and organizations use their data. Transforming the massive amounts of data into knowledge will leverage the organizations performance to the maximum.Scientific and business organizations would benefit from utilizing big data. However, there are many challenges in dealing with big data such as storage, transfer, management and manipulation of big data.Many techniques are required to explore the hidden pattern inside the big data which have limitations in terms of hardware and software implementation. This paper presents a framework for big data clustering which utilizes grid technology and ant-based algorithm.

[1]  Joydeep Ghosh,et al.  Cluster Ensembles --- A Knowledge Reuse Framework for Combining Multiple Partitions , 2002, J. Mach. Learn. Res..

[2]  Manuel López-Ibáñez,et al.  Ant colony optimization , 2010, GECCO '10.

[3]  Lawrence O. Hall.  Exploring Big Data with Scalable Soft Clustering , 2012, SMPS.

[4]  Marco Dorigo,et al.  Distributed Optimization by Ant Colonies , 1992 .

[5]  Xiaoyong Du,et al.  Big data challenge: a data management perspective , 2013, Frontiers of Computer Science.

[6]  Zhenlin Zhang,et al.  A New Hybrid Ant Colony Algorithm for Solving Vehicle Scheduling Problem , 2012 .

[7]  Roy H. Campbell,et al.  Scalable Storage for Data-Intensive Computing , 2011 .

[8]  Norbert Meyer,et al.  Analysis of Grid Storage Element Architectures: High-end Fiber-Channel vs. Emerging Cluster-based Networked Storage , 2008 .

[9]  Xueliang Fu,et al.  An Ant System-Assisted Genetic Algorithm For Solving The Traveling Salesman Problem , 2012 .

[10]  Xianmin Wei.  Study of Ant Colony Hybrid Algorithm in Grid Task Scheduling , 2012 .

[11]  Vijay Srinivas Agneeswaran.  Big-Data - Theoretical, Engineering and Analytics Perspective , 2012, BDA.

[12]  Marco Dorigo,et al.  Ant-Based Clustering and Topographic Mapping , 2006, Artificial Life.

[13]  Jean-Louis Deneubourg,et al.  The dynamics of collective sorting robot-like ants and ant-like robots , 1991 .

[14]  I. Halcu,et al.  A big data implementation based on Grid computing , 2013, 2013 11th RoEduNet International Conference.

[15]  Luca Maria Gambardella,et al.  Ant colony system: a cooperative learning approach to the traveling salesman problem , 1997, IEEE Trans. Evol. Comput..

[16]  Xiongpai Qin.  Making Use of the Big Data: Next Generation of Algorithm Trading , 2012, AICI.

[17]  Krzysztof Pancerz,et al.  Ant Based Clustering of Two-Class Sets with Well Categorized Objects , 2012, IPMU.

[18]  Amit Konar,et al.  Metaheuristic Pattern Clustering – An Overview , 2009 .

[19]  Mohamed S. Kamel,et al.  An aggregated clustering approach using multi-ant colonies algorithms , 2006, Pattern Recognit..

[20]  B. Bullnheimer,et al.  A NEW RANK BASED VERSION OF THE ANT SYSTEM: A COMPUTATIONAL STUDY , 1997 .

[21]  Lei Yu,et al.  Grid Resource Management: Toward Virtual and Services Compliant Grid Computing , 2008 .

[22]  Byung-Joo Kim.  A Classifier for Big Data , 2012, ICHIT.

[23]  Baldo Faieta,et al.  Diversity and adaptation in populations of clustering ants , 1994 .

[24]  Daniel T. Larose,et al.  Discovering Knowledge in Data: An Introduction to Data Mining , 2005 .

[25]  T. Stützle,et al.  MAX-MIN Ant System and local search for the traveling salesman problem , 1997, Proceedings of 1997 IEEE International Conference on Evolutionary Computation (ICEC '97).

[26]  Zhu Yunxia.  Study on the Resource Allocation Algorithm Based on Ant Colony Optimization , 2012 .

[27]  R. S. D. Wahida Banu,et al.  Communication Aware Co-Scheduling For Parallel Job Scheduling In Cluster Computing , 2011, ACC.