Self-Adaptive Parameters Optimization for Incremental Classification in Big Data Using Neural Network

Big Data is being touted as the next big thing arousing technical challenges that confront both academic research communities and commercial IT deployment. The root sources of Big Data are founded on infinite data streams and the curse of dimensionality. It is generally known that data which are sourced from data streams accumulate continuously making traditional batch-based model induction algorithms infeasible for real-time data mining. In the past many methods have been proposed for incrementally data mining by modifying classical machine learning algorithms, such as artificial neural network. In this paper we propose an incremental learning process for supervised learning with parameters optimization by neural network over data stream. The process is coupled with a parameters optimization module which searches for the best combination of input parameters values based on a given segment of data stream. The drawback of the optimization is the heavy consumption of time. To relieve this limitation, a loss function is proposed to look ahead for the occurrence of concept-drift which is one of the main causes of performance deterioration in data mining model. Optimization is skipped intermittently along the way so to save computation costs. Computer simulation is conducted to confirm the merits by this incremental optimization process for neural network.

[1]  Marko Tscherepanow,et al.  Incremental On-line Clustering with a Topology-Learning Hierarchical ART Neural Network Using Hyperspherical Categories , 2012, ICDM.

[2]  Robert Sabourin,et al.  Incremental adaptation of fuzzy ARTMAP neural networks for video-based face classification , 2009, 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications.

[3]  Ron Kohavi,et al.  The Power of Decision Tables , 1995, ECML.

[4]  Giovanni Soda,et al.  Learning incremental syntactic structures with recursive neural networks , 2000, KES'2000. Fourth International Conference on Knowledge-Based Intelligent Engineering Systems and Allied Technologies. Proceedings (Cat. No.00TH8516).

[5]  John A. Bullinaria,et al.  Evolving improved incremental learning schemes for neural network systems , 2005, 2005 IEEE Congress on Evolutionary Computation.

[6]  Shonali Krishnaswamy,et al.  Mining data streams: a review , 2005, SGMD.

[7]  Wei Fan,et al.  Mining big data: current status, and forecast to the future , 2013, SKDD.

[8]  Gianmarco De Francisci Morales,et al.  Distributed Decision Tree Learning for Mining Big Data Streams , 2013 .

[9]  Ping-Feng Pai,et al.  Rough set theory with discriminant analysis in analyzing electricity loads , 2009, Expert Syst. Appl..

[10]  Aristidis Likas,et al.  An incremental training method for the probabilistic RBF network , 2006, IEEE Trans. Neural Networks.

[11]  Honglak Lee,et al.  Online Incremental Feature Learning with Denoising Autoencoders , 2012, AISTATS.

[12]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  A. Ivakhnenko.  Heuristic self-organization in problems of engineering cybernetics , 1970 .

[14]  Ron Kohavi,et al.  Wrappers for performance enhancement and oblivious decision graphs , 1995 .

[15]  Christopher MacLeod,et al.  Incremental Evolution in ANNs: Neural Nets which Grow , 2001, Artificial Intelligence Review.

[16]  Mark Ring.  Sequence Learning with Incremental Higher-Order Neural Networks , 1993 .

[17]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[18]  Giulio Sandini,et al.  An incremental growing neural network and its application to robot control , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[19]  Shigeo Abe,et al.  A reinforcement learning algorithm for neural networks with incremental learning ability , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[20]  Simon Fong,et al.  Feature Selection in Life Science Classification: Metaheuristic Swarm Search , 2014, IT Professional.

[21]  P. McCullagh,et al.  Generalized Linear Models, 2nd Edn. , 1990 .

[22]  Loïc Kessous,et al.  Adaptive On-Line Neural Network Retraining for Real Life Multimodal Emotion Recognition , 2006, ICANN.

[23]  Oya Aran.  An Incremental Neural Network Construction Algorithm for Training Multilayer Perceptrons , 2003 .