A new class of multi-stable neural networks: Stability analysis and learning process

Recently, multi-stable Neural Networks (NN) with exponential number of attractors have been presented and analyzed theoretically; however, the learning process of the parameters of these systems while considering stability conditions and specifications of real world problems has not been studied. In this paper, a new class of multi-stable NNs using sinusoidal dynamics with exponential number of attractors is introduced. The sufficient conditions for multi-stability of the proposed system are posed using Lyapunov theorem. In comparison to the other methods in this class of multi-stable NNs, the proposed method is used as a classifier by applying a learning process with respect to the topological information of data and conditions of Lyapunov multi-stability. The proposed NN is applied on both synthetic and real world datasets with an accuracy comparable to classical classifiers.

[1]  Jinde Cao,et al.  Multistability and Instability of Competitive Neural Networks with Mexican-Hat-Type Activation Functions , 2014 .

[2]  H. Deutsch.  Principle Component Analysis , 2004 .

[3]  Narendra Ahuja,et al.  Learning to Recognize 3D Objects with SNoW , 2000, ECCV.

[4]  Y. Kuznetsov.  Elements of applied bifurcation theory (2nd ed.) , 1998 .

[5]  Yanke Du,et al.  Multistability and Multiperiodicity for a Class of Cohen–Grossberg BAM Neural Networks with Discontinuous Activation Functions and Time Delays , 2014, Neural Processing Letters.

[6]  J. Stoer,et al.  Introduction to Numerical Analysis , 2002 .

[7]  Jenq-Neng Hwang,et al.  Handbook of Neural Network Signal Processing , 2000, IEEE Transactions on Neural Networks.

[8]  Hakan Erdogan,et al.  Max-Margin Stacking and Sparse Regularization for Linear Classifier Combination and Selection , 2011, ArXiv.

[9]  Mohammad Rahmati,et al.  Theoretical aspects of mapping to multidimensional optimal regions as a multi-classifier , 2013, Intell. Data Anal..

[10]  Tianping Chen,et al.  Global $\mu$ -Stability of Delayed Neural Networks With Unbounded Time-Varying Delays , 2007, IEEE Transactions on Neural Networks.

[11]  D. Zhang,et al.  Principle Component Analysis , 2004 .

[12]  Tsau Young Lin,et al.  Data Mining: Foundations and Practice , 2008, Data Mining: Foundations and Practice.

[13]  G. Cauwenberghs,et al.  Memristor-based neural networks: Synaptic versus neuronal stochasticity , 2016 .

[14]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[15]  Tianping Chen,et al.  Power-Rate Global Stability of Dynamical Systems With Unbounded Time-Varying Delays , 2007, IEEE Transactions on Circuits and Systems II: Express Briefs.

[16]  R. Maronna.  Alan Julian Izenman (2008): Modern Multivariate Statistical Techniques: Regression, Classification and Manifold Learning , 2011 .

[17]  Michael T. Heath,et al.  Scientific Computing: An Introductory Survey , 1996 .

[18]  Tianping Chen,et al.  Multiple µ-stability of neural networks with unbounded time-varying delays , 2014, Neural Networks.

[19]  Günther Palm,et al.  Neural associative memories and sparse coding , 2013, Neural Networks.

[20]  Elisa Ricci,et al.  Recurrent Correlation Associative Memories: A Feature Space Perspective , 2008, IEEE Transactions on Neural Networks.

[21]  Carl D. Meyer,et al.  Matrix Analysis and Applied Linear Algebra , 2000 .

[22]  Y. Kuznetsov.  Elements of Applied Bifurcation Theory , 2023, Applied Mathematical Sciences.

[23]  徐瑞,et al.  Multistability and multiperiodicity for a class of Cohen-Grossberg BAM neural networks with discontinuous activation functions and time delays , 2015 .

[24]  W. Duch,et al.  Challenges for Computational Intelligence (Duch, W. and Mandziuk, J., Eds.; 2007) [Book reviews] , 2009, IEEE Transactions on Neural Networks.

[25]  Jacek M. Zurada,et al.  Introduction to artificial neural systems , 1992 .

[26]  Milan Sonka,et al.  Image Processing, Analysis and Machine Vision , 1993, Springer US.

[27]  Jose B. Cruz,et al.  Two coding strategies for bidirectional associative memory , 1990, IEEE Trans. Neural Networks.

[28]  RahmatiMohammad,et al.  Theoretical aspects of mapping to multidimensional optimal regions as a multi-classifier , 2013, IDA 2013.

[29]  Zhigang Zeng,et al.  Multistability of Neural Networks With Time-Varying Delays and Concave-Convex Characteristics , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[30]  Michel Verleysen,et al.  Nonlinear Dimensionality Reduction , 2021, Computer Vision.

[31]  Shuzhi Sam Ge,et al.  Nonlinear identi cation and control — a neural network approach , 2022 .

[32]  Huaguang Zhang,et al.  Multistability of complex-valued recurrent neural networks with real-imaginary-type activation functions , 2014, Appl. Math. Comput..

[33]  Weiping Li,et al.  Applied Nonlinear Control , 1991 .

[34]  Osamu Hasegawa,et al.  Associative Memory for Online Learning in Noisy Environments Using Self-Organizing Incremental Neural Network , 2009, IEEE Transactions on Neural Networks.

[35]  A. Janczak,et al.  Identification of Nonlinear Systems Using Neural Networks and Polynomial Models: A Block-Oriented Approach , 2004 .

[36]  Sio Iong Ao,et al.  Applied Time Series Analysis and Innovative Computing , 2010, Lecture Notes in Electrical Engineering.

[37]  S. Strogatz.  Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry and Engineering , 1995 .

[38]  David Terman,et al.  An Introduction to Dynamical Systems and Neuronal Dynamics , 2005 .

[39]  Niels Kjølstad Poulsen,et al.  Neural Networks for Modelling and Control of Dynamic Systems: A Practitioner’s Handbook , 2000 .

[40]  Mounir Boukadoum,et al.  A bidirectional heteroassociative memory for binary and grey-level patterns , 2006, IEEE Transactions on Neural Networks.

[41]  Martin A. Riedmiller,et al.  Incremental GRLVQ: Learning relevant features for 3D object recognition , 2008, Neurocomputing.

[42]  Chang-Yuan Cheng,et al.  Multistability and convergence in delayed neural networks , 2007 .

[43]  Jinde Cao,et al.  Multistability of competitive neural networks with time-varying and distributed delays , 2009 .

[44]  Lei Xu,et al.  A Trend on Regularization and Model Selection in Statistical Learning: A Bayesian Ying Yang Learning Perspective , 2007, Challenges for Computational Intelligence.

[45]  Narendra Ahuja,et al.  Learning to Recognize Three-Dimensional Objects , 2002, Neural Computation.

[46]  Stuart J. Russell,et al.  Online bagging and boosting , 2005, 2005 IEEE International Conference on Systems, Man and Cybernetics.

[47]  Mohammad Rahmati,et al.  Enhancing the Accuracy of Mapping to Multidimensional Optimal Regions using PCA , 2012, IJCCI.

[48]  Jose B. Cruz,et al.  Encoding strategy for maximum noise tolerance bidirectional associative memory , 2005, IEEE Transactions on Neural Networks.

[49]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[50]  Zhigang Zeng,et al.  Stability analysis of delayed cellular neural networks described using cloning templates , 2004, IEEE Trans. Circuits Syst. I Regul. Pap..

[51]  Tianping Chen,et al.  Coexistence and local stability of multiple equilibria in neural networks with piecewise linear nondecreasing activation functions , 2010, Neural Networks.

[52]  Jun Wang,et al.  Multiperiodicity and Exponential Attractivity Evoked by Periodic External Inputs in Delayed Cellular Neural Networks , 2006 .

[53]  R. K. Agrawal,et al.  Evaluation of Decision Tree SVM Framework Using Different Statistical Measures , 2009, 2009 International Conference on Advances in Recent Technologies in Communication and Computing.