Adaptive particle swarm optimization for CNN associative memories design

In this paper particle swarm optimization is used to implement a synthesis procedure for cellular neural networks autoassociative memories. The use of this optimization technique allows a global search for computing the model parameters that identify designed memories, providing a synthesis procedure that takes into account the robustness of the solution. In particular, the design parameters can be modified during the convergence in order to guarantee minimum recall performances of the network in terms of robustness to noise overlapped to input patterns. Numerical results confirm the good performances of the designed networks when patterns are affected by different kinds of noise.

[1]  Paulo Cortez,et al.  Particle swarms for feedforward neural network training , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[2]  Xiao-Hu Yu,et al.  Efficient Backpropagation Learning Using Optimal Learning Rate and Momentum , 1997, Neural Networks.

[3]  Saman K. Halgamuge,et al.  Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients , 2004, IEEE Transactions on Evolutionary Computation.

[4]  Naum N. Aizenberg,et al.  Cellular neural networks and computational intelligence in medical image processing , 2001, Image Vis. Comput..

[5]  Lin-Bao Yang,et al.  Cellular neural networks: theory , 1988 .

[6]  G. Grassi,et al.  A new approach to design cellular neural networks for associative memories , 1997 .

[7]  Zhigang Zeng,et al.  Design and Analysis of High-Capacity Associative Memories Based on a Class of Discrete-Time Recurrent Neural Networks , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[8]  Derong Liu,et al.  Cloning template design of cellular neural networks for associative memories , 1997 .

[9]  Lifeng Xi,et al.  An Improved Particle Swarm Optimization for Evolving Feedforward Artificial Neural Networks , 2007, Neural Processing Letters.

[10]  Jin-Wook Chung,et al.  Network Intrusion Detection Through Genetic Feature Selection , 2006, Seventh ACIS International Conference on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing (SNPD'06).

[11]  Hussein Baher,et al.  Analog & digital signal processing , 1990 .

[12]  R. Eberhart,et al.  Empirical study of particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[13]  Michael R. Lyu,et al.  A hybrid particle swarm optimization-back-propagation algorithm for feedforward neural network training , 2007, Appl. Math. Comput..

[14]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[15]  Elisa Ricci,et al.  Cellular Neural Networks With Virtual Template Expansion for Retinal Vessel Segmentation , 2007, IEEE Transactions on Circuits and Systems II: Express Briefs.

[16]  Te-Jen Su,et al.  Particle Swarm Optimization for Image Noise Cancellation , 2006, ICICIC.

[17]  Ángel Rodríguez-Vázquez,et al.  Toward visual microprocessors , 2002, Proc. IEEE.

[18]  Ignacio Rojas,et al.  Statistical analysis of the parameters of a neuro-genetic algorithm , 2002, IEEE Trans. Neural Networks.

[19]  Derong Liu,et al.  Sparsely interconnected neural networks for associative memories with applications to cellular neural networks , 1994 .

[20]  Qing Song,et al.  Dynamic pruning algorithm for multilayer perceptron based neural control systems , 2006, Neurocomputing.

[21]  Klaus Mainzer,et al.  Cellular Neural Networks and Visual Computing , 2003, Int. J. Bifurc. Chaos.

[22]  Enrique Romero,et al.  Margin maximization with feed-forward neural networks: a comparative study with SVM and AdaBoost , 2004, Neurocomputing.

[23]  P.V.G.D. Prasad Reddy,et al.  A Comparative Analysis of Unsupervised K-Means, PSO and Self-Organizing PSO for Image Clustering , 2007, International Conference on Computational Intelligence and Multimedia Applications (ICCIMA 2007).

[24]  Andries Petrus Engelbrecht,et al.  A Cooperative approach to particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[25]  I.W. Kao,et al.  An effective particle swarm optimization method for data clustering. , 2007, 2007 IEEE International Conference on Industrial Engineering and Engineering Management.

[26]  Erik D. Goodman,et al.  Designing templates for cellular neural networks using particle swarm optimization , 2004, 33rd Applied Imagery Pattern Recognition Workshop (AIPR'04).

[27]  M. Brucoli,et al.  Heteroassociative memories via cellular neural networks , 1998 .

[28]  Danilo P. Mandic,et al.  Towards the Optimal Learning Rate for Backpropagation , 2000, Neural Processing Letters.

[29]  Derong Liu,et al.  A new synthesis approach for feedback neural networks based on the perceptron training algorithm , 1997, IEEE Trans. Neural Networks.

[30]  Derong Liu,et al.  A new synthesis procedure for a class of cellular neural networks with space-invariant cloning template , 1998 .

[31]  Jooyoung Park,et al.  An optimization approach to design of cellular neural networks , 2000, Int. J. Syst. Sci..

[32]  R. Eberhart,et al.  Comparing inertia weights and constriction factors in particle swarm optimization , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[33]  Lifeng Xi,et al.  Evolving artificial neural networks using an improved PSO and DPSO , 2008, Neurocomputing.

[34]  Leon O. Chua,et al.  CNN: A Vision of Complexity , 1997 .

[35]  Y. Park,et al.  A synthesis procedure for associative memories based on space-varying cellular neural networks , 2001, Neural Networks.

[36]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.