Convergence Analysis of a New Self Organizing Map Based Optimization (SOMO) Algorithm

The self-organizing map (SOM) approach has been used to perform cognitive and biologically inspired computing in a growing range of cross-disciplinary fields. Recently, the SOM based neural network framework was adapted to solve continuous derivative-free optimization problems through the development of a novel algorithm, termed SOM-based optimization (SOMO). However, formal convergence questions remained unanswered which we now aim to address in this paper. Specifically, convergence proofs are developed for the SOMO algorithm using a specific distance measure. Numerical simulation examples are provided using two benchmark test functions to support our theoretical findings, which illustrate that the distance between neurons decreases at each iteration and finally converges to zero. We also prove that the function value of the winner in the network decreases after each iteration. The convergence performance of SOMO has been benchmarked against the conventional particle swarm optimization algorithm, with preliminary results showing that SOMO can provide a more accurate solution for the case of large population sizes.

[1]  Mu-Chun Su,et al.  Optimal construction sequencing for secant pile wall , 2008, 2008 IEEE International Conference on Industrial Engineering and Engineering Management.

[2]  T. Manning,et al.  Naturally selecting solutions , 2013, Bioengineered.

[3]  Yuan Lan,et al.  Two-stage extreme learning machine for regression , 2010, Neurocomputing.

[4]  Wu Wei,et al.  MaxMin-SOMO: An SOM Optimization Algorithm for Simultaneously Finding Maximum and Minimum of a Function , 2012, ISNN.

[5]  Mu-Chun Su,et al.  Comparison of SOM-based optimization and particle swarm optimization for minimizing the construction time of a secant pile wall , 2009 .

[6]  Galileo Galilei,et al.  The Controversy on the comets of 1618: Galileo Galilei, Horatio Grassi, Mario Guiducci, Johann Kepler. , 1960 .

[7]  Kenneth Alan De Jong,et al.  An analysis of the behavior of a class of genetic adaptive systems. , 1975 .

[8]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[9]  Alan S. Perelson,et al.  The immune system, adaptation, and machine learning , 1986 .

[10]  Mu-Chun Su,et al.  SOM-based optimization , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[11]  Stefan Wermter,et al.  Data mining using rule extraction from Kohonen self-organising maps , 2006, Neural Computing & Applications.

[12]  Jiuwen Cao,et al.  Protein Sequence Classification with Improved Extreme Learning Machine Algorithms , 2014, BioMed research international.

[13]  Guang-Bin Huang,et al.  Convex incremental extreme learning machine , 2007, Neurocomputing.

[14]  WuWei,et al.  Double parallel feedforward neural network based on extreme learning machine with L 1/2 regularizer , 2014 .

[15]  Qidi Wu,et al.  Mussels Wandering Optimization: An Ecologically Inspired Algorithm for Global Optimization , 2012, Cognitive Computation.

[16]  Mu-Chun Su,et al.  A variant of the SOM algorithm and its interpretation in the viewpoint of social influence and learning , 2009, Neural Computing and Applications.

[17]  Matthew Cook,et al.  Universality in Elementary Cellular Automata , 2004, Complex Syst..

[18]  J. J McDowell,et al.  Beyond continuous mathematics and traditional scientific analysis: Understanding and mining Wolfram's A New Kind of Science , 2009, Behavioural Processes.

[19]  K. Dejong,et al.  An analysis of the behavior of a class of genetic adaptive systems , 1975 .

[20]  T. Kohonen.  Self-organized formation of topographically correct feature maps , 1982 .

[21]  Jorge Nocedal,et al.  Theory of algorithms for unconstrained optimization , 1992, Acta Numerica.

[22]  Gary B. Fogel,et al.  Computational intelligence approaches for pattern discovery in biological systems , 2008, Briefings Bioinform..

[23]  Hasan Merdun,et al.  Self-organizing map artificial neural network application in multidimensional soil data analysis , 2011, Neural Computing and Applications.

[24]  Wei Wu,et al.  Double parallel feedforward neural network based on extreme learning machine with L1/2 regularizer , 2014, Neurocomputing.

[25]  Antony Galton,et al.  Artificial Development of Biologically Plausible Neural-Symbolic Networks , 2013, Cognitive Computation.

[26]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[27]  Jose A. Fernandez-Leon,et al.  How simple autonomous decisions evolve into robust behaviours?: A review from neurorobotics, cognitive, self-organized and artificial immune systems fields , 2014, Biosyst..

[28]  James Kennedy,et al.  The particle swarm: social adaptation of knowledge , 1997, Proceedings of 1997 IEEE International Conference on Evolutionary Computation (ICEC '97).

[29]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[30]  Thomas L. Dean,et al.  Neural Networks and Neuroscience-Inspired Computer Vision , 2014, Current Biology.

[31]  S. Drake,et al.  The controversy on the comets of 1618 , 1960 .

[32]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[33]  P. Caleb,et al.  Self organising maps for the investigation of clinical data: A case study , 2005, Neural Computing & Applications.

[34]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[35]  Young-Seuk Park,et al.  Application of a Kohonen's self‐organizing map for evaluation of long‐term changes in forest vegetation , 2013 .

[36]  M. Powell.  Convergence properties of algorithms for nonlinear optimization , 1986 .

[37]  Guang-Bin Huang,et al.  An Insight into Extreme Learning Machines: Random Neurons, Random Features and Kernels , 2014, Cognitive Computation.

[38]  T. Kohonen.  Analysis of a simple self-organizing process , 1982, Biological Cybernetics.

[39]  L.J. Fogel,et al.  Intelligent decision-making through a simulation of evolution , 1965 .

[40]  A. M. Turing,et al.  Computing Machinery and Intelligence , 1950, The Philosophy of Artificial Intelligence.

[41]  Marco Dorigo,et al.  Ant system: optimization by a colony of cooperating agents , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[42]  Wei Wu,et al.  SOMO-m Optimization Algorithm with Multiple Winners , 2012 .

[43]  Guang-Bin Huang,et al.  Extreme learning machine: a new learning scheme of feedforward neural networks , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).