A Novel Optimization Algorithm Based on Reinforcement Learning

In this chapter, an efficient optimization algorithm is presented for the problems with hard to evaluate objective functions. It uses the reinforcement learning principle to determine the particle move in search for the optimum process. A model of successful actions is build and future actions are based on past experience. The step increment combines exploitation of the known search path and exploration for the improved search direction. The algorithm does not require any prior knowledge of the objective function, nor does it require any characteristics of such function. It is simple, intuitive and easy to implement and tune. The optimization algorithm was tested using several multi-variable functions and compared with other widely used random search optimization algorithms. Furthermore, the training of a multi-layer perceptron, to find a set of optimized weights, is treated as an optimization problem. The optimized multi-layer perceptron was applied to Iris database classification. Finally, the algorithm is used in image recognition to find a familiar object with retina sampling and micro-saccades.

[1]  H. Zimmermann.  Towards global optimization 2: L.C.W. DIXON and G.P. SZEGÖ (eds.) North-Holland, Amsterdam, 1978, viii + 364 pages, US $ 44.50, Dfl. 100,-. , 1979 .

[2]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[3]  Hyun-Chul Kim,et al.  Support Vector Machine Ensemble with Bagging , 2002, SVM.

[4]  K. A. Robinson.  Dictionary of Eye Terminology , 1997 .

[5]  N. Salvatore,et al.  A Surrogate Assisted Hooke-Jeeves Algorithm to Optimize the Control System of a PMSM Drive , 2006, 2006 IEEE International Symposium on Industrial Electronics.

[6]  I. Lustig,et al.  Computational experience with a primal-dual interior point method for linear programming , 1991 .

[7]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[8]  Yue Li.  Active Vision through Invariant Representations and Saccade Movements , 2006 .

[9]  Derek L. G. Hill,et al.  Registration Methodology: Concepts and Algorithms , 2001 .

[10]  V. Torczon,et al.  Direct Search Methods , 2011 .

[11]  Joseph V. Hajnal,et al.  Registration Methodology: Concepts and Algorithms , 2001 .

[12]  D. Hubel,et al.  The role of fixational eye movements in visual perception , 2004, Nature Reviews Neuroscience.

[13]  J. Starzyk.  Motivation in Embodied Intelligence , 2008 .

[14]  Jan A Snyman,et al.  Practical Mathematical Optimization: An Introduction to Basic Optimization Theory and Classical and New Gradient-Based Algorithms , 2005 .

[15]  Kai-Yew Lum,et al.  Max-min surrogate-assisted evolutionary algorithm for robust design , 2006, IEEE Transactions on Evolutionary Computation.

[16]  O. SIAMJ.,et al.  ON THE CONVERGENCE OF PATTERN SEARCH ALGORITHMS , 1997 .

[17]  Zhen Zhu,et al.  Optimized Approximation Algorithm in Neural Networks Without Overfitting , 2008, IEEE Transactions on Neural Networks.

[18]  Alessandro Verri,et al.  Pattern Recognition with Support Vector Machines , 2002, Lecture Notes in Computer Science.

[19]  Tjalling J. Ypma,et al.  Historical Development of the Newton-Raphson Method , 1995, SIAM Rev..

[20]  V. Torczon,et al.  Direct search methods: then and now , 2000 .

[21]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[22]  B. V. Sheela.  An optimized step-size random search (OSSRS) , 1979 .

[23]  Robert E. Skelton,et al.  Solving matrix inequalities whose unknowns are matrices , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[24]  Ge Hongwei.  New hybrid particle swarm optimization , 2011 .

[25]  M. J. D. Powell,et al.  An efficient method for finding the minimum of a function of several variables without calculating derivatives , 1964, Comput. J..

[26]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[27]  D. Hill,et al.  Medical image registration , 2001, Physics in medicine and biology.

[28]  Robert Hooke,et al.  `` Direct Search'' Solution of Numerical and Statistical Problems , 1961, JACM.

[29]  H. H. Rosenbrock,et al.  An Automatic Method for Finding the Greatest or Least Value of a Function , 1960, Comput. J..