A comparative study on kernel smoothers in Differential Evolution with estimated comparison method for reducing function evaluations

As a new research topic for reducing the number of function evaluations effectively in function optimization, an idea of utilizing a rough approximation model, which is an approximation model with low accuracy and without learning process, has been proposed. Although the approximation errors between true function values and their approximation values estimated by the rough approximation model are not small, the rough model can estimate the order relation of two points with fair accuracy. In order to use this feature of the rough model, we have proposed the estimated comparison method, which omits the function evaluations when the result of comparison can be judged by approximation values. In this study, kernel smoothers are adopted as rough approximation models. Various types of benchmark functions are solved by Differential Evolution (DE) with the estimated comparison method and the results are compared with those obtained by DE. It is shown that the estimated comparison method is general purpose method for reducing function evaluations and can work well with kernel smoothers. It is also shown that the potential model, which is a rough approximation model proposed by us, has better ability of function reduction than kernel smoothers.

[1]  Tetsuyuki Takahama,et al.  Solving Nonlinear Constrained Optimization Problems by the ε Constrained Differential Evolution , 2006, 2006 IEEE International Conference on Systems, Man and Cybernetics.

[2]  Frederico G. Guimarães,et al.  Local Learning and Search in Memetic Algorithms , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[3]  Bernhard Sendhoff,et al.  Reducing Fitness Evaluations Using Clustering Techniques and Neural Network Ensembles , 2004, GECCO.

[4]  Tetsuyuki Takahama,et al.  Reducing function evaluations in Differential Evolution using rough approximation-based comparison , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[5]  Kilian Q. Weinberger,et al.  Convex Optimizations for Distance Metric Learning and Pattern Classification [Applications Corner] , 2010, IEEE Signal Processing Magazine.

[6]  Rainer Storn,et al.  Minimizing the real functions of the ICEC'96 contest by differential evolution , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[7]  E. Nadaraya.  On Estimating Regression , 1964 .

[8]  Bernhard Sendhoff,et al.  On Evolutionary Optimization with Approximate Fitness Functions , 2000, GECCO.

[9]  Tetsuyuki Takahama,et al.  Constrained Optimization by the ε Constrained Differential Evolution with Gradient-Based Mutation and Feasible Elites , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[10]  Kilian Q. Weinberger,et al.  Metric Learning for Kernel Regression , 2007, AISTATS.

[11]  Yew-Soon Ong,et al.  Curse and Blessing of Uncertainty in Evolutionary Algorithm Using Approximation , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[12]  Petros Koumoutsakos,et al.  Accelerating evolutionary algorithms with Gaussian process fitness function models , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[13]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[14]  Matthew P. Wand,et al.  Kernel Smoothing , 1995 .

[15]  G. S. Watson,et al.  Smooth regression analysis , 1964 .

[16]  T. Takahama,et al.  Efficient Optimization by Differential Evolution using Rough Approximation Model with Adaptive Control of Error Margin , 2008 .

[17]  Uday K. Chakraborty,et al.  Advances in Differential Evolution , 2010 .

[18]  T. W. Layne,et al.  A Comparison of Approximation Modeling Techniques: Polynomial Versus Interpolating Models , 1998 .

[19]  James W. Wisnowski,et al.  Smoothing and Regression: Approaches, Computation, and Application , 2002 .

[20]  Yaochu Jin,et al.  A comprehensive survey of fitness approximation in evolutionary computation , 2005, Soft Comput..

[21]  Timothy W. Simpson,et al.  Comparison of Response Surface and Kriging Models in the Multidisciplinary Design of an Aerospike Nozzle , 1998 .

[22]  Bernhard Sendhoff,et al.  A framework for evolutionary optimization with approximate fitness functions , 2002, IEEE Trans. Evol. Comput..

[23]  Timothy M. Mauery,et al.  COMPARISON OF RESPONSE SURFACE AND KRIGING MODELS FOR MULTIDISCIPLINARY DESIGN OPTIMIZATION , 1998 .

[24]  T. Takahama,et al.  Reducing the Number of Function Evaluations in Differential Evolution by Estimated Comparison Method Using an Approximation Model with Low Accuracy , 2008 .

[25]  Wei Shyy,et al.  Response surface and neural network techniques for rocket engine injector optimization , 1999 .