Real-coded crossover as a role of kernel density estimation

This paper presents a kernel density estimation method by means of real-coded crossovers. Estimation of density algorithms (EDAs) are evolutionary optimization techniques, which determine the sampling strategy by means of a parametric probabilistic density function estimated from the population. Real-coded Genetic Algorithm (RCGA) does not explicitly estimate any probabilistic distribution, however, the probabilistic model of the population is implicitly estimated by crossovers and the sampling strategy is determined by this implicit probabilistic model. Based on this understanding, we propose a novel density estimation algorithm by using crossovers as nonparametric kernels and apply this kernel density estimation to the Gaussian Mixture modeling. We show that the proposed method is superior in the robustness of the computation and in the accuracy of the estimation by the comparison of conventional EM estimation.

[1]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[2]  J. D. Schaffer,et al.  Real-Coded Genetic Algorithms and Interval-Schemata , 1992, FOGA.

[3]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .

[4]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[5]  Isao Ono,et al.  A Real Coded Genetic Algorithm for Function Optimization Using Unimodal Normal Distributed Crossover , 1997, ICGA.

[6]  Masayuki Yamamura,et al.  Theoretical Analysis of Simplex Crossover for Real-Coded Genetic Algorithms , 2000, PPSN.

[7]  Byoung-Tak Zhang,et al.  Convergence properties of incremental Bayesian evolutionary algorithms with single Markov chains , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[8]  Kalyanmoy Deb,et al.  On the Desired Behaviors of Self-Adaptive Evolutionary Algorithms , 2000, PPSN.

[9]  Geoffrey E. Hinton,et al.  SMEM Algorithm for Mixture Models , 1998, Neural Computation.

[10]  Shigenobu Kobayashi,et al.  Multi-Parental Extension of the Unimodal Normal Distribution Crossover for Real-Coded Genetic Algorithms , 2000 .

[11]  Dong-Yeon Cho,et al.  Continuous estimation of distribution algorithms with probabilistic principal component analysis , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[12]  Shigenobu Kobayashi,et al.  Extrapolation-directed crossover for real-coded GA: overcoming deceptive phenomena by extrapolative search , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[13]  J. Sakuma,et al.  Non-parametric expectation-maximization for Gaussian mixtures , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[14]  David E. Goldberg,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2002, Comput. Optim. Appl..

[15]  Ben J. A. Kröse,et al.  Efficient Greedy Learning of Gaussian Mixture Models , 2003, Neural Computation.