Learning Weights in Genetic Programs Using Gradient Descent for Object Recognition

This paper describes an approach to the use of gradient descent search in tree based genetic programming for object recognition problems. A weight parameter is introduced to each link between two nodes in a program tree. The weight is defined as a floating point number and determines the degree of contribution of the sub-program tree under the link with the weight. Changing a weight corresponds to changing the effect of the sub-program tree. The weight changes are learnt by gradient descent search at a particular generation. The programs are evolved and learned by both the genetic beam search and the gradient descent search. This approach is examined and compared with the basic genetic programming approach without gradient descent on three object classification problems of varying difficulty. The results suggest that the new approach works well on these problems.

[1]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[2]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[3]  Olga Štěpánková,et al.  Advanced Topics in Artificial Intelligence , 1992, Lecture Notes in Computer Science.

[4]  Walter Alden Tackett,et al.  Genetic Programming for Feature Discovery and Image Discrimination , 1993, ICGA.

[5]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[6]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[7]  John R. Koza,et al.  Genetic programming 2 - automatic discovery of reusable programs , 1994, Complex adaptive systems.

[8]  J. K. Kinnear,et al.  Advances in Genetic Programming , 1994 .

[9]  David Andre,et al.  Automatically defined features: the simultaneous evolution of 2-dimensional feature detectors and an algorithm for using them , 1994 .

[10]  Bangalore S. Manjunath,et al.  Genetic Programming for Object Detection , 1996 .

[11]  R. Poli,et al.  Discovery of Symbolic, Neuro-Symbolic and Neural Networks with Parallel Distributed Genetic Programming , 1997, ICANNGA.

[12]  Peter Nordin,et al.  Genetic programming - An Introduction: On the Automatic Evolution of Computer Programs and Its Applications , 1998 .

[13]  Daniel Howard,et al.  Target detection in SAR imagery by genetic programming , 1999 .

[14]  Victor Ciesielski,et al.  Genetic Programming for Multiple Class Object Detection , 1999, Australian Joint Conference on Artificial Intelligence.

[15]  Vic Ciesielski,et al.  Representing classification problems in genetic programming , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[16]  Vic Ciesielski,et al.  Texture classifiers generated by genetic programming , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[17]  Victor Ciesielski,et al.  A Domain-Independent Window Approach to Multiclass Object Detection Using Genetic Programming , 2003, EURASIP J. Adv. Signal Process..

[18]  Conor Ryan,et al.  An Analysis of Diversity of Constants of Genetic Programming , 2003, EuroGP.

[19]  Mengjie Zhang,et al.  Genetic Programming with Gradient Descent Search for Multiclass Object Classification , 2004, EuroGP.