Applications of multi-objective structure optimization

We present applications of multi-objective evolutionary optimization of feed-forward neural networks (NN) to two real world problems, car and face classification. The possibly conflicting requirements on the NNs are speed and classification accuracy, both of which can enhance the embedding systems as a whole. We compare the results to the outcome of a greedy optimization heuristic (magnitude-based pruning) coupled with a multi-objective performance evaluation. For the car classification problem, magnitude-based pruning yields competitive results, whereas for the more difficult face classification, we find that the evolutionary approach to NN design is clearly preferable.

[1]  Bernhard Sendhoff,et al.  Neural network regularization and ensembling using multi-objective evolutionary algorithms , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[2]  B. Michaelis,et al.  Real-time vehicle and lane detection with embedded hardware , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[3]  Lawrence Davis,et al.  Adapting Operator Probabilities in Genetic Algorithms , 1989, ICGA.

[4]  L. Darrell Whitley,et al.  The GENITOR Algorithm and Selection Pressure: Why Rank-Based Allocation of Reproductive Trials is Best , 1989, ICGA.

[5]  J. David Schaffer,et al.  Proceedings of the third international conference on Genetic algorithms , 1989 .

[6]  Erik Hjelmås,et al.  Face Detection: A Survey , 2001, Comput. Vis. Image Underst..

[7]  Joachim M. Buhmann,et al.  Distortion Invariant Object Recognition in the Dynamic Link Architecture , 1993, IEEE Trans. Computers.

[8]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[9]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Marco Laumanns,et al.  Performance assessment of multiobjective optimizers: an analysis and review , 2003, IEEE Trans. Evol. Comput..

[11]  D. Parisi,et al.  Evolution and learning in neural networks , 2002 .

[12]  Ernst D. Dickmanns,et al.  Recursive 3-D Road and Relative Ego-State Recognition , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Héctor Pomares,et al.  Multiobjective evolutionary optimization of the size, shape, and position parameters of radial basis function networks for function approximation , 2003, IEEE Trans. Neural Networks.

[14]  Zbigniew Michalewicz,et al.  Parameter control in evolutionary algorithms , 1999, IEEE Trans. Evol. Comput..

[15]  David B. Fogel,et al.  Evolutionary Computation: Towards a New Philosophy of Machine Intelligence , 1995 .

[16]  Werner von Seelen,et al.  Image processing and behavior planning for intelligent vehicles , 2003, IEEE Trans. Ind. Electron..

[17]  Bernhard Sendhoff,et al.  Synergies between Evolutionary and Neural Computation , 2005, ESANN.

[18]  Peter L. Bartlett,et al.  The Sample Complexity of Pattern Classification with Neural Networks: The Size of the Weights is More Important than the Size of the Network , 1998, IEEE Trans. Inf. Theory.

[19]  Christian Igel,et al.  Operator adaptation in evolutionary computation and its application to structure optimization of neural networks , 2003, Neurocomputing.

[20]  Jonathan E. Fieldsend,et al.  Pareto evolutionary neural networks , 2005, IEEE Transactions on Neural Networks.

[21]  Allen Allport,et al.  Visual attention , 1989 .

[22]  Hussein A. Abbass,et al.  Speeding Up Backpropagation Using Multiobjective Evolutionary Algorithms , 2003, Neural Computation.

[23]  David E. Rumelhart,et al.  BACK-PROPAGATION, WEIGHT-ELIMINATION AND TIME SERIES PREDICTION , 1991 .

[24]  Xin Yao,et al.  Evolutionary framework for the construction of diverse hybrid ensembles , 2005, ESANN.

[25]  Martin A. Riedmiller,et al.  Fast Network Pruning and Feature Extraction by using the Unit-OBS Algorithm , 1996, NIPS.

[26]  César Hervás-Martínez,et al.  Multi-objective cooperative coevolution of artificial neural networks (multi-objective cooperative networks) , 2002, Neural Networks.

[27]  Lothar Thiele,et al.  A Tutorial on the Performance Assessment of Stochastic Multiobjective Optimizers , 2006 .

[28]  A. Shashua,et al.  Pedestrian detection for driving assistance systems: single-frame classification and system level performance , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[29]  Jim Smith,et al.  Operator and parameter adaptation in genetic algorithms , 1997, Soft Comput..

[30]  Christian Igel,et al.  Evolutionary Multi-Objective Optimisation Of Neural Networks For Face Detection , 2004, Int. J. Comput. Intell. Appl..

[31]  Christian Igel,et al.  Evolutionary Optimization of Neural Networks for Face Detection , 2004, ESANN.

[32]  Ehud D. Karnin,et al.  A simple procedure for pruning back-propagation trained neural networks , 1990, IEEE Trans. Neural Networks.

[33]  Narendra Ahuja,et al.  Detecting Faces in Images: A Survey , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Visvanathan Ramesh,et al.  A system for traffic sign detection, tracking, and recognition using color, shape, and motion information , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[35]  Michael A. Arbib,et al.  The handbook of brain theory and neural networks , 1995, A Bradford book.

[36]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[37]  Rich Caruana,et al.  Overfitting in Neural Nets: Backpropagation, Conjugate Gradient, and Early Stopping , 2000, NIPS.

[38]  T. Bucher,et al.  Real-time detection and classification of cars in video sequences , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[39]  Martin A. Riedmiller,et al.  Advanced supervised learning in multi-layer perceptrons — From backpropagation to adaptive learning algorithms , 1994 .

[40]  Christian Igel,et al.  Empirical evaluation of the improved Rprop learning algorithms , 2003, Neurocomputing.

[41]  Christian Wöhler,et al.  Real-time object recognition on image sequences with the adaptable time delay neural network algorithm - applications for autonomous vehicles , 2001, Image Vis. Comput..

[42]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[43]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[44]  Guoqiang Peter Zhang,et al.  Neural networks for classification: a survey , 2000, IEEE Trans. Syst. Man Cybern. Part C.

[45]  Bernhard Sendhoff,et al.  A critical survey of performance indices for multi-objective optimisation , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..