Conjugate gradient and approximate Newton methods for an optimal probabilistic neural network for food color classification

The probabilistic neural network (PNN) is based on the esti- mation of the probability density functions. The estimation of these den- sity functions uses smoothing parameters that represent the width of the activation functions. A two-step numerical procedure is developed for the optimization of the smoothing parameters of the PNN: a rough optimiza- tion by the conjugate gradient method and a fine optimization by the approximate Newton method. The thrust is to compare the classification performances of the improved PNN and the standard back-propagation neural network (BPNN). Comparisons are performed on a food quality problem: french fry classification into three different color classes (light, normal, and dark). The optimized PNN correctly classifies 96.19% of the test data, whereas the BPNN classifies only 93.27% of the same data. Moreover, the PNN is more stable than the BPNN with regard to the random initialization. The optimized PNN requires 1464 s for training compared to only 71 s required by the BPNN. © 1998 Society of Photo-Optical Instrumentation Engineers. (S0091-3286(98)01711-5)

[1]  Mohamad T. Musavi,et al.  On the Generalization Ability of Neural Network Classifiers , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  William H. Press,et al.  Numerical Recipes in C, 2nd Edition , 1992 .

[3]  D. F. Specht,et al.  Generalization accuracy of probabilistic neural networks compared with backpropagation networks , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[4]  Robert A. Jacobs,et al.  Increased rates of convergence through learning rate adaptation , 1987, Neural Networks.

[5]  Elijah Polak,et al.  Computational methods in optimization , 1971 .

[6]  Dominique Bertrand,et al.  Reduction of the size of the learning data in a probabilistic neural network by hierarchical clustering. Application to the discrimination of seeds by artificial vision , 1996 .

[7]  Rama Chellappa,et al.  Evaluation of pattern classifiers for fingerprint and OCR applications , 1994, Pattern Recognit..

[8]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[9]  D. Broomhead,et al.  Radial Basis Functions, Multi-Variable Functional Interpolation and Adaptive Networks , 1988 .

[10]  Abhijit S. Pandya,et al.  Pattern Recognition with Neural Networks in C++ , 1995 .

[11]  Roy L. Streit,et al.  Maximum likelihood training of probabilistic neural networks , 1994, IEEE Trans. Neural Networks.

[12]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[13]  L. P. J. Veelenturf,et al.  Analysis and applications of artificial neural networks , 1995 .

[14]  Timothy Masters,et al.  Practical neural network recipes in C , 1993 .

[15]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[16]  Bernard Widrow,et al.  30 years of adaptive neural networks: perceptron, Madaline, and backpropagation , 1990, Proc. IEEE.

[17]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[18]  David J. Montana,et al.  A Weighted Probabilistic Neural Network , 1991, NIPS.

[19]  Mohamad T. Musavi,et al.  A minimum error neural network (MNN) , 1993, Neural Networks.

[20]  D. W. Scott,et al.  Multivariate Density Estimation, Theory, Practice and Visualization , 1992 .

[21]  Donald F. Specht,et al.  Probabilistic neural networks , 1990, Neural Networks.

[22]  Gunther Wyszecki,et al.  Color Science: Concepts and Methods, Quantitative Data and Formulae, 2nd Edition , 2000 .

[23]  Suranjan Panigrahi,et al.  Computer-based neuro-vision system for color classification of french fries , 1995, Other Conferences.

[24]  David W. Scott,et al.  Multivariate Density Estimation: Theory, Practice, and Visualization , 1992, Wiley Series in Probability and Statistics.