Gaussian kernel approximation algorithm for feedforward neural network design

A Gaussian kernel approximation algorithm for a feedforward neural network is presented. The approach used by the algorithm, which is based on a constructive learning algorithm, is to create the hidden units directly so that automatic design of the architecture of neural networks can be carried out. The algorithm is defined using the linear summation of input patterns and their randomized input weights. Hidden-layer nodes are defined so as to partition the input space into homogeneous regions, where each region contains patterns belonging to the same class. The largest region is used to define the center of the corresponding Gaussian hidden nodes. The algorithm is tested on three benchmark data sets of different dimensionality and sample sizes to compare the approach presented here with other algorithms. Real medical diagnoses and a biological classification of mushrooms are used to illustrate the performance of the algorithm. These results confirm the effectiveness of the proposed algorithm.

[1]  Jason Catlett,et al.  Experiments on the Costs and Benefits of Windowing in ID3 , 1988, ML.

[2]  H. Akaike.  A new look at the statistical model identification , 1974 .

[3]  Gunnar Rätsch,et al.  An Improvement of AdaBoost to Avoid Overfitting , 1998, ICONIP.

[4]  Jihoon Yang,et al.  Constructive Neural-Network Learning Algorithms for Pattern Classification , 2000 .

[5]  Jihoon Yang,et al.  DistAl: an inter-pattern distance-based constructive learning algorithm , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[6]  Tony R. Martinez,et al.  Heterogeneous radial basis function networks , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[7]  Pat Langley,et al.  Trading Off Simplicity and Coverage in Incremental concept Learning , 1988, ML.

[8]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[9]  Thomas F. Coleman,et al.  Large-Scale Numerical Optimization , 1990 .

[10]  L. L. Cam,et al.  The Central Limit Theorem Around 1935 , 1986 .

[11]  J. Ross Quinlan,et al.  Improved Use of Continuous Attributes in C4.5 , 1996, J. Artif. Intell. Res..

[12]  K. W. Lee,et al.  Optimal sizing of feedforward neural networks: Case studies , 1995, Proceedings 1995 Second New Zealand International Two-Stream Conference on Artificial Neural Networks and Expert Systems.

[13]  Osamu Fujita,et al.  Statistical estimation of the number of hidden units for feedforward neural networks , 1998, Neural Networks.

[14]  Kemal Polat,et al.  A New Classification Method for Breast Cancer Diagnosis: Feature Selection Artificial Immune Recognition System (FS-AIRS) , 2005, ICNC.

[15]  O. Mangasarian,et al.  Pattern Recognition Via Linear Programming: Theory and Application to Medical Diagnosis , 1989 .

[16]  J. C. Schlimmer,et al.  Concept acquisition through representational adjustment , 1987 .

[17]  John E. Moody,et al.  The Effective Number of Parameters: An Analysis of Generalization and Regularization in Nonlinear Learning Systems , 1991, NIPS.

[18]  C. Jutten,et al.  Gal: Networks That Grow When They Learn and Shrink When They Forget , 1991 .

[19]  Rudolf Kruse,et al.  Obtaining interpretable fuzzy classification rules from medical data , 1999, Artif. Intell. Medicine.

[20]  Separable Regions.  On Hidden Nodes for Neural Nets , 1989 .

[21]  K. S. Banerjee.  Generalized Inverse of Matrices and Its Applications , 1973 .

[22]  C. R. Rao,et al.  Generalized Inverse of Matrices and its Applications , 1972 .

[23]  Lois Boggess,et al.  ARTIFICIAL IMMUNE SYSTEM CLASSIFICATION OF MULTIPLE- CLASS PROBLEMS , 2002 .

[24]  Lutz Prechelt,et al.  PROBEN 1 - a set of benchmarks and benchmarking rules for neural network training algorithms , 1994 .

[25]  F. L. Xiong,et al.  A method for estimating the number of hidden neurons in feed-forward neural networks based on information entropy , 2003 .

[26]  O. Mangasarian,et al.  Multisurface method of pattern separation for medical diagnosis applied to breast cytology. , 1990, Proceedings of the National Academy of Sciences of the United States of America.