Reducing the number of neurons in radial basis function networks with dynamic decay adjustment

Classification is a common task for supervised neural networks. A specific radial basis function network for classification is the so-called RBF network with dynamic decay adjustment (RBFN-DDA). Fast training and good classification performance are properties of this network. RBFN-DDA is a dynamically growing network, i.e. neurons are inserted during training. A drawback of RBFN-DDA is its greedy insertion behavior. Too many superfluous neurons are inserted for noisy data, overlapping data or for outliers. We propose an online technique to reduce the number of neurons during training. We achieve our goal by deleting neurons after each training of one epoch. By using the improved algorithm on benchmark data and current medical data, the number of neurons is reduced clearly (up to 93.9% less neurons). Thus, we achieve a network with less complexity compared to the original RBFN-DDA.

[1]  D. L. Reilly,et al.  A neural model for category learning , 1982, Biological Cybernetics.

[2]  Rüdiger W. Brause,et al.  A neuro-fuzzy approach as medical diagnostic interface , 2000, ESANN.

[3]  Donald F. Specht,et al.  Probabilistic neural networks , 1990, Neural Networks.

[5]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[6]  Michael R. Berthold,et al.  Boosting the Performance of RBF Networks with Dynamic Decay Adjustment , 1994, NIPS.

[7]  Lutz Prechelt,et al.  PROBEN 1 - a set of benchmarks and benchmarking rules for neural network training algorithms , 1994 .

[8]  Michael R. Berthold,et al.  Constructive training of probabilistic neural networks , 1998, Neurocomputing.

[9]  E. Faist.  Immunological Screening and Immunotherapy in Critically ill Patients with Abdominal Infections , 2001, Springer Berlin Heidelberg.

[10]  Jürgen Paetz,et al.  About the Analysis of Septic Shock Patient Data , 2000, ISMDA.

[11]  Jürgen Paetz,et al.  A Neuro-fuzzy Based Alarm System for Septic Shock Patients with a Comparison to Medical Scores , 2002, ISMDA.

[12]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[13]  Michael R. Berthold.  Fuzzy models and potential outliers , 1999, 18th International Conference of the North American Fuzzy Information Processing Society - NAFIPS (Cat. No.99TH8397).

[14]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[15]  Cao Feng,et al.  STATLOG: COMPARISON OF CLASSIFICATION ALGORITHMS ON LARGE REAL-WORLD PROBLEMS , 1995 .

[16]  E. Hanisch,et al.  Intensive Care Management in Abdominal Surgical Patients with Septic Complications , 2001 .

[17]  A. Fein,et al.  Sepsis and multiorgan failure , 1997 .