A fast and robust learning algorithm for feedforward neural networks

The back propagation algorithm caused a tremendous breakthrough in the application of multilayer perceptrons. However, it has some important drawbacks: long training times and sensitivity to the presence of local minima. Another problem is the network topology; the exact number of units in a particular hidden layer, as well as the number of hidden layers need to be known in advance. A lot of time is often spent in finding the optimal topology. In this article, we consider multilayer networks with one hidden layer of Gaussian units and an output layer of conventional units. We show that for this kind of networks, it is possible to perform a fast dimensionality analysis, by analyzing only a small fraction of the input patterns. Moreover, as a result of this approach, it is possible to initialize the weights of the network before starting the back propagation training. Several classification problems are taken as examples.

[1]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory, Third Edition , 1989, Springer Series in Information Sciences.

[2]  Alex Waibel,et al.  Phoneme recognition: neural networks vs. hidden Markov models vs. hidden Markov models , 1988, ICASSP-88., International Conference on Acoustics, Speech, and Signal Processing.

[3]  P. Vandamme,et al.  Identification of EF group 22 campylobacters from gastroenteritis cases as Campylobacter concisus , 1989, Journal of clinical microbiology.

[4]  Mikko Kokkonen,et al.  Using self-organizing maps and multi-layered feed-forward nets to obtain phonemic transcriptions of spoken utterances , 1990, Speech Commun..

[5]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[6]  S. Renals,et al.  Phoneme classification experiments using radial basis functions , 1989, International 1989 Joint Conference on Neural Networks.

[7]  Alex Waibel,et al.  Phoneme Recognition: Neural Networks vs , 1988 .

[8]  Jean-Pierre Martens,et al.  Broad phonetic classification and segmentation of continuous speech by means of neural networks and dynamic programming , 1991, Speech Commun..

[9]  Lawrence R. Rabiner,et al.  A modified K-means clustering algorithm for use in isolated work recognition , 1985, IEEE Trans. Acoust. Speech Signal Process..

[10]  Stan Davis,et al.  Comparison of Parametric Representations for Monosyllabic Word Recognition in Continuously Spoken Se , 1980 .

[11]  Richard P. Lippmann,et al.  An introduction to computing with neural nets , 1987 .

[12]  Jean Swings,et al.  Taxonomic Analysis of Xanthomonas campestris pv. begoniae and X. campestris pv. pelargonii by Means of Phytopathological, Phenotypic, Protein Electrophoretic and DNA Hybridization Methods , 1990 .

[13]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .