Iterative fast orthogonal search algorithm for sparse self-structuring generalized single-layer networks

The generalized single-layer network (GSLN) architecture, which implements a sum of arbitrary basis functions defined on its inputs, is potentially a flexible and efficient structure for approximating arbitrary nonlinear functions. A drawback of GSLNs is that a large number of weights and basis functions may be required to provide satisfactory approximations. In this paper, we present a new approach in which an algorithm known as iterative fast orthogonal search (IFOS) is coupled with the minimum description length (MDL) criterion to provide automatic structure selection and parameter estimation for GSLNs. The resulting algorithm, dubbed IFOS with dynamic model resizing (IFOS-DMR) performs both network growth and pruning to construct sparse GSLNs from potentially large spaces of candidate basis functions.

[1]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[2]  Ilan Ziskind,et al.  Maximum likelihood localization of multiple sources by alternating projection , 1988, IEEE Trans. Acoust. Speech Signal Process..

[3]  S. Chen,et al.  Fast orthogonal least squares algorithm for efficient subset model selection , 1995, IEEE Trans. Signal Process..

[4]  Rosalind W. Picard,et al.  On the efficiency of the orthogonal least squares training method for radial basis function networks , 1996, IEEE Trans. Neural Networks.

[5]  M. Korenberg,et al.  Orthogonal approaches to time-series analysis and system identification , 1991, IEEE Signal Processing Magazine.

[6]  Peter J. W. Rayner,et al.  Generalization and PAC learning: some new results for the class of generalized single-layer networks , 1995, IEEE Trans. Neural Networks.

[7]  Alan A. Desrochers.  On an improved model reduction technique for nonlinear systems , 1981, Autom..

[8]  Jorma Rissanen,et al.  Universal coding, information, prediction, and estimation , 1984, IEEE Trans. Inf. Theory.

[9]  Robin D. Morris,et al.  Fast probabilistic self-structuring of generalized single-layer networks , 1996, IEEE Trans. Neural Networks.

[10]  Manoel Fernando Tenorio,et al.  Self-organizing network for optimum supervised learning , 1990, IEEE Trans. Neural Networks.

[11]  M. Korenberg,et al.  Fast orthogonal search for array processing and spectrum estimation , 1994 .

[12]  A. Desrochers,et al.  On determining the structure of a non-linear system , 1984 .

[13]  Monique Frize,et al.  Prediction of ventilation requirements in an intensive care unit , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).