Convergent design of a piecewise linear neural network

A piecewise linear neural network (PLNN) is discussed which maps N-dimensional input vectors into M-dimensional output vectors. A convergent algorithm for designing the PLNN from training data is described The design algorithm is based on a variation of backtracking algorithm known as the 'branch-and-bound' method. The performance of the PLNN is compared with that of a multilayer perceptron (MLP) of equivalent size. The results show that the PLNN is capable of performing as well as an equivalent MLP.

[1]  Michael T. Manry,et al.  Output weight optimization for the multi-layer perceptron , 1992, [1992] Conference Record of the Twenty-Sixth Asilomar Conference on Signals, Systems & Computers.

[2]  Michael T. Manry,et al.  A neural network training algorithm utilizing multiple sets of linear equations , 1996, Conference Record of The Thirtieth Asilomar Conference on Signals, Systems and Computers.

[3]  Kamal Sarabandi,et al.  An empirical model and an inversion technique for radar scattering from bare soil surfaces , 1992, IEEE Trans. Geosci. Remote. Sens..

[4]  Rolf Unbehauen,et al.  Canonical representation of piecewise-polynomial functions with nondegenerate linear-domain partitions , 1998 .

[5]  Michael T. Manry,et al.  Comparison of very short-term load forecasting techniques , 1996 .

[6]  Carlos S. Kubrusly,et al.  Stochastic approximation algorithms and applications , 1973, CDC 1973.

[7]  A. Lapedes,et al.  Nonlinear Signal Processing Using Neural Networks , 1987 .

[8]  Keinosuke Fukunaga,et al.  A Branch and Bound Clustering Algorithm , 1975, IEEE Transactions on Computers.

[9]  A. Lapedes,et al.  Nonlinear signal processing using neural networks: Prediction and system modelling , 1987 .

[10]  James S. Albus,et al.  New Approach to Manipulator Control: The Cerebellar Model Articulation Controller (CMAC)1 , 1975 .

[11]  Adrian K. Fung,et al.  Backscattering from a randomly rough dielectric surface , 1992, IEEE Trans. Geosci. Remote. Sens..

[12]  M.T. Manry,et al.  Modular neural network architecture using piece-wise linear mapping , 1996, Conference Record of The Thirtieth Asilomar Conference on Signals, Systems and Computers.

[13]  Ernest S. Kuh,et al.  Solving nonlinear resistive networks using piecewise-linear analysis and simplicial subdivision , 1977 .

[14]  Jenq-Neng Hwang,et al.  Regression modeling in back-propagation and projection pursuit learning , 1994, IEEE Trans. Neural Networks.

[15]  Rolf Unbehauen,et al.  Canonical piecewise-linear networks , 1995, IEEE Trans. Neural Networks.

[16]  B.W. Dickinson,et al.  An introduction to statistical signal processing with applications , 1979, Proceedings of the IEEE.

[17]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[18]  H. Akaike.  A new look at the statistical model identification , 1974 .

[19]  Robert F. Ling,et al.  Cluster analysis algorithms for data reduction and classification of objects , 1981 .

[20]  Keinosuke Fukunaga,et al.  A Branch and Bound Algorithm for Feature Subset Selection , 1977, IEEE Transactions on Computers.

[21]  J. Freidman,et al.  Multivariate adaptive regression splines , 1991 .

[22]  David B. Fogel.  An information criterion for optimal neural network selection , 1991, IEEE Trans. Neural Networks.

[23]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[24]  Richard M. Stanley.  Optimal Estimation With an Introduction to Stochastic Control , 1988 .

[25]  Shinichi Morishita,et al.  On Classification and Regression , 1998, Discovery Science.

[26]  J.C. Principe,et al.  Non-linear time series modeling with self-organization feature maps , 1995, Proceedings of 1995 IEEE Workshop on Neural Networks for Signal Processing.

[27]  Ramanathan Gnanadesikan.  Methods for Statistical Data Analysis of Multivariate Observations: Gnanadesikan/Methods , 1997 .

[28]  Ramanathan Gnanadesikan,et al.  Methods for statistical data analysis of multivariate observations , 1977, A Wiley publication in applied statistics.

[29]  James S. Albus,et al.  Data Storage in the Cerebellar Model Articulation Controller (CMAC) , 1975 .

[30]  Edoardo Amaldi,et al.  A Perceptron-Based Approach to Piecewise Linear Modeling with an Application to Time Series , 1997, ICANN.

[31]  F. Lewis.  Optimal Estimation: With an Introduction to Stochastic Control Theory , 1986 .