Cascade network architectures

A novel incremental cascade network architecture based on error minimization is presented. The properties of this and related cascade architectures are discussed, and the influence of the objective function is investigated. The performance of the network is achieved by several layers of nonlinear units that are trained in a strictly feedforward manner and one after the other. Nonlinearity is generated by using sigmoid units and, optionally, additional powers of their activity values. Extensive benchmarking results for the XOR problem are reported, as are various classification tasks, and time series prediction. These are compared to other results reported in the literature. Direct cascading is proposed as promising approach to introducing context information in the approximation process.<<ETX>>

[1]  C. Gielen,et al.  Neural computation and self-organizing maps, an introduction , 1993 .

[2]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[3]  R. Scott Crowder,et al.  Predicting the Mackey-Glass Timeseries With Cascade-Correlation Learning , 1990 .

[4]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[5]  T. Kohonen,et al.  Statistical pattern recognition with neural networks: benchmarking studies , 1988, IEEE 1988 International Conference on Neural Networks.

[6]  Farmer,et al.  Predicting chaotic time series. , 1987, Physical review letters.

[7]  J. Nadal,et al.  Learning in feedforward layered networks: the tiling algorithm , 1989 .

[8]  Helge Ritter,et al.  Learning with the Self-Organizing Map , 1991 .

[9]  Jörg A. Walter,et al.  Nonlinear prediction with self-organizing maps , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[10]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[11]  Stephen M. Omohundro,et al.  Bumptrees for Efficient Function, Constraint and Classification Learning , 1990, NIPS.

[12]  David Haussler,et al.  What Size Net Gives Valid Generalization? , 1989, Neural Computation.

[13]  John A. Hertz,et al.  Exploiting Neurons with Localized Receptive Fields to Learn Chaos , 1990, Complex Syst..

[14]  Yann LeCun,et al.  Optimal Brain Damage , 1989, NIPS.

[15]  L. Glass,et al.  Oscillation and chaos in physiological control systems. , 1977, Science.

[16]  Geoffrey E. Hinton,et al.  Evaluation of Adaptive Mixtures of Competing Experts , 1990, NIPS.

[17]  A. Lapedes,et al.  Nonlinear signal processing using neural networks: Prediction and system modelling , 1987 .

[18]  D. Shepard.  A two-dimensional interpolation function for irregularly-spaced data , 1968, ACM National Conference.

[19]  Jon Louis Bentley,et al.  An Algorithm for Finding Best Matches in Logarithmic Expected Time , 1977, TOMS.

[20]  T Poggio,et al.  Regularization Algorithms for Learning That Are Equivalent to Multilayer Networks , 1990, Science.

[21]  H. D. Block.  The perceptron: a model for brain functioning. I , 1962 .

[22]  John M. Zelle,et al.  Growing layers of perceptrons: introducing the Extentron algorithm , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.