Dynamic Programming Algorithm for Training Functional Networks

The paper proposes a dynamic programming algorithm for training of functional networks. The algorithm considers each node as a state. The problem is formulated as finding the sequence of states which minimizes the sum of the squared errors approximation. Each node is optimized with regard to its corresponding neural functions and its estimated neuron functions. The dynamic programming algorithm tries to find the best path from the final layer nodes to the input layer which minimizes an optimization criterion. Finally, in the pruning stage, the unused nodes are deleted. The output layer can be taken as a summation node using some linearly independent families, such as, polynomial, exponential, Fourier,...etc. The algorithm is demonstrated by two examples and compared with other common algorithms in both computer science and statistics communities.

[1]  E. C. Ron,et al.  Functional equations and modelling in science and engineering , 1992 .

[2]  Robert A. Jacobs,et al.  Increased rates of convergence through learning rate adaptation , 1987, Neural Networks.

[3]  David G. Stork,et al.  Pattern Classification , 1973 .

[4]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[5]  Scott E. Fahlman,et al.  An empirical study of learning speed in back-propagation networks , 1988 .

[6]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[7]  Alexander G. Parlos,et al.  Nonlinear dynamic system identification using artificial neural networks (ANNs) , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[8]  Kurt Hornik,et al.  FEED FORWARD NETWORKS ARE UNIVERSAL APPROXIMATORS , 1989 .

[9]  Hitoshi Iba,et al.  Learning polynomial feedforward neural networks by genetic programming and backpropagation , 2003, IEEE Trans. Neural Networks.

[10]  José Manuel Gutiérrez,et al.  Introduction to Functional Networks , 1999 .

[11]  D. E. Rumelhart,et al.  Learning internal representations by back-propagating errors , 1986 .

[12]  Y. S. Tarng,et al.  An abductive network for predicting tool life in drilling , 1999 .

[13]  Ali S. Hadi,et al.  Iterative Least Squares Functional Networks Classifier , 2007, IEEE Transactions on Neural Networks.

[14]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .