Learning for Reducing the Configuration eedforward Neural Network

In this paper, we present two learning mechanisms for artificial neural networks (ANN'S) that can be applied to solve classification problems with binary outputs. These mechanisms are used to reduce the number of hidden units of an ANN when trained by the cascade-correlation learning algorithm (CAS). Since CAS adds hidden units incrementally as learning proceeds, it is difficult to predict the number of hidden units required when convergence is reached. Further, learning must be restarted when the number of hidden units is larger than expected. Our key idea in this paper is to provide alternatives in the learning process and to select the best alternative dynamically based on rnn- time information obtained. Mixed-mode learning (MM), our first algorithm, provides alternative output matrices so that learning is extended to find one of the many one-to-many mappings instead of finding a unique one-to-one mapping. Since the objective of learning is relaxed by this transformation, the number of learning epochs can be reduced. This in turn leads to a smaller number of hidden units required for convergence. Population-based learning for ANN'S (PLAN), our second algorithm, maintains alternative network configurations to select at run time promising networks to train based on error information obtained and time remaining. This dynamic scheduling avoids training possibly unpromising ANN'S to completion before exploring new ones. We show the performance of these two mechanisms by applying them to solve the two-spiral problem, a two-region classification problem, and the Pima Indian Diabetes Diagnosis problem.

[1]  Nazif Tepedelenlioglu,et al.  A fast new algorithm for training feedforward neural networks , 1992, IEEE Trans. Signal Process..

[2]  Michael C. Mozer,et al.  Using Relevance to Reduce Network Size Automatically , 1989 .

[3]  S. Ergezinger,et al.  An accelerated learning algorithm for multilayer perceptrons: optimization layer by layer , 1995, IEEE Trans. Neural Networks.

[4]  Kenneth Steiglitz,et al.  Combinatorial Optimization: Algorithms and Complexity , 1981 .

[5]  Alberto Del Bimbo,et al.  Block-structured recurrent neural networks , 1995, Neural Networks.

[6]  Jean-Pierre Martens,et al.  On the initialization and optimization of multilayer perceptrons , 1994, IEEE Trans. Neural Networks.

[7]  H. S. M. Beigi,et al.  Learning algorithms for neural networks based on Quasi-Newton methods with self-scaling , 1993 .

[8]  Albert Y. Zomaya,et al.  Toward generating neural network structures for function approximation , 1994, Neural Networks.

[9]  Shixin Cheng,et al.  Dynamic learning rate optimization of the backpropagation algorithm , 1995, IEEE Trans. Neural Networks.

[10]  Joachim Diederich,et al.  Connectionist Recruitment Learning , 1988, ECAI.

[11]  Benjamin W. Wah,et al.  Population-Based Learning: A Method for Learning from Examples Under Resource Constraints , 1992, IEEE Trans. Knowl. Data Eng..

[12]  Malcolm I. Heywood,et al.  A framework for improved training of Sigma-Pi networks , 1995, IEEE Trans. Neural Networks.

[13]  Stephen H. Friedberg,et al.  Linear Algebra , 2018, Computational Mathematics with SageMath.

[14]  Babak Hassibi,et al.  Second Order Derivatives for Network Pruning: Optimal Brain Surgeon , 1992, NIPS.

[15]  Alberto Del Bimbo,et al.  Recurrent neural networks can be trained to be maximum a posteriori probability classifiers , 1995, Neural Networks.

[16]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[17]  Bernd Fritzke,et al.  Growing cell structures--A self-organizing network for unsupervised and supervised learning , 1994, Neural Networks.

[18]  Jenq-Neng Hwang,et al.  Regression modeling in back-propagation and projection pursuit learning , 1994, IEEE Trans. Neural Networks.

[19]  C. Lee Giles,et al.  Constructive learning of recurrent neural networks: limitations of recurrent cascade correlation and a simple solution , 1995, IEEE Trans. Neural Networks.

[20]  Timur Ash,et al.  Dynamic node creation in backpropagation networks , 1989 .

[21]  Mark S. Nixon,et al.  Generating-shrinking algorithm for learning arbitrary classification , 1994, Neural Networks.

[22]  Benjamin W. Wah,et al.  Global Optimization for Neural Network Training , 1996, Computer.

[23]  Benjamin W. Wah,et al.  Adaptive load-balancing strategies for distributed systems , 1992, Proceedings of the Second International Conference on Systems Integration.