论文信息 - Some n-bit parity problems are solvable by feedforward networks with less than n hidden units

Some n-bit parity problems are solvable by feedforward networks with less than n hidden units

Starting with two hidden units, we train a simple single hidden layer feedforward neural network to solve the n-bit parity problem. If the network fails to recognize correctly all the input patterns, an additional hidden unit is added to the hidden layer and the network is retrained. This process is repeated until a network that correctly classifies all the input patterns has been constructed. Using a variant of the quasi-Newton methods for training, we have been able to find networks with a single layer containing less than n hidden units that solve the n-bit parity problem for some value of n. This proves the power of combining quasi-Newton method and node incremental approach.

Rudy Setiono | Lucas Chi Kwong Hui | L. Hui | R. Setiono

[1] Korris Fu-Lai Chung,et al. A Node Pruning Algorithm for Backpropagation Networks , 1992, Int. J. Neural Syst..

[2] David G. Stork,et al. How to solve the N-bit parity problem with two hidden units , 1992, Neural Networks.

[3] D. Shanno. Conditioning of Quasi-Newton Methods for Function Minimization , 1970 .

[4] R. Fletcher,et al. A New Approach to Variable Metric Algorithms , 1970, Comput. J..

[5] David A. Wismer,et al. Introduction to nonlinear optimization , 1978 .

[6] M. C. Biggs. A Note on Minimization Algorithms which make Use of Non-quardratic Properties of the Objective Function , 1973 .

[7] Michael C. Mozer,et al. Using Relevance to Reduce Network Size Automatically , 1989 .

[8] D. Goldfarb. A family of variable-metric methods derived by variational means , 1970 .

[9] Timur Ash,et al. Dynamic node creation in backpropagation networks , 1989 .

[10] C. G. Broyden. The Convergence of a Class of Double-rank Minimization Algorithms 2. The New Algorithm , 1970 .

[11] Osamu Fujita,et al. Optimization of the hidden unit function in feedforward neural networks , 1992, Neural Networks.

[12] David Haussler,et al. What Size Net Gives Valid Generalization? , 1989, Neural Computation.