Learning nonoverlapping perceptron networks from examples and membership queries

We investigate, within the PAC learning model, the problem of learning nonoverlapping perceptron networks (also known as read-once formulas over a weighted threshold basis). These are loop-free neural nets in which each node has only one outgoing weight. We give a polynomial time algorithm that PAC learns any nonverlapping perceptron network using examples and membership queries. The algorithm is able to identify both the architecture and the weight values necessary to represent the function to be learned. Our results shed some light on the effect of the overlap on the complexity of learning in neural networks.

[1]  Dana Angluin,et al.  When won't membership queries help? , 1991, STOC '91.

[2]  Leslie G. Valiant,et al.  On the learnability of Boolean formulae , 1987, STOC.

[3]  Marek Karpinski,et al.  Learning read-once formulas with queries , 1993, JACM.

[4]  ERIC B. BAUM,et al.  On learning a union of half spaces , 1990, J. Complex..

[5]  Eric B. Baum,et al.  Neural net algorithms that learn in polynomial time from examples and queries , 1991, IEEE Trans. Neural Networks.

[6]  Lisa Hellerstein,et al.  Learning Boolean read-once formulas with arbitrary symmetric and constant fan-in gates , 1992, COLT '92.

[7]  Lisa Hellerstein,et al.  Learning Arithmetic Read-Once Formulas , 1995, SIAM J. Comput..

[8]  Ronald L. Rivest,et al.  Training a 3-node neural network is NP-complete , 1988, COLT '88.

[9]  Mario Marchand,et al.  On Learning µ-Perceptron Networks with Binary Weights , 1992, NIPS.

[10]  I. Kanter,et al.  Storage Capacity of a Multilayer Neural Network with Binary Weights , 1991 .

[11]  J. Stephen Judd,et al.  On the complexity of loading shallow neural networks , 1988, J. Complex..

[12]  David Haussler,et al.  What Size Net Gives Valid Generalization? , 1989, Neural Computation.

[13]  Eric B. Baum,et al.  A Polynomial Time Algorithm That Learns Two Hidden Unit Nets , 1990, Neural Computation.

[14]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, Comb..

[15]  Thomas R. Hancock,et al.  Learning 2µ DNF Formulas and kµ Decision Trees , 1991, COLT.

[16]  Leslie G. Valiant,et al.  Cryptographic Limitations on Learning Boolean Formulae and Finite Automata , 1993, Machine Learning: From Theory to Applications.

[17]  David Haussler,et al.  Learnability and the Vapnik-Chervonenkis dimension , 1989, JACM.

[18]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.