Neural Networks and Complexity Theory

We survey some of the central results in the complexity theory of neural networks, with pointers to the literature.

[1]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[2]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry , 1969 .

[3]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[4]  Ingo Wegener,et al.  The complexity of Boolean functions , 1987 .

[5]  S C Kleene,et al.  Representation of Events in Nerve Nets and Finite Automata , 1951 .

[6]  J. Reif,et al.  On Threshold Circuits and Polynomial Computation , 1992, SIAM J. Comput..

[7]  Martin Hasler,et al.  Recursive neural networks for associative memory , 1990, Wiley-interscience series in systems and optimization.

[8]  Michael Luby,et al.  Steepest Descent Can Take Exponential Time for Symmetric Connection Networks , 1988, Complex Syst..

[9]  Alon Orlitsky,et al.  A geometric approach to threshold circuit complexity , 1991, COLT '91.

[10]  Georg Schnitger,et al.  On the computational power of sigmoid versus Boolean threshold circuits , 1991, [1991] Proceedings 32nd Annual Symposium of Foundations of Computer Science.

[11]  Pekka Orponen,et al.  Attraction Radii in Binary Hopfield Nets are Hard to Compute , 1993, Neural Computation.

[12]  Alexander A. Razborov,et al.  Majority gates vs. general weighted threshold gates , 1992, [1992] Proceedings of the Seventh Annual Structure in Complexity Theory Conference.

[13]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[14]  J. Stephen Judd,et al.  Neural network design and the complexity of learning , 1990, Neural network modeling and connectionism.

[15]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[16]  Zoran Obradovic,et al.  Analog Neural Networks of Limited Precision I: Computing with Multilinear Threshold Functions , 1989, NIPS.

[17]  Marvin Minsky,et al.  Computation : finite and infinite machines , 2016 .

[18]  José L. Balcázar,et al.  On Characterizations of the Class PSPACE/POLY , 1987, Theor. Comput. Sci..

[19]  Eric Goles Ch.,et al.  Exponential Transient Classes of Symmetric Neural Networks for Synchronous and Sequential Updating , 1989, Complex Syst..

[20]  Max H. Garzon,et al.  Global Dynamics in Neural Networks II , 1990, Complex Syst..

[21]  Marvin Minsky,et al.  An introduction to computational geometry , 1969 .

[22]  Eric Goles Ch.,et al.  Decreasing energy functions as a tool for studying threshold networks , 1985, Discret. Appl. Math..

[23]  Michael Sipser,et al.  Parity, circuits, and the polynomial-time hierarchy , 1981, 22nd Annual Symposium on Foundations of Computer Science (sfcs 1981).

[24]  S. Muroga,et al.  Theory of majority decision elements , 1961 .

[25]  Jehoshua Bruck,et al.  Neural computation of arithmetic functions , 1990 .

[26]  Noga Alon,et al.  Efficient simulation of finite automata by neural nets , 1991, JACM.

[27]  Georg Schnitger,et al.  Relating Boltzmann machines to conventional models of computation , 1987, Neural Networks.

[28]  James A. Anderson,et al.  Neurocomputing: Foundations of Research , 1988 .

[29]  Pekka Orponen,et al.  On the Computational Power of Discrete Hopfield Nets , 1993, ICALP.

[30]  Leslie G. Valiant,et al.  Cryptographic Limitations on Learning Boolean Formulae and Finite Automata , 1993, Machine Learning: From Theory to Applications.

[31]  William H. Kautz,et al.  On the Size of Weights Required for Linear-Input Switching Functions , 1961, IRE Transactions on Electronic Computers.

[32]  E. Goles,et al.  Exponential Transient Classes of Symmetric Neural Networks for Synchronous and Sequential Updating , 1989, Complex Syst..

[33]  G. Kane.  Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol 1: Foundations, vol 2: Psychological and Biological Models , 1994 .

[34]  Max H. Garzon,et al.  Global Dynamics in Neural Networks , 1989, Complex Syst..

[35]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[36]  Saburo Muroga,et al.  Threshold logic and its applications , 1971 .

[37]  S. Franklin,et al.  Neural computability. II , 1989, International 1989 Joint Conference on Neural Networks.

[38]  Pekka Orponen,et al.  On the Computational Complexity of Analyzing Hopfield Nets , 1989, Complex Syst..

[39]  A. Yao.  Separating the polynomial-time hierarchy by oracles , 1985 .

[40]  P. Raghavan,et al.  Learning in threshold networks , 1988, COLT '88.

[41]  John Astad.  On the size of weights for threshold , 1994 .

[42]  Stephen D. Collins.  Neurocomputing 2 , 1992, Neurology.

[43]  Yves Robert,et al.  Automata networks in computer science : theory and applications , 1987 .

[44]  Eric Goles Ch.,et al.  The Convergence of Symmetric Threshold Automata , 1981, Inf. Control..

[45]  Jehoshua Bruck,et al.  On the Power of Threshold Circuits with Small Weights , 1991, SIAM J. Discret. Math..

[46]  J. Stephen Judd,et al.  On the complexity of loading shallow neural networks , 1988, J. Complex..

[47]  Pavel Pudlák,et al.  Threshold circuits of bounded depth , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[48]  Noga Alon.  Asynchronous threshold networks , 1985, Graphs Comb..

[49]  Jehoshua Bruck.  On the convergence properties of the Hopfield model , 1990, Proc. IEEE.

[50]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[51]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[52]  Geoffrey E. Hinton,et al.  Learning and relearning in Boltzmann machines , 1986 .

[53]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[54]  Hava T. Siegelmann,et al.  On the computational power of neural nets , 1992, COLT '92.

[55]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[56]  Jirí Wiedermann,et al.  Complexity Issues in Discrete Neurocomputing , 1990, IMYCS.

[57]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[58]  Eric Goles,et al.  Neural and automata networks , 1990 .

[59]  Eric Goles Ch.,et al.  Transient length in sequential iteration of threshold functions , 1983, Discret. Appl. Math..

[60]  Johan Håstad,et al.  Almost optimal lower bounds for small depth circuits , 1986, STOC '86.

[61]  Ronald L. Rivest,et al.  Training a 3-node neural network is NP-complete , 1988, COLT '88.