“Neural” computation of decisions in optimization problems

Highly-interconnected networks of nonlinear analog neurons are shown to be extremely effective in computing. The networks can rapidly provide a collectively-computed solution (a digital output) to a problem on the basis of analog input information. The problems to be solved must be formulated in terms of desired optima, often subject to constraints. The general principles involved in constructing networks to solve specific problems are discussed. Results of computer simulations of a network designed to solve a difficult but well-defined optimization problem-the Traveling-Salesman Problem-are presented and used to illustrate the computational power of the networks. Good solutions to this problem are collectively computed within an elapsed time of only a few neural time constants. The effectiveness of the computation involves both the nonlinear analog response of the neurons and the large connectivity among them. Dedicated networks of biological or microelectronic neurons could provide the computational capabilities described for a wide class of problems having combinatorial complexity. The power and speed naturally displayed by such collective networks may contribute to the effectiveness of biological information processing.

[1]  George S. Sebestyen,et al.  Decision-making processes in pattern recognition , 1962 .

[2]  George S Sebestyen,et al.  Decision-making processes in pattern recognition (ACM monograph series) , 1962 .

[3]  Rajko Tomović,et al.  High-speed analog computers , 1970 .

[4]  Brian W. Kernighan,et al.  An Effective Heuristic Algorithm for the Traveling-Salesman Problem , 1973, Oper. Res..

[5]  R. Palmer,et al.  Solution of 'Solvable model of a spin glass' , 1977 .

[6]  Lynn Conway,et al.  Introduction to VLSI systems , 1978 .

[7]  G. Likens,et al.  The Flow of Energy in a Forest Ecosystem , 1978 .

[8]  G. Shepherd.  Microcircuits in the nervous system. , 1978, Scientific American.

[9]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[10]  Katsushi Ikeuchi,et al.  Numerical Shape from Shading and Occluding Boundaries , 1981, Artif. Intell..

[11]  B. Julesz.  Textons, the elements of texture perception, and their interactions , 1981, Nature.

[12]  Jerome A. Feldman,et al.  Connectionist Models and Their Properties , 1982, Cogn. Sci..

[13]  Demetri Terzopoulos.  Multi-Level Reconstruction of Visual Surfaces: Variational Principles and Finite Element Representations , 1982 .

[14]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[15]  T. Poggio,et al.  Nonlinear interactions in a dendritic tree: localization, timing, and role in information processing. , 1983, Proceedings of the National Academy of Sciences of the United States of America.

[16]  P. Anderson.  Basic Notions of Condensed Matter Physics , 1983 .

[17]  Geoffrey E. Hinton,et al.  OPTIMAL PERCEPTUAL INFERENCE , 1983 .

[18]  Geoffrey E. Hinton,et al.  Parallel visual computation , 1983, Nature.

[19]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[20]  P. Goldman-Rakic.  Modular organization of prefrontal cortex , 1984, Trends in Neurosciences.

[21]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[22]  M. Mézard,et al.  The simplest spin glass , 1984 .

[23]  T. Poggio,et al.  Ill-Posed Problems and Regularization Analysis in Early Vision , 1984 .

[24]  Tomaso Poggio,et al.  An Analog Model of Computation for the Ill-Posed Problems of Early Vision, , 1984 .

[25]  T. Poggio,et al.  III-Posed problems early vision: from computational theory to analogue networks , 1985, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[26]  Eugene L. Lawler,et al.  Traveling Salesman Problem , 2016 .

[27]  J. Hopfield,et al.  The Logic of Limax Learning , 1985 .

[28]  Dana H. Ballard,et al.  Cortical connections and parallel processing: Structure and function , 1986, Behavioral and Brain Sciences.

[29]  M. Bertero,et al.  Ill-posed problems in early vision , 1988, Proc. IEEE.

[30]  Michael A. Arbib,et al.  Artificial intelligence and brain theory: Unities and diversities , 1975, Annals of Biomedical Engineering.