Computations in massively parallel networks based on the Boltzmann machine: a review

Boltzmann machines offer an exciting approach to connectionist networks. Salient features of these networks are their distributed internal representations and their use of massive parallelism. This paper reviews some of the achievements in the research on Boltzmann machines and discusses in particular two different fields of application, viz. (1) solving combinatorial optimization problems and (ii) carrying out learning tasks. Some open problems are also touched upon.

[1]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[2]  Geoffrey E. Hinton,et al.  Connectionist Architectures for Artificial Intelligence , 1990, Computer.

[3]  Mervyn Jack,et al.  Speech processing with a Boltzmann machine , 1986, ICASSP '86. IEEE International Conference on Acoustics, Speech, and Signal Processing.

[4]  Geoffrey E. Hinton,et al.  Massively Parallel Architectures for AI: NETL, Thistle, and Boltzmann Machines , 1983, AAAI.

[5]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[6]  Emile H. L. Aarts,et al.  Combinatorial Optimization on a Boltzmann Machine , 1989, J. Parallel Distributed Comput..

[7]  Dana H. Ballard,et al.  Modular Learning in Neural Networks , 1987, AAAI.

[8]  J J Hopfield,et al.  Neural computation by concentrating information in time. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[9]  Mark Derthick,et al.  Variations on the Boltzmann Machine Learning Algorithm , 1984 .

[10]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[11]  Jerome A. Feldman,et al.  Connectionist Models and Their Properties , 1982, Cogn. Sci..

[12]  Geoffrey E. Hinton,et al.  Learning and relearning in Boltzmann machines , 1986 .

[13]  Harrison H. Barrett,et al.  Optical Implementations In Boltzmann Machines , 1987 .

[14]  S. Kullback,et al.  Information Theory and Statistics , 1959 .

[15]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[16]  D B Bounds.  Numerical simulations of Boltzmann Machines , 1987 .

[17]  Solomon Kullback,et al.  Information Theory and Statistics , 1960 .

[18]  W. Daniel Hillis,et al.  The connection machine , 1985 .

[19]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[20]  M M Waldrop,et al.  Artificial Intelligence in Parallel: Working with many processors at once could accelerate computation enormously--and suggest new ways to think about thinking. , 1984, Science.

[21]  T. Sejnowski.  Higher‐order Boltzmann machines , 1987 .

[22]  Emile H. L. Aarts,et al.  Simulated Annealing: Theory and Applications , 1987, Mathematics and Its Applications.

[23]  Terrence J. Sejnowski,et al.  Separating figure from ground with a Boltzmann machine , 1990 .

[24]  D. E. Rumelhart,et al.  Learning internal representations by back-propagating errors , 1986 .

[25]  E. Aarts,et al.  Boltzmann machines for travelling salesman problems , 1989 .

[26]  Geoffrey E. Hinton,et al.  Learning distributed representations of concepts. , 1989 .

[27]  Igor Aleksander,et al.  Prospects for Adaptive Window Architectures , 1986, Other Conferences.

[28]  Emile H. L. Aarts,et al.  Boltzmann Machines and their Applications , 1987, PARLE.

[29]  John J. Hopfield,et al.  Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit , 1986 .

[30]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[32]  Geoffrey E. Hinton.  Connectionist Learning Procedures , 1989, Artif. Intell..

[33]  T. D. Harrison,et al.  Boltzmann machines for speech recognition , 1986 .

[34]  Philip C. Treleaven,et al.  Computer Architectures for Artificial Intelligence , 1986, Future Parallel Computers.

[35]  Geoffrey E. Hinton,et al.  OPTIMAL PERCEPTUAL INFERENCE , 1983 .

[36]  Jerome A. Feldman,et al.  Connectionist Models and Their Properties , 1982, Cogn. Sci..

[37]  John K. Stevens,et al.  Reverse engineering the brain , 1985 .