TOTEM: a digital processor for neural networks and Reactive Tabu Search

The Reactive Tabu Search (RTS) algorithm permits the training of neural networks with low number of bits per weight, low computational accuracy, no local minima "trapping", and limited sensitivity to the initial conditions. Two architectures for the implementation of multilayer perceptrons based directly on the properties of RTS are presented. They are characterized by high-speed integer computation, tunable precision, I/O balanced design, internal storage of the weights and compact processing units. TOTEM, a silicon implementation which uses only 8 bits per weight and 16 bits per input, is then presented. A triggering problem in high energy physics is used as a test-bed for the algorithm and the processor. The expected performance is briefly compared with those of two alternative processors.

[1]  Roberto Battiti,et al.  First- and Second-Order Methods for Learning: Between Steepest Descent and Newton's Method , 1992, Neural Computation.

[2]  Roberto Battiti,et al.  Learning with first, second, and no derivatives: A case study in high energy physics , 1994, Neurocomputing.

[3]  Jeff Brauch,et al.  Analog VLSI neural networks for impact signal processing , 1992, IEEE Micro.

[4]  Bruce Denby,et al.  The Use of Neural Networks in High-Energy Physics , 1993, Neural Computation.

[5]  武藤 佳恭.  Analog VLSI neural networks , 1993 .

[6]  John F. Kolen,et al.  Backpropagation is Sensitive to Initial Conditions , 1990, Complex Syst..

[7]  Bruce A. Wooley,et al.  A Two's Complement Parallel Array Multiplication Algorithm , 1973, IEEE Transactions on Computers.

[8]  Roberto Battiti,et al.  Training neural nets with the reactive tabu search , 1995, IEEE Trans. Neural Networks.

[9]  Eric A. Vittoz,et al.  Analog VLSI implementation of neural networks , 1990, IEEE International Symposium on Circuits and Systems.

[10]  Carver Mead,et al.  Analog VLSI and neural systems , 1989 .

[11]  Bertil Svensson,et al.  Using and Designing Massively Parallel Computers for Artificial Neural Neural Networks , 1992, J. Parallel Distributed Comput..

[12]  Claudio Bruschini,et al.  Results from an on-line neural trigger within a fixed target experiment for the production of beauty particles , 1993 .

[13]  Fred Glover,et al.  Tabu Search - Part II , 1989, INFORMS J. Comput..

[14]  George Cybenko,et al.  Ill-Conditioning in Neural Network Training Problems , 1993, SIAM J. Sci. Comput..

[15]  Roberto Battiti,et al.  The Reactive Tabu Search , 1994, INFORMS J. Comput..

[16]  Vittorio Maniezzo,et al.  Genetic evolution of the topology and weight distribution of neural networks , 1994, IEEE Trans. Neural Networks.

[17]  Henk Termeer,et al.  A 32 kbit Variable Length Shift Register for Digital Audio Application , 1986, ESSCIRC '86: Twelfth European Solid-State Circuits Conference.

[18]  Francesco Piazza,et al.  Fast neural networks without multipliers , 1993, IEEE Trans. Neural Networks.

[19]  Ulrich Ramacher,et al.  SYNAPSE - A Neurocomputer that Synthesizes Neural Algorithms on a Parallel Systolic Engine , 1992, J. Parallel Distributed Comput..

[20]  Fred W. Glover,et al.  Tabu Search - Part I , 1989, INFORMS J. Comput..