A Current-Mode Analog Circuit for Reinforcement Learning Problems

Reinforcement learning is important for machine-intelligence and neurophysiological modelling applications to provide time-critical decision making. Analog circuit implementation has been demonstrated as a powerful computational platform for power-efficient, bio-implantable and real-time applications. This paper presents a current-mode analog circuit design for solving reinforcement learning problem with simple and efficient computational network architecture. The design has been fabricated and a new procedure to validate the fabricated reinforcement learning circuit will also be presented. This work provides a preliminary study for future biomedical application using CMOS VLSI reinforcement learning model.

[1]  A. Redish,et al.  Addiction as a Computational Process Gone Awry , 2004, Science.

[2]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[3]  Peter Dayan,et al.  Dopamine: generalization and bonuses , 2002, Neural Networks.

[4]  Bernabé Linares-Barranco,et al.  A high-precision current-mode WTA-MAX circuit with multichip capability , 1998 .

[5]  W. Schultz.  Getting Formal with Dopamine and Reward , 2002, Neuron.

[6]  J. Nazuno.  Haykin, Simon. Neural networks: A comprehensive foundation, Prentice Hall, Inc. Segunda Edición, 1999 , 2000 .

[7]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[8]  K. P. Lam,et al.  Closed semiring connectionist network for the Bellman-Ford computation , 1996 .

[9]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[10]  Eric A. Vittoz,et al.  Analog VLSI signal processing: Why, where, and how? , 1994, J. VLSI Signal Process..

[11]  Richard S. Sutton,et al.  Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dynamic Programming , 1990, ML.

[12]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[13]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[14]  P. Dayan,et al.  Reward, Motivation, and Reinforcement Learning , 2002, Neuron.

[15]  Michel Verleysen,et al.  A current-mode CMOS loser-take-all with minimum function for neural computations , 2000, 2000 IEEE International Symposium on Circuits and Systems. Emerging Technologies for the 21st Century. Proceedings (IEEE Cat No.00CH36353).

[16]  T. Lindvall.  ON A ROUTING PROBLEM , 2004, Probability in the Engineering and Informational Sciences.

[17]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[18]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.