A Quantum-inspired Q-learning Algorithm for Indoor Robot Navigation

A quantum-inspired Q-learning (QIQL) algorithm is proposed for indoor robot navigation control. Q- learning is an action-dependent reinforcement learning method and has been widely used in robot navigation. Inspired by the fundamental characteristics of quantum computation, e.g. state superposition principle and quantum parallel computation, probability is introduced to Q-learning and along with the learning process the probability of each action to be selected at a certain state is updated, which leads to a natural exploration strategy instead of a pointed one with configured parameters. The simulated navigation experiments show that the proposed QIQL algorithm keeps a good balance of exploration and exploitation, which can avoid the local optimal policies and accelerate the learning process as well.

[1]  张陈斌,et al.  Quantum Mechanics Helps in Learning for More Intelligent Robots , 2006 .

[2]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[3]  Zhang Chenbin,et al.  Quantum Mechanics Helps in Learning for More Intelligent Robots , 2006 .

[4]  Zonghai Chen,et al.  QUANTUM COMPUTATION FOR ACTION SELECTION USING REINFORCEMENT LEARNING , 2006 .

[5]  Leslie Pack Kaelbling,et al.  Effective reinforcement learning for mobile robots , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[6]  Lov K. Grover.  Quantum Mechanics Helps in Searching for a Needle in a Haystack , 1997, quant-ph/9706033.

[7]  R. Jozsa,et al.  Quantum Computation and Shor's Factoring Algorithm , 1996 .

[8]  Ajit Narayanan,et al.  Quantum artificial neural network architectures and components , 2000, Inf. Sci..

[9]  Thierry Paul,et al.  Quantum computation and quantum information , 2007, Mathematical Structures in Computer Science.

[10]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[11]  Spyros G. Tzafestas,et al.  Fuzzy reinforcement learning control for compliance tasks of robotic manipulators , 2002, IEEE Trans. Syst. Man Cybern. Part B.

[12]  Zhongzhi Shi,et al.  Adaptive action selection using utility-based reinforcement learning , 2009, 2009 IEEE International Conference on Granular Computing.

[13]  Mehmet Tomak,et al.  Quantum Genetic Algorithm Method In Self-Consistent Electronic Structure Calculations Of A Quantum Dot With Many Electrons , 2005 .

[14]  Yang Liu,et al.  A new Q-learning algorithm based on the metropolis criterion , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[15]  Luis Moreno,et al.  Navigation of mobile robots: open questions , 2000, Robotica.

[16]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[17]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[18]  Toshiyuki Kondo,et al.  A Reinforcement Learning with Evolutionary State Recruitment Strategy for Autonomous Mobile Robots Control , 2003 .

[19]  Tony R. Martinez,et al.  Quantum associative memory , 2000, Inf. Sci..

[20]  Spyros G. Tzafestas,et al.  Parallelization of a fuzzy control algorithm using quantum computation , 2002, IEEE Trans. Fuzzy Syst..

[21]  J. Preskill.  Work: Quantum Information and Computation , 1998 .

[22]  Peter W. Shor,et al.  Algorithms for quantum computation: discrete logarithms and factoring , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[23]  Lov K. Grover.  A fast quantum mechanical algorithm for database search , 1996, STOC '96.

[24]  Tzyh Jong Tarn,et al.  Quantum Reinforcement Learning , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).