Incorporating artificial intelligence in shopping assistance robot using Markov Decision Process

There are many challenges involved in the realization of a shopping assistance robot (SAR). The specific challenge addressed in this paper is that of incorporating artificial intelligence or decision making capability in such robot. Markov Decision Process (MDP) based formulation of the problem has been presented for this purpose. The major advantage of the MDP based approach over simple search based artificial intelligence techniques is that it can incorporate uncertainty. The proposed MDP model has been solved for optimal policy using value iteration algorithm. Furthermore, it has been shown how the reward function influences the structure of the resulting policy. The results show encouraging potential in the use of MDP based formulation for SAR.

[1]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[2]  Horst-Michael Groß,et al.  ShopBot: Progress in developing an interactive mobile shopping assistant for everyday use , 2008, 2008 IEEE International Conference on Systems, Man and Cybernetics.

[3]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[4]  Brahim Chaib-draa,et al.  Decomposition techniques for a loosely-coupled resource allocation problem , 2005, IEEE/WIC/ACM International Conference on Intelligent Agent Technology.

[5]  A. Barto,et al.  Learning and Sequential Decision Making , 1989 .

[6]  Vladimir A. Kulyukin,et al.  Robot-assisted shopping for the blind: issues in spatial cognition and product selection , 2008, Intell. Serv. Robotics.

[7]  Warren B. Powell,et al.  What you should know about approximate dynamic programming , 2009, Naval Research Logistics (NRL).

[8]  Andrew G. Barto,et al.  A causal approach to hierarchical decomposition of factored MDPs , 2005, ICML.

[9]  Ronen I. Brafman,et al.  Prioritized Goal Decomposition of Markov Decision Processes: Toward a Synthesis of Classical and Decision Theoretic Planning , 1997, IJCAI.

[10]  Z. Dziong,et al.  An analysis of near optimal call admission and routing model for multi-service loss networks , 1992, [Proceedings] IEEE INFOCOM '92: The Conference on Computer Communications.

[11]  Francisco Herrera,et al.  A Sequential Selection Process in Group Decision Making with a Linguistic Assessment Approach , 1995, Inf. Sci..

[12]  Takayuki Kanda,et al.  A Communication Robot in a Shopping Mall , 2010, IEEE Transactions on Robotics.

[13]  Philip Bachman,et al.  Data Generation as Sequential Decision Making , 2015, NIPS.

[14]  Chee Wei Tan,et al.  Automatic human guided shopping trolley with smart shopping system , 2015 .

[15]  Horst-Michael Groß,et al.  Vision-based Monte Carlo self-localization for a mobile service robot acting as shopping assistant in a home store , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[16]  A. Ohya,et al.  Remote Food Shopping Robot System in a Supermarket -Realization of the shopping task from remote places , 2007, 2007 International Conference on Mechatronics and Automation.

[17]  Ella M. Atkins,et al.  Human Intent Prediction Using Markov Decision Processes , 2012, J. Aerosp. Inf. Syst..

[18]  Bhaskara Marthi,et al.  Automatic shaping and decomposition of reward functions , 2007, ICML '07.

[19]  Mohammed Abbad,et al.  A decomposition algorithm for limiting average Markov decision problems , 2003, Oper. Res. Lett..

[20]  Andrea Lockerd Thomaz,et al.  Automatic task decomposition and state abstraction from demonstration , 2012, AAMAS.

[21]  Richard S. Sutton,et al.  Learning and Sequential Decision Making , 1989 .

[22]  John Nicholson,et al.  RoboCart: toward robot-assisted navigation of grocery stores by the visually impaired , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[23]  James M. Conrad,et al.  Human-robot collaboration: A survey , 2015, SoutheastCon 2015.

[24]  Dragan Bosnacki,et al.  GPU-Based Graph Decomposition into Strongly Connected and Maximal End Components , 2014, CAV.

[25]  Jianhui Wu,et al.  Solving large TÆMS problems efficiently by selective exploration and decomposition , 2007, AAMAS '07.

[26]  Simon X. Yang,et al.  Hierarchical Approximate Policy Iteration With Binary-Tree State Space Decomposition , 2011, IEEE Transactions on Neural Networks.

[27]  Horst-Michael Groß,et al.  TOOMAS: Interactive Shopping Guide robots in everyday use - final implementation and experiences from long-term field trials , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[28]  Moshe Tennenholtz,et al.  Sequential decision making with vector outcomes , 2014, ITCS.

[29]  Kee-Eung Kim,et al.  Solving Very Large Weakly Coupled Markov Decision Processes , 1998, AAAI/IAAI.

[30]  Angela J. Yu,et al.  Active Sensing as Bayes-Optimal Sequential Decision Making , 2013, UAI.

[31]  Feng Wu,et al.  Online planning for large MDPs with MAXQ decomposition , 2012, AAMAS.

[32]  Alain Haurie,et al.  Two-Time Scale Controlled Markov Chains: A Decomposition and Parallel Processing Approach , 2007, IEEE Transactions on Automatic Control.

[33]  Takayuki Kanda,et al.  An affective guide robot in a shopping mall , 2009, 2009 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[34]  Horst-Michael Groß,et al.  User-Centered Design and Evaluation of a Mobile Shopping Robot , 2015, Int. J. Soc. Robotics.

[35]  Milos Hauskrecht,et al.  Hierarchical Solution of Markov Decision Processes using Macro-actions , 1998, UAI.

[36]  Krishnendu Chatterjee,et al.  Faster and dynamic algorithms for maximal end-component decomposition and related graph problems in probabilistic verification , 2011, SODA '11.

[37]  Claudia V. Goldman,et al.  Communication-Based Decomposition Mechanisms for Decentralized MDPs , 2008, J. Artif. Intell. Res..

[38]  Takayuki Kanda,et al.  Do elderly people prefer a conversational humanoid as a shopping assistant partner in supermarkets? , 2011, 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[39]  A. Markman,et al.  Journal of Experimental Psychology : General Retrospective Revaluation in Sequential Decision Making : A Tale of Two Systems , 2012 .

[40]  Shin'ichi Yuta,et al.  Remote Shopping Robot System, -Development of a hand mechanism for grasping fresh foods in a supermarket , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[41]  Xin Chen,et al.  Model-based learning with Bayesian and MAXQ value function decomposition for hierarchical task , 2010, 2010 8th World Congress on Intelligent Control and Automation.

[42]  Ronald Parr,et al.  Flexible Decomposition Algorithms for Weakly Coupled Markov Decision Problems , 1998, UAI.