Cooperative Active Perception using POMDPs

This paper studies active perception in an urban scenario, focusing on the cooperation between a set of surveillance cameras and mobile robots. The fixed cameras provide a global but incomplete and possibly inaccurate view of the environment, which can be enhanced by a robot’s local sensors. Active perception means that the robot considers the effects of its actions on its sensory capabilities. In particular, it tries to improve its sensors’ performance, for instance by pointing a pan-and-tilt camera. In this paper, we present a decision-theoretic approach to cooperative active perception, by formalizing the problem as a Partially Observable Markov Decision Process (POMDP). POMDPs provide an elegant way to model the interaction of an active sensor with its environment. The goal of this paper is to provide first steps towards an integrated decision-theoretic approach of cooperative active

[1]  E. J. Sondik,et al.  The Optimal Control of Partially Observable Markov Decision Processes. , 1971 .

[2]  J. Satia,et al.  Markovian Decision Processes with Probabilistic Observation of States , 1973 .

[3]  Keiji Kanazawa,et al.  A model for reasoning about persistence and causation , 1989 .

[4]  Ross D. Shachter,et al.  Dynamic programming and influence diagrams , 1990, IEEE Trans. Syst. Man Cybern..

[5]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[6]  Reid G. Simmons,et al.  Probabilistic Robot Navigation in Partially Observable Environments , 1995, IJCAI.

[7]  Leslie Pack Kaelbling,et al.  Learning Policies for Partially Observable Environments: Scaling Up , 1997, ICML.

[8]  Lawrence B. Holder,et al.  Decision-Theoretic Cooperative Sensor Planning , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Craig Boutilier,et al.  Context-Specific Independence in Bayesian Networks , 1996, UAI.

[10]  Alex Pentland,et al.  Active gesture recognition using partially observable Markov decision processes , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[11]  Leslie Pack Kaelbling,et al.  Acting under uncertainty: discrete Bayesian models for mobile-robot navigation , 1996, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems. IROS '96.

[12]  Francesco Mondada,et al.  Autonomous vacuum cleaner , 1997, Robotics Auton. Syst..

[13]  Karen Zita Haigh,et al.  A layered architecture for office delivery robots , 1997, AGENTS '97.

[14]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[15]  Wolfram Burgard,et al.  The Interactive Museum Tour-Guide Robot , 1998, AAAI/IAAI.

[16]  Jim Blythe,et al.  An Overview of Planning Under Certainty , 1999, Artificial Intelligence Today.

[17]  Craig Boutilier,et al.  Decision-Theoretic Planning: Structural Assumptions and Computational Leverage , 1999, J. Artif. Intell. Res..

[18]  Jesse Hoey,et al.  SPUDD: Stochastic Planning using Decision Diagrams , 1999, UAI.

[19]  Neil Immerman,et al.  The Complexity of Decentralized Control of Markov Decision Processes , 2000, UAI.

[20]  Kurt Konolige,et al.  A gradient method for realtime robot control , 2000, Proceedings. 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000) (Cat. No.00CH37113).

[21]  Milos Hauskrecht,et al.  Value-Function Approximations for Partially Observable Markov Decision Processes , 2000, J. Artif. Intell. Res..

[22]  Craig Boutilier,et al.  Stochastic dynamic programming with factored representations , 2000, Artif. Intell..

[23]  Wolfram Burgard,et al.  Robust Monte Carlo localization for mobile robots , 2001, Artif. Intell..

[24]  Stuart J. Russell,et al.  Dynamic bayesian networks: representation, inference and learning , 2002 .

[25]  Sebastian Thrun,et al.  Planning under Uncertainty for Reliable Health Care Robotics , 2003, FSR.

[26]  Luis Miguel Bergasa,et al.  Visually Augmented POMDP for Indoor Robot Navigation , 2003, Applied Informatics.

[27]  A. Cassandra A Survey of POMDP Applications , 2003 .

[28]  AnYuan Guo,et al.  Decision-theoretic active sensing for autonomous agents , 2003, AAMAS '03.

[29]  Sebastian Thrun,et al.  Perspectives on standardization in mobile robot programming: the Carnegie Mellon Navigation (CARMEN) Toolkit , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[30]  Joelle Pineau,et al.  Towards robotic assistants in nursing homes: Challenges and results , 2003, Robotics Auton. Syst..

[31]  Nikos A. Vlassis,et al.  Perseus: Randomized Point-based Value Iteration for POMDPs , 2005, J. Artif. Intell. Res..

[32]  Jesse Hoey,et al.  A Decision-Theoretic Approach to Task Assistance for Persons with Dementia , 2005, IJCAI.

[33]  Jeff G. Schneider,et al.  Game Theoretic Control for Robot Teams , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[34]  Geoffrey J. Gordon,et al.  Finding Approximate POMDP solutions Through Belief Compression , 2011, J. Artif. Intell. Res..

[35]  P. Poupart Exploiting structure to efficiently solve large scale partially observable Markov decision processes , 2005 .

[36]  Alessandro Saffiotti,et al.  PEIS ecologies: ambient intelligence meets autonomous robotics , 2005, sOc-EUSAI '05.

[37]  J. Andrade-Cetto,et al.  Ubiquitous networking robotics in urban settings , 2006 .

[38]  Pascal Poupart,et al.  Point-Based Value Iteration for Continuous POMDPs , 2006, J. Mach. Learn. Res..

[39]  Joelle Pineau,et al.  Planning under uncertainty in robotics , 2006, Robotics Auton. Syst..

[40]  Peter Stone,et al.  Non-commercial Research and Educational Use including without Limitation Use in Instruction at Your Institution, Sending It to Specific Colleagues That You Know, and Providing a Copy to Your Institution's Administrator. All Other Uses, Reproduction and Distribution, including without Limitation Comm , 2022 .

[41]  Pedro U. Lima,et al.  RAPOSA: Semi-Autonomous Robot for Rescue Operations , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[42]  Shlomo Zilberstein,et al.  Formal models and algorithms for decentralized decision making under uncertainty , 2008, Autonomous Agents and Multi-Agent Systems.

[43]  Pedro U. Lima,et al.  MERMAID – MULTIPLE-ROBOT MIDDLEWARE FOR INTELLIGENT DECISION-MAKING , 2007 .

[44]  Lawrence Carin,et al.  Cost-sensitive feature acquisition and classification , 2007, Pattern Recognit..

[45]  Takayuki Kanda,et al.  Interactive Humanoid Robots for a Science Museum , 2006, IEEE Intelligent Systems.

[46]  Jong-Hwan Kim,et al.  Ubiquitous Robot: A New Paradigm for Integrated Services , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[47]  Manuela M. Veloso,et al.  Exploiting factored representations for decentralized execution in multiagent teams , 2007, AAMAS '07.

[48]  Kevin P. Murphy,et al.  A non-myopic approach to visual search , 2007, Fourth Canadian Conference on Computer and Robot Vision (CRV '07).

[49]  Brahim Chaib-draa,et al.  AEMS: An Anytime Online Search Algorithm for Approximate Policy Refinement in Large POMDPs , 2007, IJCAI.

[50]  Sriraam Natarajan,et al.  A Decision-Theoretic Model of Assistance , 2007, IJCAI.

[51]  Makoto Yokoo,et al.  Letting loose a SPIDER on a network of POMDPs: generating quality guaranteed policies , 2007, AAMAS '07.

[52]  G. Kraetzschmar,et al.  Two "Hot Issues" in Cooperative Robotics: Network Robot Systems, and Formal Models and Methods for Cooperation , 2008 .

[53]  Nikos A. Vlassis,et al.  Optimal and Approximate Q-value Functions for Decentralized POMDPs , 2008, J. Artif. Intell. Res..

[54]  Francisco S. Melo,et al.  Interaction-driven Markov games for decentralized multiagent planning under uncertainty , 2008, AAMAS.

[55]  Paolo Dario,et al.  Networked and Cooperating Robots for Urban Hygiene: the EU funded DustBot Project , 2008 .

[56]  Nan Rong,et al.  A point-based POMDP planner for target tracking , 2008, 2008 IEEE International Conference on Robotics and Automation.

[57]  Shimon Whiteson,et al.  Exploiting locality of interaction in factored Dec-POMDPs , 2008, AAMAS.

[58]  Alessandro Saffiotti,et al.  Network robot systems , 2008, Robotics Auton. Syst..

[59]  David Hsu,et al.  POMDPs for robotic tasks with mixed observability , 2009, Robotics: Science and Systems.

[60]  Pedro U. Lima,et al.  Decision-theoretic robot guidance for active cooperative perception , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[61]  Pedro U. Lima,et al.  A Decision-Theoretic Approach to Dynamic Sensor Selection in Camera Networks , 2009, ICAPS.

[62]  Pedro U. Lima,et al.  ISROBOTNET: A testbed for sensor and robot network systems , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[63]  David Hsu,et al.  Motion Planning for People Tracking in Uncertain and Dynamic Environments , 2009 .