Detection and correction of subtle context-dependent robot model inaccuracies using parametric regions

Autonomous robots frequently rely on models of their sensing and actions for intelligent decision making. Unfortunately, in complex environments, robots are bound to encounter situations in which their models do not accurately represent the world. Furthermore, these context-dependent model inaccuracies may be subtle, such that multiple observations may be necessary to distinguish them from noise. This paper formalizes the problem of detection and correction of such subtle contextual model inaccuracies in autonomous robots, and presents an algorithm to address this problem. The solution relies on reasoning about these contextual inaccuracies as parametric regions of inaccurate modeling (RIMs) in the robot’s planning space. Empirical results from various real robot domains demonstrate that, by explicitly searching for RIMs, robots are capable of efficiently detecting subtle contextual model inaccuracies, which in turn can lead to task performance improvement.

[1]  Paulo Martins Engel,et al.  Dealing with non-stationary environments using context detection , 2006, ICML.

[2]  Manuela M. Veloso,et al.  Selectively Reactive Coordination for a Team of Robot Soccer Champions , 2016, AAAI.

[3]  Marcel Staroswiecki,et al.  Conflicts versus analytical redundancy relations: a comparative analysis of the model based diagnosis approach from the artificial intelligence and automatic control perspectives , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[4]  M. Kulldorff,et al.  An elliptic spatial scan statistic , 2006, Statistics in medicine.

[5]  T. Tango,et al.  International Journal of Health Geographics a Flexibly Shaped Spatial Scan Statistic for Detecting Clusters , 2005 .

[6]  Manuela M. Veloso,et al.  Focused optimization for online detection of anomalous regions , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[7]  Peter Stone,et al.  Reinforcement Learning for RoboCup Soccer Keepaway , 2005, Adapt. Behav..

[8]  Aleksandrs Slivkins,et al.  Contextual Bandits with Similarity Information , 2009, COLT.

[9]  E. S. Page CONTINUOUS INSPECTION SCHEMES , 1954 .

[10]  Dit-Yan Yeung,et al.  Hidden-Mode Markov Decision Processes for Nonstationary Sequential Decision Making , 2001, Sequence Learning.

[11]  Manuela M. Veloso,et al.  Detecting and Correcting Model Anomalies in Subspaces of Robot Planning Domains , 2015, AAMAS.

[12]  Malik Ghallab,et al.  Learning Behaviors Models for Robot Execution Control , 2006, ICAPS.

[13]  Ronen I. Brafman,et al.  R-MAX - A General Polynomial Time Algorithm for Near-Optimal Reinforcement Learning , 2001, J. Mach. Learn. Res..

[14]  R. Rubinstein The Cross-Entropy Method for Combinatorial and Continuous Optimization , 1999 .

[15]  Inseok Hwang,et al.  A Survey of Fault Detection, Isolation, and Reconfiguration Methods , 2010, IEEE Transactions on Control Systems Technology.

[16]  Peter Stone,et al.  Keepaway Soccer: From Machine Learning Testbed to Benchmark , 2005, RoboCup.

[17]  Igor V. Nikiforov,et al.  A generalized change detection problem , 1995, IEEE Trans. Inf. Theory.

[18]  Stephanie Rosenthal,et al.  CoBots: Collaborative robots servicing multi-floor buildings , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[19]  Mitsuo Kawato,et al.  Multiple Model-Based Reinforcement Learning , 2002, Neural Computation.

[20]  Jian Pei,et al.  WAT: Finding Top-K Discords in Time Series Database , 2007, SDM.

[21]  Manuela M. Veloso,et al.  Localization and navigation of the CoBots over long-term deployments , 2013, Int. J. Robotics Res..

[22]  Reid Simmons,et al.  Detection of Subtle Context-Dependent Model Inaccuracies in High-Dimensional Robot Domains , 2016, Big Data.

[23]  Eamonn J. Keogh,et al.  Finding surprising patterns in a time series database in linear time and space , 2002, KDD.

[24]  Ola Pettersson,et al.  Execution monitoring in robotics: A survey , 2005, Robotics Auton. Syst..

[25]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  Juan Pablo Mendoza,et al.  Regions of Inaccurate Modeling for Robot Anomaly Detection and Model Correction , 2017 .

[27]  R. Bellman A Markovian Decision Process , 1957 .

[28]  Alessandro Saffiotti,et al.  Model-free execution monitoring by learning from simulation , 2005, 2005 International Symposium on Computational Intelligence in Robotics and Automation.

[29]  Alessandro Saffiotti,et al.  Model-free execution monitoring in behavior-based mobile robotics , 2003 .

[30]  Eamonn J. Keogh,et al.  HOT SAX: efficiently finding the most unusual time series subsequence , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[31]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[32]  Michael L. Littman,et al.  Multi-resolution Exploration in Continuous Spaces , 2008, NIPS.

[33]  Renato Assunção,et al.  A Simulated Annealing Strategy for the Detection of Arbitrarily Shaped Spatial Clusters , 2022 .

[34]  Andrew W. Moore,et al.  Detecting Significant Multidimensional Spatial Clusters , 2004, NIPS.

[35]  Daniel B. Neill,et al.  Fast subset scan for multivariate event detection , 2013, Statistics in medicine.

[36]  Michael Kearns,et al.  Near-Optimal Reinforcement Learning in Polynomial Time , 1998, Machine Learning.

[37]  Manuela M. Veloso,et al.  Opponent-driven planning and execution for pass, attack, and defense in a multi-robot soccer team , 2014, AAMAS.

[38]  A. Willsky,et al.  A generalized likelihood ratio approach to the detection and estimation of jumps in linear systems , 1976 .

[39]  Gianluca Antonelli A Survey of Fault Detection/Tolerance Strategies for AUVs and ROVs , 2003 .

[40]  Peter Stone,et al.  Model-based function approximation in reinforcement learning , 2007, AAMAS '07.

[41]  Brett Browning,et al.  STP: Skills, tactics, and plays for multi-robot control in adversarial environments , 2005 .

[42]  Peter Stone,et al.  TEXPLORE: real-time sample-efficient reinforcement learning for robots , 2012, Machine Learning.

[43]  Manuela M. Veloso,et al.  CMDragons 2015: Coordinated Offense and Defense of the SSL Champions , 2015, RoboCup.

[44]  Daniel B. Neill,et al.  Fast subset scan for spatial pattern detection , 2012 .

[45]  Andrew W. Moore,et al.  Detection of spatial and spatio-temporal clusters , 2006 .

[46]  Malik Ghallab,et al.  Robot introspection through learned hidden Markov models , 2006, Artif. Intell..

[47]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[48]  M. Kulldorff A spatial scan statistic , 1997 .

[49]  R. Khan,et al.  Sequential Tests of Statistical Hypotheses. , 1972 .