Attention-based active visual search for mobile robots

We present an active visual search model for finding objects in unknown environments. The proposed algorithm guides the robot towards the sought object using the relevant stimuli provided by the visual sensors. Existing search strategies are either purely reactive or use simplified sensor models that do not exploit all the visual information available. In this paper, we propose a new model that actively extracts visual information via visual attention techniques and, in conjunction with a non-myopic decision-making algorithm, leads the robot to search more relevant areas of the environment. The attention module couples both top-down and bottom-up attention models enabling the robot to search regions with higher importance first. The proposed algorithm is evaluated on a mobile robot platform in a 3D simulated environment. The results indicate that the use of visual attention significantly improves search, but the degree of improvement depends on the nature of the task and the complexity of the environment. In our experiments, we found that performance enhancements of up to 42\% in structured and 38\% in highly unstructured cluttered environments can be achieved using visual attention mechanisms.

[1]  Ming-Hsuan Yang,et al.  Top-down visual saliency via joint CRF and dictionary learning , 2012, CVPR.

[2]  Rynson W. H. Lau,et al.  Exemplar-Driven Top-Down Saliency Detection via Deep Association , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  José Antonio López Orozco,et al.  Minimum Time Search in Uncertain Dynamic Domains with Complex Sensorial Platforms , 2014, Sensors.

[4]  Yizhou Yu,et al.  Visual saliency based on multiscale deep features , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Lihi Zelnik-Manor,et al.  Context-Aware Saliency Detection , 2012, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Matthew W. Hoffman,et al.  Probabilistic Gaze Imitation and Saliency Learning in a Robotic Head , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[7]  Gordon Cheng,et al.  Biologically Based Top-Down Attention Modulation for Humanoid Interactions , 2008, Int. J. Humanoid Robotics.

[8]  Kuo-Shih Tseng,et al.  Near-optimal probabilistic search via submodularity and sparse regression , 2017, Auton. Robots.

[9]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[10]  John K. Tsotsos On the relative complexity of active vs. passive visual search , 2004, International Journal of Computer Vision.

[11]  R. Bajcsy Active perception , 1988 .

[12]  Xiaogang Wang,et al.  Saliency detection by multi-context deep learning , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Tingting Xu,et al.  Autonomous Behavior-Based Switched Top-Down and Bottom-Up Visual Attention for Mobile Robots , 2010, IEEE Transactions on Robotics.

[14]  Tao Zhang,et al.  Unsupervised learning to detect loops using deep neural networks for visual SLAM system , 2017, Auton. Robots.

[15]  Xi Chen,et al.  Visual Search of an Object in Cluttered Environments for Robotic Errand Service , 2013, 2013 IEEE International Conference on Systems, Man, and Cybernetics.

[16]  Jorge Dias,et al.  Designing an artificial attention system for social robots , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[17]  John K. Tsotsos,et al.  Attention based on information maximization , 2010 .

[18]  James J. Little,et al.  Informed visual search: Combining attention and object recognition , 2008, 2008 IEEE International Conference on Robotics and Automation.

[19]  John K. Tsotsos,et al.  Visual search for an object in a 3D environment using a mobile robot , 2010, Comput. Vis. Image Underst..

[20]  John K. Tsotsos The Complexity of Perceptual Search Tasks , 1989, IJCAI.

[21]  Giulio Sandini,et al.  Object-based Visual Attention: a Model for a Behaving Robot , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[22]  James N. Eagle The Optimal Search for a Moving Target When the Search Path Is Constrained , 1984, Oper. Res..

[23]  Joel W. Burdick,et al.  A Decision-Making Framework for Control Strategies in Probabilistic Search , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[24]  John K. Tsotsos Analyzing vision at the complexity level , 1990, Behavioral and Brain Sciences.

[25]  K E Trummel,et al.  Technical Note - The Complexity of the Optimal Searcher Path Problem , 1986, Oper. Res..

[26]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[27]  John K. Tsotsos,et al.  Towards the Quantitative Evaluation of Visual Attention Models Bottom−up Top-down Dynamic Static 0 0 0 , 2022 .

[28]  Massimo Vergassola,et al.  ‘Infotaxis’ as a strategy for searching without gradients , 2007, Nature.

[29]  Rui Zhang,et al.  Top-Down Saliency Detection via Contextual Pooling , 2014, J. Signal Process. Syst..

[30]  Gonzalo Pajares,et al.  Multi-UAV target search using decentralized gradient-based negotiation with expected observation , 2014, Inf. Sci..

[31]  Frank Dellaert,et al.  Saliency detection and model-based tracking: a two part vision system for small robot navigation in forested environment , 2012, Defense, Security, and Sensing.

[32]  Jeremy M. Wolfe,et al.  Guided Search 4.0: Current Progress With a Model of Visual Search , 2007, Integrated Models of Cognitive Systems.

[33]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[34]  Salah Sukkarieh,et al.  Multi-UAV target search using explicit decentralized gradient-based negotiation , 2011, 2011 IEEE International Conference on Robotics and Automation.

[35]  Patric Jensfelt,et al.  Topological spatial relations for active visual search , 2012, Robotics Auton. Syst..

[36]  K. Cave The FeatureGate model of visual selection , 1999, Psychological research.

[37]  John K. Tsotsos,et al.  Revisiting active perception , 2016, Autonomous Robots.

[38]  Nanning Zheng,et al.  Automatic salient object segmentation based on context and shape prior , 2011, BMVC.

[39]  Simone Frintrop,et al.  VOCUS: A Visual Attention System for Object Detection and Goal-Directed Search , 2006, Lecture Notes in Computer Science.

[40]  Danica Kragic,et al.  Vision for robotic object manipulation in domestic settings , 2005, Robotics Auton. Syst..

[41]  L. Stone Theory of Optimal Search , 1975 .

[42]  Miguel P Eckstein,et al.  Visual search: a retrospective. , 2011, Journal of vision.

[43]  Tadej Petric,et al.  Robotic assembly solution by human-in-the-loop teaching method based on real-time stiffness modulation , 2018, Auton. Robots.

[44]  Garrison W. Cottrell,et al.  Visual saliency model for robot cameras , 2008, 2008 IEEE International Conference on Robotics and Automation.

[45]  Jorge Dias,et al.  Attentional Mechanisms for Socially Interactive Robots–A Survey , 2014, IEEE Transactions on Autonomous Mental Development.

[46]  Yiming Ye,et al.  A Complexity‐Level Analysis of the Sensor Planning Task for Object Search , 2001, Comput. Intell..

[47]  Sylvain Chartier,et al.  An Introduction to Independent Component Analysis: InfoMax and FastICA algorithms , 2010 .

[48]  John K. Tsotsos,et al.  The Effect of Color Space Selection on Detectability and Discriminability of Colored Objects , 2017, ArXiv.

[49]  P. Pradas Búsqueda de objetivos móviles en tiempo mínimo sobre entornos con incertidumbre. Minimum time search of moving targets in uncertain environments , 2013 .

[50]  John K. Tsotsos,et al.  Attention in Autonomous Robotic Visual Search , 2014 .

[51]  Ryan M. Eustice,et al.  Real-Time Visual SLAM for Autonomous Underwater Hull Inspection Using Visual Saliency , 2013, IEEE Transactions on Robotics.

[52]  Patric Jensfelt,et al.  Active Visual Object Search in Unknown Environments Using Uncertain Semantics , 2013, IEEE Transactions on Robotics.

[53]  Yiming Ye,et al.  Sensor Planning for 3D Object Search, , 1999, Comput. Vis. Image Underst..

[54]  John K. Tsotsos,et al.  Saliency, attention, and visual search: an information theoretic approach. , 2009, Journal of vision.

[55]  Hugh F. Durrant-Whyte,et al.  Optimal Search for a Lost Target in a Bayesian World , 2003, FSR.

[56]  Gordon Cheng,et al.  A Tactile-Based Framework for Active Object Learning and Discrimination using Multimodal Robotic Skin , 2017, IEEE Robotics and Automation Letters.

[57]  Gordon Cheng,et al.  Distributed visual attention on a humanoid robot , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[58]  Andrew Howard,et al.  Design and use paradigms for Gazebo, an open-source multi-robot simulator , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[59]  Liqing Zhang,et al.  Dynamic visual attention: searching for coding length increments , 2008, NIPS.

[60]  John K. Tsotsos,et al.  Visual Saliency Improves Autonomous Visual Search , 2014, 2014 Canadian Conference on Computer and Robot Vision.

[61]  Shang-Hong Lai,et al.  Fusing generic objectness and visual saliency for salient object detection , 2011, 2011 International Conference on Computer Vision.

[62]  Ali Borji,et al.  Quantitative Analysis of Human-Model Agreement in Visual Saliency Modeling: A Comparative Study , 2013, IEEE Transactions on Image Processing.