Development and Evaluation of Object-Based Visual Attention for Automatic Perception of Robots

Bottom-up visual attention is an automatic behavior to guide visual perception to a conspicuous object in a scene. This paper develops a new object-based bottom-up attention (OBA) model for robots. This model includes four modules: Extraction of preattentive features, preattentive segmentation, estimation of space-based saliency, and estimation of proto-object-based saliency. In terms of computation, preattentive segmentation serves as a bridge to connect the space-based saliency and object-based saliency. This paper therefore proposes a preattentive segmentation algorithm, which is able to self-determine the number of proto-objects, has low computational cost, and is robust in a variety of conditions such as noise and spatial transformations. Experimental results have shown that the propsoed OBA model outperforms space-based attention model and other object-based attention methods in terms of accuracy of attentional selection, consistency under a series of noise settings and object completion.

[1]  Robert B. Fisher,et al.  Object-based visual attention for computer vision , 2003, Artif. Intell..

[2]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Harry Shum,et al.  Image segmentation by data driven Markov chain Monte Carlo , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[4]  Ronen Basri,et al.  Fast multiscale image segmentation , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[5]  Gareth Funka-Lea,et al.  Graph Cuts and Efficient N-D Image Segmentation , 2006, International Journal of Computer Vision.

[6]  Ulrich Engelke,et al.  Visual Attention in Quality Assessment , 2011, IEEE Signal Processing Magazine.

[7]  Rainer Stiefelhagen,et al.  Multimodal saliency-based attention for object-based scene analysis , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[8]  C. J. Downing.  Expectancy and visual-spatial attention: effects on perceptual quality. , 1988, Journal of experimental psychology. Human perception and performance.

[9]  Anatole Lécuyer,et al.  Design and Application of Real-Time Visual Attention Model for the Exploration of 3D Virtual Environments , 2012, IEEE Transactions on Visualization and Computer Graphics.

[10]  T. Hoya,et al.  Notions of intuition and attention modeled by a hierarchically arranged generalized regression neural network , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[11]  Leo Grady,et al.  Isoperimetric graph partitioning for image segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Bärbel Mertsching,et al.  Evaluation of Visual Attention Models for Robots , 2006, Fourth IEEE International Conference on Computer Vision Systems (ICVS'06).

[13]  Giulio Sandini,et al.  Object-based Visual Attention: a Model for a Behaving Robot , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[14]  M. Posner,et al.  Orienting of Attention* , 1980, The Quarterly journal of experimental psychology.

[15]  T. Poggio,et al.  What and where: A Bayesian inference theory of attention , 2010, Vision Research.

[16]  Fiora Pirri,et al.  A biologically plausible robot attention model, based on space and time , 2006, Cognitive Processing.

[17]  Pietro Perona,et al.  Overcomplete steerable pyramid filters and rotation invariance , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Xuelong Li,et al.  Visual Attention Accelerated Vehicle Detection in Low-Altitude Airborne Video of Urban Environment , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[19]  George K. I. Mann,et al.  A Goal-Directed Visual Perception System Using Object-Based Top–Down Attention , 2012, IEEE Transactions on Autonomous Mental Development.

[20]  Jon Driver,et al.  Shifting visual attention between objects and locations: Normality and pathology. , 1994 .

[21]  W. Singer,et al.  In search of common foundations for cortical computation , 1997, Behavioral and Brain Sciences.

[22]  Edgar Arce Santana,et al.  Hidden Markov Measure Field Models for Image Segmentation , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  George K. I. Mann,et al.  An Object-Based Visual Attention Model for Robotic Applications , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[25]  Richard M. Leahy,et al.  An Optimal Graph Theoretic Approach to Data Clustering: Theory and Its Application to Image Segmentation , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  B. Scholl.  Objects and attention: the state of the art , 2001, Cognition.

[27]  N. Kanwisher,et al.  Objects, Attributes, and Visual Attention: Which, What, and Where , 1992 .

[28]  Donald Geman,et al.  Boundary Detection by Constrained Optimization , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Brian Scassellati,et al.  Active vision for sociable robots , 2001, IEEE Trans. Syst. Man Cybern. Part A.

[30]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[31]  Jitendra Malik,et al.  Contour and Texture Analysis for Image Segmentation , 2001, International Journal of Computer Vision.

[32]  Carl R Olson,et al.  Object-based vision and attention in primates , 2001, Current Opinion in Neurobiology.

[33]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[34]  Ronen Basri,et al.  Texture segmentation by multiscale aggregation of filter responses and shape elements , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[35]  智一 吉田,et al.  Efficient Graph-Based Image Segmentationを用いた圃場図自動作成手法の検討 , 2014 .

[36]  Pierre Baldi,et al.  Of bits and wows: A Bayesian theory of surprise with applications to attention , 2010, Neural Networks.

[37]  Leyuan Shi,et al.  IEEE Transactions on Automation Science and Engineering , 2009, IEEE Transactions on Automation Science and Engineering.

[38]  S P Tipper,et al.  Action-based mechanisms of attention. , 1998, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[39]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[40]  Pierre Baldi,et al.  Bayesian surprise attracts human attention , 2005, Vision Research.

[41]  Azriel Rosenfeld,et al.  A Pyramid Framework for Early Vision , 1994 .

[42]  J. Duncan.  Selective attention and the organization of visual information. , 1984, Journal of experimental psychology. General.

[43]  Fiora Pirri,et al.  Robot task-driven attention , 2006, PCAR '06.

[44]  Fakhri Karray,et al.  A Probabilistic Model of Overt Visual Attention for Cognitive Robots , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[45]  Fiora Pirri,et al.  Bottom-Up Gaze Shifts and Fixations Learning by Imitation , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[46]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[47]  Robert B. Fisher,et al.  A computer vision model for visual-object-based attention and eye movements , 2008, Comput. Vis. Image Underst..

[48]  Eckehard G. Steinbach,et al.  A Probabilistic Appearance Representation and Its Application to Surprise Detection in Cognitive Robots , 2010, IEEE Transactions on Autonomous Mental Development.

[49]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[50]  J. Wolfe,et al.  Preattentive Object Files: Shapeless Bundles of Basic Features , 1997, Vision Research.

[51]  Azriel Rosenfeld,et al.  Hierarchical Image Analysis Using Irregular Tessellations , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[52]  Dina E. Melas,et al.  Double Markov random fields and Bayesian image segmentation , 2002, IEEE Trans. Signal Process..

[53]  S. Tipper,et al.  Object-based and environment-based inhibition of return of visual attention. , 1994, Journal of experimental psychology. Human perception and performance.

[54]  M. Bravo,et al.  Preattentive Vision and Perceptual Groups , 1990, Perception.

[55]  Jean-Michel Jolion.  Stochastic pyramid revisited , 2003, Pattern Recognit. Lett..

[56]  Peter Meer,et al.  Stochastic image pyramids , 1989, Comput. Vis. Graph. Image Process..

[57]  Simone Frintrop,et al.  A Component-Based Approach to Visual Person Tracking from a Mobile Platform , 2010, Int. J. Soc. Robotics.

[58]  Bärbel Mertsching,et al.  Data- and Model-Driven Gaze Control for an Active-Vision System , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[59]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[60]  Simone Frintrop,et al.  Attentional Landmarks and Active Gaze Control for Visual SLAM , 2008, IEEE Transactions on Robotics.

[61]  Ronen Basri,et al.  Segmentation and boundary detection using multiscale intensity measurements , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[62]  James T. Enns,et al.  Attention and Visual Memory in Visualization and Computer Graphics , 2012, IEEE Transactions on Visualization and Computer Graphics.

[63]  Zhi-Qiang Liu,et al.  Self-Validated Labeling of Markov Random Fields for Image Segmentation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[64]  L. Itti,et al.  Modeling the influence of task on attention , 2005, Vision Research.

[65]  Barry D. Vaughan,et al.  Object-Based Visual Selection: Evidence From Perceptual Completion , 1998 .

[66]  Simone Frintrop,et al.  VOCUS: A Visual Attention System for Object Detection and Goal-Directed Search , 2006, Lecture Notes in Computer Science.

[67]  J. Wolfe,et al.  Guided Search 2.0 A revised model of visual search , 1994, Psychonomic bulletin & review.

[68]  John K. Tsotsos,et al.  Modeling Visual Attention via Selective Tuning , 1995, Artif. Intell..

[69]  George K. I. Mann,et al.  Target tracking for moving robots using object-based visual attention , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[70]  Leo Grady,et al.  Random Walks for Image Segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[71]  Ronen Basri,et al.  Hierarchy and adaptivity in segmenting visual scenes , 2006, Nature.

[72]  R. Rafal,et al.  Shifting visual attention between objects and locations: evidence from normal and parietal lesion subjects. , 1994, Journal of experimental psychology. General.

[73]  C. Eriksen,et al.  Visual attention within and around the field of focal attention: A zoom lens model , 1986, Perception & psychophysics.

[74]  Azriel Rosenfeld,et al.  A Pyramid Framework for Early Vision: Multiresolutional Computer Vision , 1993 .

[75]  Pietro Perona,et al.  On the usefulness of attention for object recognition , 2004 .

[76]  Stephen Grossberg,et al.  Consciousness CLEARS the mind , 2007, Neural Networks.

[77]  Alberto Finzi,et al.  Model-based control architecture for attentive robots in rescue scenarios , 2008, Auton. Robots.