Ray Saliency: Bottom-Up Visual Saliency for a Rotating and Zooming Camera

We extend the classical notion of computational visual saliency to multi-image data collected using a stationary pan-tilt-zoom (PTZ) camera by introducing the concept of consistency: the requirement that the set of generated saliency maps should each assign the same saliency value to unique regions of the environment that appear in more than one image. We show that processing each image independently will often fail to provide a consistent measure of saliency, and that using an image mosaic to quantify saliency suffers from several drawbacks. We then propose ray saliency and an immediate extension, approximate ray saliency: a mosaic-free method for calculating a consistent measure of bottom-up saliency. Experimental results demonstrating the effectiveness of the proposed approach are presented.

[1]  Sven J. Dickinson,et al.  TurboPixels: Fast Superpixels Using Geometric Flows , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  John K. Tsotsos,et al.  Saliency, attention, and visual search: an information theoretic approach. , 2009, Journal of vision.

[3]  Antonio Torralba,et al.  Top-down control of visual attention in object detection , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[4]  Jitendra Malik,et al.  Learning a classification model for segmentation , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[5]  Matthew A. Brown,et al.  Automatic Panoramic Image Stitching using Invariant Features , 2007, International Journal of Computer Vision.

[6]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[7]  Nanning Zheng,et al.  Learning to Detect a Salient Object , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Yuhong Yang.  Elements of Information Theory (2nd ed.). Thomas M. Cover and Joy A. Thomas , 2008 .

[9]  Chak-Kuen Wong,et al.  Worst-case analysis for region and partial region searches in multidimensional binary search trees and balanced quad trees , 1977, Acta Informatica.

[10]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[11]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[12]  Steve McLaughlin,et al.  Comparative study of textural analysis techniques to characterise tissue from intravascular ultrasound , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[13]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[14]  Nuno Vasconcelos,et al.  Discriminant Saliency, the Detection of Suspicious Coincidences, and Applications to Visual Recognition , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[16]  Richard I. Hartley,et al.  Self-Calibration of Stationary Cameras , 1997, International Journal of Computer Vision.

[17]  Nuno Vasconcelos,et al.  The discriminant center-surround hypothesis for bottom-up saliency , 2007, NIPS.

[18]  Yael Pritch,et al.  Saliency filters: Contrast based filtering for salient region detection , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Heinz Hügli,et al.  Visual Attention on the Sphere , 2008, IEEE Transactions on Image Processing.

[20]  Amitabh Varshney,et al.  Saliency-Assisted Navigation of Very Large Landscape Images , 2011, IEEE Transactions on Visualization and Computer Graphics.

[21]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[22]  Rama Chellappa,et al.  Entropy-Rate Clustering: Cluster Analysis via Maximizing a Submodular Function Subject to a Matroid Constraint , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Peter Kovesi,et al.  MATLAB Functions for Computer Vision and Image Analysis , 2004 .

[24]  Alexandre Bernardino,et al.  Multimodal saliency-based bottom-up attention a framework for the humanoid robot iCub , 2008, 2008 IEEE International Conference on Robotics and Automation.

[25]  Eli Brenner,et al.  Reliable Identification by Color under Natural Conditions the Locations Baseline Measurement , 2022 .

[26]  Jon Louis Bentley,et al.  Multidimensional binary search trees used for associative searching , 1975, CACM.

[27]  John K. Tsotsos,et al.  Saliency Based on Information Maximization , 2005, NIPS.

[28]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[29]  Heinz Hügli,et al.  Dynamic visual attention on the sphere , 2010, Comput. Vis. Image Underst..

[30]  Richard Alan Peters,et al.  Image Mapping and Visual Attention on a Sensory Ego-Sphere , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[31]  Jingdong Wang,et al.  Salient Object Detection: A Discriminative Regional Feature Integration Approach , 2013, International Journal of Computer Vision.

[32]  Ian D. Reid,et al.  Self-Calibration of Rotating and Zooming Cameras , 2002, International Journal of Computer Vision.

[33]  Alan Yuille,et al.  Active Vision , 2014, Computer Vision, A Reference Guide.

[34]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[35]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[36]  Yiannis Aloimonos,et al.  Active vision , 2004, International Journal of Computer Vision.

[37]  Dana H. Ballard,et al.  Animate Vision , 1991, Artif. Intell..

[38]  Neil A. Dodgson,et al.  Proceedings Ninth IEEE International Conference on Computer Vision , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[39]  Laurent Itti,et al.  Biologically-inspired robotics vision monte-carlo localization in the outdoor environment , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[40]  Ali Borji,et al.  Salient Object Detection: A Benchmark , 2015, IEEE Transactions on Image Processing.

[41]  Nuno Vasconcelos,et al.  Saliency-based discriminant tracking , 2009, CVPR.