Ranking Regions of Visual Saliency in RGB-D Content

Effective immersion takes place when the user can relate to the 3D environment presented and interact with key objects. Efficiently predicting which objects in a scene are in the user’s attention, without using additional hardware, such as eye tracking solutions, provides an opportunity for creating more immersive scenes in real time and at lower costs. This is nonetheless algorithmically challenging. In this paper, we are proposing a technique that efficiently and effectively identifies the most salient objects in a scene. We show how it accurately matches user selection within 0.04s and is over 95% faster than other saliency algorithms while also providing a ranking of the most salient segments in a scene.

[1]  Xiangming Wen,et al.  An effective visual saliency detection method based on maximum entropy random walk , 2016, 2016 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[2]  Manuel Gértrudix Barrio,et al.  Immersive Environments and Virtual Reality: Systematic Review and Advances in Communication, Interaction and Simulation , 2017, Multimodal Technol. Interact..

[3]  Robert J. Teather,et al.  Guidelines for 3D positioning techniques , 2007, Future Play.

[4]  Ali Borji,et al.  Boosting bottom-up and top-down visual features for saliency estimation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Yihua Tan,et al.  Maximal Entropy Random Walk for Region-Based Visual Saliency , 2014, IEEE Transactions on Cybernetics.

[6]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.

[7]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[8]  Puneet Sharma,et al.  Evaluating Visual Saliency Algorithms: Past, Present and Future , 2015 .

[9]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[10]  Gayoung Lee,et al.  ELD-Net: An Efficient Deep Learning Architecture for Accurate Saliency Detection , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Carl James Debono,et al.  Monoscopic inpainting approach using depth information , 2016, 2016 18th Mediterranean Electrotechnical Conference (MELECON).

[12]  Stephen Lin,et al.  Object-based RGBD image co-segmentation with mutex constraint , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Huchuan Lu,et al.  Amulet: Aggregating Multi-level Convolutional Features for Salient Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[14]  John K. Tsotsos,et al.  Saliency Based on Information Maximization , 2005, NIPS.

[15]  Atsuto Maki,et al.  Towards a simulation driven stereo vision system , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[16]  Frédo Durand,et al.  Learning to predict where humans look , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[17]  Ge Li,et al.  Automatic Salient Object Detection for Panoramic Images Using Region Growing and Fixation Prediction Model , 2017, ArXiv.

[18]  Yu Hu,et al.  Learning to Detect Saliency with Deep Structure , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.