Mesh Saliency: An Independent Perceptual Measure or A Derivative of Image Saliency?

While mesh saliency aims to predict regional importance of 3D surfaces in agreement with human visual perception and is well researched in computer vision and graphics, latest work with eye-tracking experiments shows that state-of-the-art mesh saliency methods remain poor at predicting human fixations. Cues emerging prominently from these experiments suggest that mesh saliency might associate with the saliency of 2D natural images. This paper proposes a novel deep neural network for learning mesh saliency using image saliency ground truth to 1) investigate whether mesh saliency is an independent perceptual measure or just a derivative of image saliency and 2) provide a weakly supervised method for more accurately predicting mesh saliency. Through extensive experiments, we not only demonstrate that our method outperforms the current state-of-the-art mesh saliency method by 116% and 21% in terms of linear correlation coefficient and AUC respectively, but also reveal that mesh saliency is intrinsically related with both image saliency and object categorical information. Codes are available at https://github.com/rsong/MIMO-GAN.

[1]  Neil A. Dodgson,et al.  Cluster-Based Point Set Saliency , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[2]  Ayellet Tal,et al.  Surface Regions of Interest for Viewpoint Selection , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Thomas A. Funkhouser,et al.  Distinctive regions of 3D surfaces , 2007, TOGS.

[4]  David W. Jacobs,et al.  Mesh saliency , 2005, ACM Trans. Graph..

[5]  T. Poggio,et al.  Predicting the visual world: silence is golden , 1999, Nature Neuroscience.

[6]  Pat Hanrahan,et al.  Semantically-enriched 3D models for common-sense knowledge , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[7]  Thomas A. Funkhouser,et al.  Schelling points on 3D surface meshes , 2012, ACM Trans. Graph..

[8]  Hong Qin,et al.  Admissible Diffusion Wavelets and Their Applications in Space-Frequency Processing , 2013, IEEE Transactions on Visualization and Computer Graphics.

[9]  Mohamed-Chaker Larabi,et al.  Visual Attention for Rendered 3D Shapes , 2018, Comput. Graph. Forum.

[10]  Ligang Liu,et al.  Mesh saliency with global rarity , 2013, Graph. Model..

[11]  Huchuan Lu,et al.  Saliency Detection via Graph-Based Manifold Ranking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Jianxiong Xiao,et al.  3D ShapeNets: A deep representation for volumetric shapes , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  J. Wolfe,et al.  Guided Search 2.0 A revised model of visual search , 1994, Psychonomic bulletin & review.

[14]  P. Cavanagh,et al.  The Spatial Resolution of Visual Attention , 2001, Cognitive Psychology.

[15]  Marc Alexa,et al.  Tracking the gaze on objects in 3D , 2018, ACM Trans. Graph..

[16]  Thomas Brox,et al.  Orientation-boosted Voxel Nets for 3D Object Recognition , 2016, BMVC.

[17]  Weisi Lin,et al.  Point Cloud Saliency Detection by Local and Global Feature Fusion , 2019, IEEE Transactions on Image Processing.

[18]  Ralph R. Martin,et al.  Saliency-guided integration of multiple scans , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Serge J. Belongie,et al.  Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[20]  Yonghuai Liu,et al.  Mesh Saliency via Weakly Supervised Classification-for-Saliency CNN , 2021, IEEE Transactions on Visualization and Computer Graphics.

[21]  Subhransu Maji,et al.  Multi-view Convolutional Neural Networks for 3D Shape Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[22]  Shuai Li,et al.  Multi-scale mesh saliency based on low-rank and sparse analysis in shape feature space , 2015, Comput. Aided Geom. Des..

[23]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[24]  Leonidas J. Guibas,et al.  Volumetric and Multi-view CNNs for Object Classification on 3D Data , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Leonidas J. Guibas,et al.  A concise and provably informative multi-scale signature based on heat diffusion , 2009 .

[26]  Ralph R. Martin,et al.  Mesh saliency via spectral processing , 2014, ACM Trans. Graph..

[27]  Andrea Vedaldi,et al.  Improved Texture Networks: Maximizing Quality and Diversity in Feed-Forward Stylization and Texture Synthesis , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Ersin Yumer,et al.  Learning Local Shape Descriptors from Part Correspondences with Multiview Convolutional Networks , 2017, ACM Trans. Graph..

[29]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[30]  Manfred Lau,et al.  Tactile mesh saliency , 2016, ACM Trans. Graph..

[31]  Daniel Cohen-Or,et al.  Upright orientation of man-made objects , 2008, ACM Trans. Graph..

[32]  David Lindlbauer,et al.  Measuring the Visual Salience of 3D Printed Objects , 2016, IEEE Computer Graphics and Applications.

[33]  Frédo Durand,et al.  What Do Different Evaluation Metrics Tell Us About Saliency Models? , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Qi Zhao,et al.  SALICON: Saliency in Context , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  David W. Jacobs,et al.  Mesh saliency and human eye fixations , 2010, TAP.

[36]  Daniel Cohen-Or,et al.  Unsupervised Detection of Distinctive Regions on 3D Shapes , 2020, ACM Trans. Graph..

[37]  Subhransu Maji,et al.  3D Shape Segmentation with Projective Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Daniel Cohen-Or,et al.  Salient geometric features for partial shape matching and similarity , 2006, TOGS.