3D Saliency for Finding Landmark Buildings

In urban environments the most interesting and effective factors for localization and navigation are landmark buildings. This paper proposes a novel method to detect such buildings that stand out, i.e. would be given the status of 'landmark'. The method works in a fully unsupervised way, i.e. it can be applied to different cities without requiring annotation. First, salient points are detected, based on the analysis of their features as well as those found in their spatial neighborhood. Second, learning refines the points by finding connected landmark components and training a classifier to distinguish these from common building components. Third, landmark components are aggregated into complete landmark buildings. Experiments on city-scale point clouds show the viability and efficiency of our approach on various tasks.

[1]  Alexei A. Efros,et al.  Unsupervised Discovery of Mid-Level Discriminative Patches , 2012, ECCV.

[2]  W. Chu.  Studying Aesthetics in Photographic Images Using a Computational Approach , 2013 .

[3]  Jason Weston,et al.  Curriculum learning , 2009, ICML '09.

[4]  Luc Van Gool,et al.  Efficient edge-aware surface mesh reconstruction for urban scenes , 2017, Comput. Vis. Image Underst..

[5]  Yong Jae Lee,et al.  Learning the easy things first: Self-paced visual category discovery , 2011, CVPR 2011.

[6]  Stephen C. Hirtle,et al.  The Nature of Landmarks for Real and Electronic Spaces , 1999, COSIT.

[7]  Luc Van Gool,et al.  Frankenhorse: Automatic Completion of Articulating Objects from Image-based Reconstruction , 2014, BMVC.

[8]  Sergio Escalera,et al.  Complex Salient Regions for Computer Vision Problems , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  R. Venkatesh Babu,et al.  DeepFix: A Fully Convolutional Neural Network for Predicting Human Eye Fixations , 2015, IEEE Transactions on Image Processing.

[10]  Martial Hebert,et al.  Segmentation of Salient Regions in Outdoor Scenes Using Imagery and 3-D Data , 2008, 2008 IEEE Workshop on Applications of Computer Vision.

[11]  Matthew de Brecht,et al.  A neural network implementation of a saliency map model , 2006, Neural Networks.

[12]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Sudeep Sarkar,et al.  Saliency in images and video: a brief survey , 2012 .

[14]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Lihi Zelnik-Manor,et al.  Context-Aware Saliency Detection , 2012, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Igor Guskov,et al.  Multi-scale features for approximate alignment of point-based surfaces , 2005, SGP '05.

[17]  Frédo Durand,et al.  A Benchmark of Computational Models of Saliency to Predict Human Fixations , 2012 .

[18]  Luc Van Gool,et al.  Ensemble Projection for Semi-supervised Image Classification , 2013, 2013 IEEE International Conference on Computer Vision.

[19]  Pieter P. Jonker,et al.  Computing Saliency Map from Spatial Information in Point Cloud Data , 2010, ACIVS.

[20]  J. Morgan.  Landmarks? , 2013 .

[21]  Luc Van Gool,et al.  Ensemble Partitioning for Unsupervised Image Categorization , 2012, ECCV.

[22]  Alexei A. Efros,et al.  What makes Paris look like Paris? , 2015, Commun. ACM.

[23]  Nuno Vasconcelos,et al.  Bottom-up saliency is a discriminant process , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[24]  Luc Van Gool,et al.  Mobile phone and cloud — A dream team for 3D reconstruction , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[25]  Hans-Peter Kriegel,et al.  LOF: identifying density-based local outliers , 2000, SIGMOD '00.

[26]  Maneesh Agrawala,et al.  Automatic generation of tourist maps , 2008, ACM Trans. Graph..

[27]  Delbert Dueck,et al.  Clustering by Passing Messages Between Data Points , 2007, Science.

[28]  Luc Van Gool,et al.  3D all the way: Semantic segmentation of urban scenes from start to end in 3D , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Nico Blodow,et al.  Fast Point Feature Histograms (FPFH) for 3D registration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[30]  Alexei A. Efros,et al.  Mid-level Visual Element Discovery as Discriminative Mode Seeking , 2013, NIPS.

[31]  Luc Van Gool,et al.  A unified framework for content-aware view selection and planning through view importance , 2014, BMVC.

[32]  K. Madhava Krishna,et al.  Depth really Matters: Improving Visual Salient Region Detection with Depth , 2013, BMVC.

[33]  Jieping Ye,et al.  Discriminative K-means for Clustering , 2007, NIPS.

[34]  Jianbo Shi,et al.  Detecting unusual activity in video , 2004, CVPR 2004.

[35]  Yali Amit,et al.  Shape Quantization and Recognition with Randomized Trees , 1997, Neural Computation.

[36]  Ayellet Tal,et al.  Saliency Detection in Large Point Sets , 2013, 2013 IEEE International Conference on Computer Vision.

[37]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[38]  Andreas Krause,et al.  Discriminative Clustering by Regularized Information Maximization , 2010, NIPS.

[39]  Luc Van Gool,et al.  Navigation using special buildings as signposts , 2014, MapInteract '14.

[40]  Luc Van Gool,et al.  Efficient architectural structural element decomposition , 2017, Comput. Vis. Image Underst..

[41]  Luc Van Gool,et al.  Learning Where to Classify in Multi-view Semantic Segmentation , 2014, ECCV.