Viewpoint Selection for Photographing Architectures

This paper studies the problem of how to choose good viewpoints for taking photographs of architectures. We achieve this by learning from professional photographs of world famous landmarks that are available on the Internet. Unlike previous efforts devoted to photo quality assessment which mainly rely on 2D image features, we show in this paper combining 2D image features extracted from images with 3D geometric features computed on the 3D models can result in more reliable evaluation of viewpoint quality. Specifically, we collect a set of photographs for each of 15 world famous architectures as well as their 3D models from the Internet. Viewpoint recovery for images is carried out through an image-model registration process, after which a newly proposed viewpoint clustering strategy is exploited to validate users' viewpoint preferences when photographing landmarks. Finally, we extract a number of 2D and 3D features for each image based on multiple visual and geometric cues and perform viewpoint recommendation by learning from both 2D and 3D features using a specifically designed SVM-2K multi-view learner, achieving superior performance over using solely 2D or 3D features. We show the effectiveness of the proposed approach through extensive experiments. The experiments also demonstrate that our system can be used to recommend viewpoints for rendering textured 3D models of buildings for the use of architectural design, in addition to viewpoint evaluation of photographs and recommendation of viewpoints for photographing architectures in practice.

[1]  Yanwen Guo,et al.  Viewpoint Selection for Taking a good Photograph of Architecture , 2016, PG.

[2]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[3]  Bastian Leibe,et al.  Discovering favorite views of popular places with iconoid shift , 2011, 2011 International Conference on Computer Vision.

[4]  Michael M. Kazhdan,et al.  Poisson surface reconstruction , 2006, SGP '06.

[5]  David W. Jacobs,et al.  Mesh saliency , 2005, SIGGRAPH 2005.

[6]  John Shawe-Taylor,et al.  Two view learning: SVM-2K, Theory and Practice , 2005, NIPS.

[7]  Kenneth Levenberg A METHOD FOR THE SOLUTION OF CERTAIN NON – LINEAR PROBLEMS IN LEAST SQUARES , 1944 .

[8]  Kun Zhou,et al.  Interactive images , 2012, ACM Trans. Graph..

[9]  Vicente Ordonez,et al.  High level describable attributes for predicting aesthetics and interestingness , 2011, CVPR 2011.

[10]  Tao Mei,et al.  Query-Dependent Aesthetic Model With Deep Learning for Photo Quality Assessment , 2015, IEEE Transactions on Multimedia.

[11]  Yan Ke,et al.  The Design of High-Level Features for Photo Quality Assessment , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[12]  Mateu Sbert,et al.  Automatic View Selection Using Viewpoint Entropy and its Application to Image‐Based Modelling , 2003, Comput. Graph. Forum.

[13]  Adam Finkelstein,et al.  Automatic triage for a photo series , 2016, ACM Trans. Graph..

[14]  Maneesh Agrawala,et al.  Image warps for artistic perspective manipulation , 2010, ACM Trans. Graph..

[15]  Matthew A. Brown,et al.  Unsupervised 3D object recognition and reconstruction in unordered datasets , 2005, Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM'05).

[16]  Dacheng Tao,et al.  A Survey on Multi-view Learning , 2013, ArXiv.

[17]  Wenping Wang,et al.  Efficient view manipulation for cuboid-structured images , 2014, Comput. Graph..

[18]  Hamid Laga Semantics-Driven Approach for Automatic Selection of Best Views of 3D Shapes , 2010, 3DOR@Eurographics.

[19]  Mateu Sbert,et al.  Viewpoint Selection using Viewpoint Entropy , 2001, VMV.

[20]  M J Tarr,et al.  What Object Attributes Determine Canonical Views? , 1999, Perception.

[21]  Yanwen Guo,et al.  Improving Photo Composition Elegantly: Considering Image Similarity During Composition Optimization , 2012, Comput. Graph. Forum.

[22]  Mubarak Shah,et al.  A framework for photo-quality assessment and enhancement based on visual aesthetics , 2010, ACM Multimedia.

[23]  Wei Luo,et al.  Content-Based Photo Quality Assessment , 2013, IEEE Trans. Multim..

[24]  W. Chu Studying Aesthetics in Photographic Images Using a Computational Approach , 2013 .

[25]  Shotaro Akaho,et al.  A kernel method for canonical correlation analysis , 2006, ArXiv.

[26]  W. Rossmann Lie Groups: An Introduction through Linear Groups , 2002 .

[27]  Hong Liu,et al.  Web-image driven best views of 3D shapes , 2011, The Visual Computer.

[28]  Minh N. Do,et al.  Common Visual Pattern Discovery via Nonlinear Mean Shift Clustering , 2015, IEEE Transactions on Image Processing.

[29]  Frédo Durand,et al.  Computational rephotography , 2010, TOGS.

[30]  Gabriela Csurka,et al.  Assessing the aesthetic quality of photographs using generic image descriptors , 2011, 2011 International Conference on Computer Vision.

[31]  Adam Finkelstein,et al.  Perceptual models of viewpoint preference , 2011, TOGS.

[32]  J. Feldman,et al.  Information along contours and object boundaries. , 2005, Psychological review.

[33]  Jitendra Malik,et al.  Viewpoints and keypoints , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Luiz Velho,et al.  Learning good views through intelligent galleries , 2009, Comput. Graph. Forum.

[35]  Jianfei Cai,et al.  Robust surface reconstruction via dictionary learning , 2014, ACM Trans. Graph..

[36]  Ayellet Tal,et al.  Surface Regions of Interest for Viewpoint Selection , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Michela Spagnuolo,et al.  Semantics-driven best view of 3D shapes , 2009, Comput. Graph..

[38]  Steven M. Seitz,et al.  Finding paths through the world's photos , 2008, SIGGRAPH 2008.

[39]  Patrick Olivier,et al.  Camera Control in Computer Graphics , 2008, Comput. Graph. Forum.

[40]  H. Hotelling Relations Between Two Sets of Variates , 1936 .

[41]  Xiaoou Tang,et al.  Photo and Video Quality Evaluation: Focusing on the Subject , 2008, ECCV.

[42]  Bastian Leibe,et al.  Discovering Details and Scene Structure with Hierarchical Iconoid Shift , 2013, 2013 IEEE International Conference on Computer Vision.

[43]  Vincent Lepetit,et al.  Learning descriptors for object recognition and 3D pose estimation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Hongbin Zha,et al.  Simultaneous Vanishing Point Detection and Camera Calibration from Single Images , 2010, ISVC.