Spectral camera clustering

We propose an algorithm for clustering large sets of images of a scene into smaller subsets covering different parts of the scene suitable for 3D reconstruction. Unlike the canonical view selection of [13], we do not focus only on the visibility information, but introduce an alternative similarity measure which takes into account the relative camera orientations and their distance from the scene. This allows us to formalize the clustering problem as a graph partitioning and solve it using spectral clustering. The obtained image clusters bring down the amount of data that has to be considered by the reconstruction algorithms simultaneously, thereby allowing traditional algorithms to take advantage of large multi-view data sets processing them significantly faster and at smaller memory costs compared to using the full image datasets. We tested our approach on a number of multi-view data sets and demonstrated that the clustering we obtain is suitable for 3D reconstruction and coincides with what a human observer would consider as a good clustering.

[1]  Steven M. Seitz,et al.  Scene Summarization for Online Image Collections , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[2]  Ben J. A. Kröse,et al.  Sparse appearance based modeling for robot localization , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Olivier D. Faugeras,et al.  Multi-View Stereo Reconstruction and Scene Flow Estimation with a Global Image-Based Matching Score , 2007, International Journal of Computer Vision.

[5]  David P. Dobkin,et al.  On sparse spanners of weighted graphs , 1993, Discret. Comput. Geom..

[6]  Steven M. Seitz,et al.  Scene Segmentation Using the Wisdom of Crowds , 2008, ECCV.

[7]  Sven J. Dickinson,et al.  Selecting canonical views for view-based 3-D object recognition , 2004, ICPR 2004.

[8]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[9]  Steven M. Seitz,et al.  Finding paths through the world's photos , 2008, SIGGRAPH 2008.

[10]  Francis Schmitt,et al.  Silhouette and stereo fusion for 3D object modeling , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[11]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Leif Kobbelt,et al.  Hierarchical Volumetric Multi-view Stereo Reconstruction of Manifold Surfaces based on Dual Graph Embedding , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[13]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[14]  Cedric Cagniart,et al.  Camera-Clustering for Multi-Resolution 3-D Surface Reconstruction , 2008 .

[15]  Richard Szeliski,et al.  Skeletal graphs for efficient structure from motion , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[17]  Leif Kobbelt,et al.  Image selection for improved Multi-View Stereo , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Pascal Fua,et al.  On benchmarking camera calibration and multi-view stereo for high resolution imagery , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Francis Schmitt,et al.  Silhouette and stereo fusion for 3D object modeling , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[20]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..