Automated photo-consistency test for voxel colouring based on fuzzy adaptive hysteresis thresholding

Voxel colouring is a popular method for reconstructing a three-dimensional surface model from a set of a few calibrated images. However, the reconstruction quality is largely dependent on a thresholding procedure allowing the authors to decide, for each voxel, whether it is photo-consistent or not. Nevertheless, in addition to the absence of any information on the neighbouring voxels during the photo-consistency test, it is extremely difficult to define the appropriate thresholds, which should be precise and stable on all surface voxels. In this study, the authors propose an automated photo-consistency test based on fuzzy hysteresis thresholding. The proposed method allows the incorporation of the spatial coherence during volume reconstruction, while avoiding ‘floating voxels’ and holes. Moreover, the ambiguity of choosing the thresholds is extremely minimised by defining a fuzzy degree of membership of each voxel into the class of consistent voxels. Also, there is no need for preset thresholds since the hysteresis ones are defined automatically and adaptively depending on the number of images that the voxel is projected onto. Preliminary results are very promising and demonstrate that the proposed method performs automatically precise and smooth volumetric scene reconstruction.

[1]  Olivier D. Faugeras,et al.  Multi-View Stereo Reconstruction and Scene Flow Estimation with a Global Image-Based Matching Score , 2007, International Journal of Computer Vision.

[2]  Kiriakos N. Kutulakos,et al.  A Theory of Shape by Space Carving , 2000, International Journal of Computer Vision.

[3]  Susan Hackwood,et al.  Computational geometric methods in volumetric intersection for 3D reconstruction , 1990, Pattern Recognit..

[4]  Szymon Rusinkiewicz,et al.  Spacetime Stereo: A Unifying Framework for Depth from Triangulation , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Marc Pollefeys,et al.  Multi-view reconstruction using photo-consistency and exact silhouette constraints: a maximum-flow formulation , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[6]  Jake K. Aggarwal,et al.  TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE , 2008 .

[7]  Diego F. Nehab,et al.  Efficiently combining positions and normals for precise 3D geometry , 2005, SIGGRAPH 2005.

[8]  Peter Eisert,et al.  3-D object reconstruction using spatially extended voxels and multi-hypothesis voxel coloring , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[9]  Qionghai Dai,et al.  Noisy Depth Maps Fusion for Multiview Stereo Via Matrix Completion , 2012, IEEE Journal of Selected Topics in Signal Processing.

[10]  Kiriakos N. Kutulakos.  Approximate N-View Stereo , 2000, ECCV.

[11]  Ruigang Yang,et al.  Dealing with textureless regions and specular highlights - a progressive space carving scheme using a novel photo-consistency measure , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[12]  Leif Kobbelt,et al.  Robust and Efficient Photo-Consistency Estimation for Volumetric 3D Reconstruction , 2006, ECCV.

[13]  Yanxi Liu,et al.  Deformed Lattice Detection in Real-World Images Using Mean-Shift Belief Propagation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Matthew Harker,et al.  Least squares surface reconstruction from measured gradient fields , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Kiriakos N. Kutulakos,et al.  A Probabilistic Theory of Occupancy and Emptiness , 2002, ECCV.

[16]  Qionghai Dai,et al.  Continuous depth estimation for multi-view stereo , 2009, CVPR.

[17]  Thomas Malzbender,et al.  Generalized Voxel Coloring , 1999, Workshop on Vision Algorithms.

[18]  Changming Sun,et al.  Embedded Voxel Colouring with Adaptive Threshold Selection Using Globally Minimal Surfaces , 2012, International Journal of Computer Vision.

[19]  Steven M. Seitz,et al.  Photorealistic Scene Reconstruction by Voxel Coloring , 1997, International Journal of Computer Vision.

[20]  J. Davis,et al.  Spacetime stereo: a unifying framework for depth from triangulation , 2005, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[21]  Ezzeddine Zagrouba,et al.  An online approach for multi-sprite generation based on camera parameters estimation , 2013, Signal Image Video Process..

[22]  Xin Liu,et al.  An active volumetric model for 3D reconstruction , 2005, IEEE International Conference on Image Processing 2005.

[23]  Roberto Cipolla,et al.  A Statistical Consistency Check for the Space Carving Algorithm , 2000, BMVC.

[24]  Jean Ponce,et al.  Carved Visual Hulls for Image-Based Modeling , 2006, International Journal of Computer Vision.

[25]  Paul A. Viola,et al.  Roxels: responsibility weighted 3D volume reconstruction , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[26]  Mark R. Stevens,et al.  A histogram-based color consistency test for voxel coloring , 2002, Object recognition supported by user interaction for service robots.

[27]  Michael Goesele,et al.  Multi-View Stereo Revisited , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[28]  B. Lovell,et al.  3D Reconstruction through Segmentation of Multi-View Image Sequences , 2003 .

[29]  Yun Tae Soo,et al.  3D object reconstruction using voxel coloring with mean charts color consistency checking , 2007 .

[30]  Carlos Hernández Esteban,et al.  Modélisation d'objets 3D par fusion silhouettes-stéréo à partir de séquences d'images en rotation non calibrées. (Stereo and silhouette fusion for 3D object modeling from uncalibrated images under circular motion) , 2004 .

[31]  Ronald W. Schafer,et al.  Image-based photo hulls for fast and photo-realistic new view synthesis , 2003, Real Time Imaging.

[32]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[33]  David A. Forsyth,et al.  Capturing and animating occluded cloth , 2007, ACM Trans. Graph..

[34]  Takeshi Oishi,et al.  Flying Laser Range Sensor for Large-Scale Site-Modeling and Its Applications in Bayon Digital Archival Project , 2008, International Journal of Computer Vision.

[35]  R. Cipolla,et al.  A probabilistic framework for space carving , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[36]  Larry S. Davis,et al.  A probabilistic framework for surface reconstruction from multiple images , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[37]  Rico Jonschkowski,et al.  GPU-friendly multi-view stereo reconstruction using surfel representation and graph cuts , 2011 .

[38]  Mark R. Stevens,et al.  Methods for Volumetric Reconstruction of Visual Scenes , 2004, International Journal of Computer Vision.

[39]  Yehu Shen.  Efficient normalized cross correlation calculation method for stereo vision based robot navigation , 2011, Frontiers of Computer Science in China.

[40]  Larry S. Davis,et al.  3D Surface Reconstruction Using Graph Cuts with Surface Constraints , 2006, ECCV.

[41]  Stefano Soatto,et al.  3-D Shape Estimation and Image Restoration - Exploiting Defocus and Motion Blur , 2006 .

[42]  P.H.N. de With,et al.  Automatic video-object segmentation employing multi-sprites with constrained delay , 2006, 2006 Digest of Technical Papers International Conference on Consumer Electronics.

[43]  Roberto Cipolla,et al.  Multi-view stereo via volumetric graph-cuts , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[44]  Jean-Yves Guillemaut,et al.  Joint Multi-Layer Segmentation and Reconstruction for Free-Viewpoint Video Applications , 2011, International Journal of Computer Vision.

[45]  Richard Szeliski,et al.  A multi-view approach to motion and stereo , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[46]  James M. Coughlan,et al.  Elevation-based MRF stereo implemented in real-time on a GPU , 2009, 2009 Workshop on Applications of Computer Vision (WACV).

[47]  Richard Szeliski,et al.  Extracting View-Dependent Depth Maps from a Collection of Images , 2004, International Journal of Computer Vision.

[48]  Vikram Chhabra.  Reconstructing specular objects with Image Based Rendering using Color Caching , 2001 .