Geometric inpainting of 3D structures

In this paper, we address the problem of inpainting in 3D digital models with large holes. The missing region inference problem is solved with a dictionary learning-based method that harnesses a geometric prior derived from a single self-similar structure and online depth databases. The underlying surface is recovered by adaptively propagating local 3D surface smoothness from around the boundary of the hole by appropriately harvesting the cue provided by the geometric prior. We showcase the relevance of our method in the archaeological domain which warrants `filling-in' missing information in damaged heritage sites. The performance of our method is demonstrated on holes with different complexities and sizes on synthetic as well as real examples.

[1]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[2]  Christophe Geuzaine,et al.  Gmsh: A 3‐D finite element mesh generator with built‐in pre‐ and post‐processing facilities , 2009 .

[3]  Peter Liepa,et al.  Filling Holes in Meshes , 2003, Symposium on Geometry Processing.

[4]  Guillermo Sapiro,et al.  Online Learning for Matrix Factorization and Sparse Coding , 2009, J. Mach. Learn. Res..

[5]  Chi-Keung Tang,et al.  Inference of segmented color and texture description by tensor voting , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Alan C. Bovik,et al.  Texas 3D Face Recognition Database , 2010, 2010 IEEE Southwest Symposium on Image Analysis & Interpretation (SSIAI).

[7]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[8]  Wolfram Burgard,et al.  A benchmark for the evaluation of RGB-D SLAM systems , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[10]  Andriy Myronenko,et al.  Point Set Registration: Coherent Point Drift , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Gerhard Rigoll,et al.  Depth Inpainting with Tensor Voting using Local Geometry , 2012, VISAPP.

[12]  Paolo Cignoni,et al.  Visualization and 3D data processing in the David restoration , 2004, IEEE Computer Graphics and Applications.

[13]  Mandar Kulkarni,et al.  Depth inpainting by tensor voting. , 2013, Journal of the Optical Society of America. A, Optics, image science, and vision.

[14]  Guillermo Sapiro,et al.  Online dictionary learning for sparse coding , 2009, ICML '09.

[15]  Robert B. Fisher,et al.  Three-Dimensional Surface Relief Completion Via Nonparametric Techniques , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Guillermo Sapiro,et al.  Inpainting surface holes , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[17]  Marc Levoy,et al.  The digital Michelangelo project: 3D scanning of large statues , 2000, SIGGRAPH.

[18]  Paolo Cignoni,et al.  MeshLab: an Open-Source 3D Mesh Processing System , 2008, ERCIM News.

[19]  Dieter Fox,et al.  RGB-D Object Recognition: Features, Algorithms, and a Large Scale Benchmark , 2013, Consumer Depth Cameras for Computer Vision.

[20]  Leonidas J. Guibas,et al.  Example-Based 3D Scan Completion , 2005 .

[21]  Bruno A. Olshausen,et al.  Learning Sparse Representations of Depth , 2010, IEEE Journal of Selected Topics in Signal Processing.

[22]  Derek Hoiem,et al.  Indoor Segmentation and Support Inference from RGBD Images , 2012, ECCV.

[23]  Steve Marschner,et al.  Filling holes in complex surfaces using volumetric diffusion , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[24]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[25]  A. N. Rajagopalan,et al.  Harnessing self-similarity for reconstruction of large missing regions in 3D Models , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[26]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .