Shape Completion from a Single RGBD Image

We present a novel approach for constructing a complete 3D model for an object from a single RGBD image. Given an image of an object segmented from the background, a collection of 3D models of the same category are non-rigidly aligned with the input depth, to compute a rough initial result. A volumetric-patch-based optimization algorithm is then performed to refine the initial result to generate a 3D model that not only is globally consistent with the overall shape expected from the input image but also possesses geometric details similar to those in the input image. The optimization with a set of high-level constraints, such as visibility, surface confidence and symmetry, can achieve more robust and accurate completion over state-of-the art techniques. We demonstrate the efficiency and robustness of our approach with multiple categories of objects with various geometries and details, including busts, chairs, bikes, toys, vases and tables.

[1]  Hao Li,et al.  Realtime performance-based facial animation , 2011, ACM Trans. Graph..

[2]  Vladlen Koltun,et al.  Color map optimization for 3D reconstruction with consumer depth cameras , 2014, ACM Trans. Graph..

[3]  Alla Sheffer,et al.  Analogy‐driven 3D style transfer , 2014, Comput. Graph. Forum.

[4]  M. Pauly,et al.  Embedded deformation for shape manipulation , 2007, SIGGRAPH 2007.

[5]  Sylvain Lefebvre,et al.  State of the Art in Example-based Texture Synthesis , 2009, Eurographics.

[6]  Daniel Cohen-Or,et al.  Active co-analysis of a set of shapes , 2012, ACM Trans. Graph..

[7]  Sebastian Thrun,et al.  SCAPE: shape completion and animation of people , 2005, SIGGRAPH 2005.

[8]  Eitan Grinspun,et al.  Context-based coherent surface completion , 2014, ACM Trans. Graph..

[9]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[10]  Jianxiong Xiao,et al.  3D ShapeNets: A deep representation for volumetric shapes , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Daniel Cohen-Or,et al.  Non-local scan consolidation for 3D urban scenes , 2010, ACM Trans. Graph..

[12]  Derek Hoiem,et al.  Completing 3D object shape from one depth image , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Marc Alexa,et al.  Context-based surface completion , 2004, ACM Trans. Graph..

[14]  D. Cohen-Or,et al.  Context-based surface completion , 2004, SIGGRAPH 2004.

[15]  Shi-Min Hu,et al.  Structure recovery by part assembly , 2012, ACM Trans. Graph..

[16]  Ligang Liu,et al.  Multi-scale partial intrinsic symmetry detection , 2012, ACM Trans. Graph..

[17]  Lei Zheng,et al.  Image Noise Level Estimation by Principal Component Analysis , 2013, IEEE Transactions on Image Processing.

[18]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[19]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[20]  Eli Shechtman,et al.  Space-Time Completion of Video , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Adam Finkelstein,et al.  The Generalized PatchMatch Correspondence Algorithm , 2010, ECCV.

[22]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[23]  Eli Shechtman,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, ACM Trans. Graph..

[24]  Eli Shechtman,et al.  Image melding , 2012, ACM Trans. Graph..

[25]  Michael M. Kazhdan,et al.  Screened poisson surface reconstruction , 2013, TOGS.

[26]  Kun Zhou,et al.  An interactive approach to semantic modeling of indoor scenes with an RGBD camera , 2012, ACM Trans. Graph..

[27]  Qing Zhang,et al.  Edge-preserving photometric stereo via depth fusion , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Kun Zhou,et al.  Imagining the unseen , 2014, ACM Trans. Graph..

[29]  H. Seidel,et al.  Multi-level partition of unity implicits , 2003 .

[30]  Leonidas J. Guibas,et al.  Estimating image depth using shape collections , 2014, ACM Trans. Graph..

[31]  D. Cohen-Or,et al.  Upright orientation of man-made objects , 2008, SIGGRAPH 2008.

[32]  Ligang Liu,et al.  Scanning 3D Full Human Bodies Using Kinects , 2012, IEEE Transactions on Visualization and Computer Graphics.

[33]  Alexei A. Efros,et al.  Image quilting for texture synthesis and transfer , 2001, SIGGRAPH.

[34]  Michael J. Black,et al.  Home 3D body scans from noisy image and range data , 2011, 2011 International Conference on Computer Vision.

[35]  Reinhard Klein,et al.  Efficient RANSAC for Point‐Cloud Shape Detection , 2007, Comput. Graph. Forum.

[36]  Kun Zhou,et al.  Interactive images , 2012, ACM Trans. Graph..

[37]  Ariel Shamir,et al.  Cone carving for surface reconstruction , 2010, SIGGRAPH 2010.

[38]  Alla Sheffer,et al.  Template-based mesh completion , 2005, SGP '05.

[39]  In-So Kweon,et al.  High Quality Shape from a Single RGB-D Image under Uncalibrated Natural Illumination , 2013, 2013 IEEE International Conference on Computer Vision.

[40]  居涛.  Fixing Geometric Errors on Polygonal Models： A Survey , 2009 .

[41]  Leonidas J. Guibas,et al.  Example-Based 3D Scan Completion , 2005 .

[42]  Leif Kobbelt,et al.  Automatic restoration of polygon models , 2005, TOGS.

[43]  Ronald Fedkiw,et al.  Level set methods and dynamic implicit surfaces , 2002, Applied mathematical sciences.