Kinect Deform: Enhanced 3D Reconstruction of Non-rigidly Deforming Objects

In this work we propose KinectDeform, an algorithm which targets enhanced 3D reconstruction of scenes containing non-rigidly deforming objects. It provides an innovation to the existing class of algorithms which either target scenes with rigid objects only or allow for very limited non-rigid deformations or use precomputed templates to track them. KinectDeform combines a fast non-rigid scene tracking algorithm based on octree data representation and hierarchical voxel associations with a recursive data filtering mechanism. We analyze its performance on both real and simulated data and show improved results in terms of smoothness and feature preserving 3D reconstructions with reduced noise.

[1]  James F. O'Brien,et al.  Folding and crumpling adaptive sheets , 2013, ACM Trans. Graph..

[2]  Hans-Peter Seidel,et al.  Lightweight binocular facial performance capture under uncontrolled lighting , 2012, ACM Trans. Graph..

[3]  Dieter Schmalstieg,et al.  OmniKinect: real-time dense volumetric data acquisition and applications , 2012, VRST '12.

[4]  Yael Moses,et al.  Multi-view scene flow estimation: A view centered variational approach , 2010, CVPR.

[5]  Hao Li,et al.  Global Correspondence Optimization for Non‐Rigid Registration of Depth Scans , 2008, Comput. Graph. Forum.

[6]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[7]  Peter-Pike J. Sloan,et al.  Interactive ray tracing for isosurface rendering , 1998 .

[8]  Daniel Cremers,et al.  Real-Time Camera Tracking and 3D Reconstruction Using Signed Distance Functions , 2013, Robotics: Science and Systems.

[9]  Ming Zeng,et al.  Octree-based fusion for realtime 3D reconstruction , 2013, Graph. Model..

[10]  Didier Stricker,et al.  KinectAvatar: Fully Automatic Body Capture Using a Single Kinect , 2012, ACCV Workshops.

[11]  Björn E. Ottersten,et al.  Depth Super-Resolution by Enhanced Shift and Add , 2013, CAIP.

[12]  Leonidas J. Guibas,et al.  Robust single-view geometry and motion reconstruction , 2009, ACM Trans. Graph..

[13]  Andreas Uhl,et al.  BlenSor: Blender Sensor Simulation Toolbox , 2011, ISVC.

[14]  Ming Zeng,et al.  Templateless Quasi-rigid Shape Modeling with Implicit Loop-Closure , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  John J. Leonard,et al.  Kintinuous: Spatially Extended KinectFusion , 2012, AAAI 2012.

[16]  Gabriel J. Brostow,et al.  Patch Based Synthesis for Single Depth Image Super-Resolution , 2012, ECCV.

[17]  Marc Neveu,et al.  Toward a real-time tracking of dense point-sampled geometry , 2012, 2012 19th IEEE International Conference on Image Processing.

[18]  Sebastian Thrun,et al.  LidarBoost: Depth superresolution for ToF 3D shape scanning , 2009, CVPR.

[19]  Tim Weyrich,et al.  Real-Time 3D Reconstruction in Dynamic Scenes Using Point-Based Fusion , 2013, 2013 International Conference on 3D Vision.

[20]  Richard Bowden,et al.  Kinecting the dots: Particle based scene flow from depth sensors , 2011, 2011 International Conference on Computer Vision.

[21]  J. Giesen,et al.  Delaunay Triangulation Based Surface Reconstruction: Ideas and Algorithms , 2004 .

[22]  Antonio L. Apolinario,et al.  KinectFusion for Faces: Real-Time 3D Face Tracking and Modeling Using a Kinect Camera for a Markerless AR System , 2013 .

[23]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[24]  Marsette Vona,et al.  Moving Volume KinectFusion , 2012, BMVC.

[25]  James F. O'Brien,et al.  Adaptive anisotropic remeshing for cloth simulation , 2012, ACM Trans. Graph..

[26]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[27]  Daniel Cremers,et al.  Large-Scale Multi-resolution Surface Reconstruction from RGB-D Sequences , 2013, 2013 IEEE International Conference on Computer Vision.

[28]  Marc Levoy,et al.  Real-time 3D model acquisition , 2002, ACM Trans. Graph..

[29]  Daniel Cremers,et al.  CopyMe3D: Scanning and Printing Persons in 3D , 2013, GCPR.

[30]  Joachim Giesen,et al.  Delaunay Triangulation Based Surface Reconstruction , 2006 .

[31]  Andrew W. Fitzgibbon,et al.  Real-time non-rigid reconstruction using an RGB-D camera , 2014, ACM Trans. Graph..

[32]  Radu Horaud,et al.  Scene flow estimation by growing correspondence seeds , 2011, CVPR 2011.

[33]  Matthias Nießner,et al.  Real-time 3D reconstruction at scale using voxel hashing , 2013, ACM Trans. Graph..

[34]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.