KinectAvatar: Fully Automatic Body Capture Using a Single Kinect

We present a novel scanning system for capturing a full 3D human body model using just a single depth camera and no auxiliary equipment. We claim that data captured from a single Kinect is sufficient to produce a good quality full 3D human model. In this setting, the challenges we face are the sensor's low resolution with random noise and the subject's non-rigid movement when capturing the data. To overcome these challenges, we develop an improved super-resolution algorithm that takes color constraints into account. We then align the super-resolved scans using a combination of automatic rigid and non-rigid registration. As the system is of low price and obtains impressive results in several minutes, full 3D human body scanning technology can now become more accessible to everyday users at home.

[1]  Didier Stricker,et al.  3D shape scanning with a Kinect , 2011, SIGGRAPH '11.

[2]  Luc Van Gool,et al.  Online loop closure for real-time interactive 3D scanning , 2011, Comput. Vis. Image Underst..

[3]  Robert Bergevin,et al.  Towards a General Multi-View Registration Technique , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Dieter Fox,et al.  RGB-D Mapping: Using Depth Cameras for Dense 3D Modeling of Indoor Environments , 2010, ISER.

[5]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[6]  Matthias Zwicker,et al.  Global registration of dynamic range scans for articulated model reconstruction , 2011, TOGS.

[7]  Wolfgang Heidrich,et al.  Globally Consistent Space‐Time Reconstruction , 2010, Comput. Graph. Forum.

[8]  Bodo Rosenhahn,et al.  Pose estimation revisited , 2006 .

[9]  Ligang Liu,et al.  Scanning 3D Full Human Bodies Using Kinects , 2012, IEEE Transactions on Visualization and Computer Graphics.

[10]  Dietmar Hildenbrand,et al.  Pose estimation based on Geometric Algebra , 2009 .

[11]  Sebastian Thrun,et al.  3D shape scanning with a time-of-flight camera , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  Leonidas J. Guibas,et al.  Dynamic geometry registration , 2007, Symposium on Geometry Processing.

[13]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Richard M. Murray,et al.  A Mathematical Introduction to Robotic Manipulation , 1994 .

[15]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[16]  Sebastian Thrun,et al.  SCAPE: shape completion and animation of people , 2005, SIGGRAPH 2005.

[17]  Sebastian Thrun,et al.  LidarBoost: Depth superresolution for ToF 3D shape scanning , 2009, CVPR.

[18]  J. Weickert,et al.  Visualization and Processing of Tensor Fields (Mathematics and Visualization) , 2005 .

[19]  Miguel Á. Carreira-Perpiñán,et al.  Non-rigid point set registration: Coherent Point Drift , 2006, NIPS.

[20]  Francis Schmitt,et al.  A Solution for the Registration of Multiple 3D Point Sets Using Unit Quaternions , 1998, ECCV.

[21]  Hans-Peter Seidel,et al.  Efficient reconstruction of nonrigid shape and motion from real-time 3D scanner data , 2009, TOGS.

[22]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Maher Moakher,et al.  Visualization and Processing of Tensor Fields , 2006, Mathematics and Visualization.

[24]  Pieter Peers,et al.  Temporally coherent completion of dynamic shapes , 2012, TOGS.

[25]  Leonidas J. Guibas,et al.  Non-Rigid Registration Under Isometric Deformations , 2008 .

[26]  Hans-Peter Seidel,et al.  Animation cartography—intrinsic reconstruction of shape and motion , 2012, TOGS.

[27]  Didier Stricker,et al.  Efficient Packing of Arbitrary Shaped Charts for Automatic Texture Atlas Generation , 2011, EGSR '11.

[28]  Hans Hagen,et al.  04172 Abstracts Collection - Perspectives Workshop: Visualization and Image Processing of Tensor Fields , 2004, Perspectives Workshop: Visualization and Image Processing of Tensor Fields.

[29]  Michael J. Black,et al.  Home 3D body scans from noisy image and range data , 2011, 2011 International Conference on Computer Vision.

[30]  Michael M. Kazhdan,et al.  Poisson surface reconstruction , 2006, SGP '06.

[31]  Bernd Neumann,et al.  Computer Vision — ECCV’98 , 1998, Lecture Notes in Computer Science.

[32]  Pedro V. Sander,et al.  Multi-Chart Geometry Images , 2003, Symposium on Geometry Processing.

[33]  Didier Stricker,et al.  Algorithms for 3D Shape Scanning with a Depth Camera , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.