Supervoxels for human and clothes in RGB-D image

In order to reconstruct human body and remodel clothes in virtual try on and size-fitting scenario, we preprocess the input point cloud into supervoxels through over-segmentation. Over-segmentation of an image into superpixels is a widely used preprocessing approach in image segmentation. Recently, over-segmentation algorithm, which clusters points with similar feature into a small group, known as supervoxel here, has been used on RGB-D image or the combination of observations from several calibrated RGB-D cameras to reduce occurrence of clusters crossing object boundaries. This method makes use of both color information and the 3D geometric relationships to describe features and segmented regions. We apply this method to RGB-D images of human body and clothes, so that largely reduce the number of regions that must be considered in following remodel process. Experiments were taken on point clouds captured by Kinect. Because the final goal is to predict object boundaries that represent different part of human and clothes, we further compare the recall with SLIC method for boundaries we have marked previously, and give a suggestion on choosing a better seeding radius in our scenario.

[1]  Jitendra Malik,et al.  Recovering human body configurations: combining segmentation and recognition , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[2]  FuaPascal,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012 .

[3]  Nadia Magnenat-Thalmann,et al.  Real‐time Animation of Dressed Virtual Humans , 2002, Comput. Graph. Forum.

[4]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[6]  Florentin Wörgötter,et al.  Voxel Cloud Connectivity Segmentation - Supervoxels for Point Clouds , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Nadia Magnenat-Thalmann,et al.  A simple approach to nonlinear tensile stiffness for accurate cloth simulation , 2009, TOGS.

[8]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[9]  Guillermo Sapiro,et al.  Interactive Image Segmentation via Adaptive Weighted Distances , 2007, IEEE Transactions on Image Processing.

[10]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[11]  Ligang Liu,et al.  Scanning 3D Full Human Bodies Using Kinects , 2012, IEEE Transactions on Visualization and Computer Graphics.

[12]  Sven J. Dickinson,et al.  TurboPixels: Fast Superpixels Using Geometric Flows , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Paria Mehrani,et al.  Superpixels and Supervoxels in an Energy Optimization Framework , 2010, ECCV.

[14]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..