A Data-Driven Approach for 3D Human Body Pose Reconstruction from a Kinect Sensor

In the study of virtual fitting techniques, human body modeling has always occupied a very important position. Whether a model that is roughly consistent with users can be established has a direct impact on the fitting experience of users. Based on this, we propose an automatic human body pose registration algorithm which can efficiently construct a posed model using priori data and a single Kinect sensor, and provide a good foundation for the later shape registration. Because of the complexity of the human body, there are so many methods that just reconstruct a human body model but no rigged animation skeleton inside. To solve this, we use SMPL which is a recently published statistical body model to fit 3D joints acquired by a Kinect sensor. And finally we project the posed model to the corresponding person in the color image to improve the fitting experience. Our experiments show that the speed and the estimation error of the algorithm are within the tolerance of virtual fitting.

[1]  Marcus A. Magnor,et al.  Video Based Reconstruction of 3D People Models , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[2]  Ralph R. Martin,et al.  Parametric modeling of 3D human body shape - A survey , 2017, Comput. Graph..

[3]  Mamoru Sato ’Scape , 2017, Encyclopedia of Food and Agricultural Ethics.

[4]  Michel Sarkis,et al.  Parametric Human Body Reconstruction Based on Sparse Key Points , 2016, IEEE Transactions on Visualization and Computer Graphics.

[5]  Bodo Rosenhahn,et al.  Human Pose Estimation from Video and IMUs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Ralph R. Martin,et al.  Realtime Reconstruction of an Animating Human Body from a Single Depth Camera , 2016, IEEE Transactions on Visualization and Computer Graphics.

[7]  Peter V. Gehler,et al.  Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image , 2016, ECCV.

[8]  Michael J. Black,et al.  SMPL: A Skinned Multi-Person Linear Model , 2023 .

[9]  Michael J. Black,et al.  Dyna: a model of dynamic human shape in motion , 2015, ACM Trans. Graph..

[10]  Liujuan Cao,et al.  Estimation of human body shape and cloth field in front of a kinect , 2015, Neurocomputing.

[11]  Yang Li,et al.  Measuring Accurate Body Parameters of Dressed Humans with Large-Scale Motion Using a Kinect Sensor , 2013, Sensors.

[12]  Michael J. Black,et al.  Coregistration: Simultaneous Alignment and Modeling of Articulated 3D Shape , 2012, ECCV.

[13]  Michael J. Black,et al.  Home 3D body scans from noisy image and range data , 2011, 2011 International Conference on Computer Vision.

[14]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[15]  Hans-Peter Seidel,et al.  MovieReshape: tracking and reshaping of humans in videos , 2010, ACM Trans. Graph..

[16]  Dragomir Anguelov,et al.  SCAPE: shape completion and animation of people , 2005, ACM Trans. Graph..

[17]  Chin-Hui Lee,et al.  Maximum a posteriori estimation for multivariate Gaussian mixture observations of Markov chains , 1994, IEEE Trans. Speech Audio Process..

[18]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..