iHuman3D: Intelligent Human Body 3D Reconstruction using a Single Flying Camera

Aiming at autonomous, adaptive and real-time human body reconstruction technique, this paper presents iHuman3D: an intelligent human body 3D reconstruction system using a single aerial robot integrated with an RGB-D camera. Specifically, we propose a real-time and active view planning strategy based on a highly efficient ray casting algorithm in GPU and a novel information gain formulation directly in TSDF. We also propose the human body reconstruction module by revising the traditional volumetric fusion pipeline with a compactly-designed non-rigid deformation for slight motion of the human target. We unify both the active view planning and human body reconstruction in the same TSDF volume-based representation. Quantitative and qualitative experiments are conducted to validate that the proposed iHuman3D system effectively removes the constraint of extra manual labor, enabling real-time and autonomous reconstruction of human body.

[1]  Jacopo Aleotti,et al.  Contour-based next-best view planning from point cloud segmentation of unknown objects , 2018, Auton. Robots.

[2]  Guofeng Zhang,et al.  Templateless Non-Rigid Reconstruction and Motion Tracking With a Single RGB-D Camera , 2017, IEEE Transactions on Image Processing.

[3]  Imran Khan,et al.  Robust Sparse and Dense Nonrigid Structure From Motion , 2018, IEEE Transactions on Multimedia.

[4]  Didier Stricker,et al.  3D shape scanning with a Kinect , 2011, SIGGRAPH '11.

[5]  Lu Fang,et al.  Guidance: A visual sensing platform for robotic applications , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[6]  Zoran Popovic,et al.  The space of human body shapes: reconstruction and parameterization from range scans , 2003, ACM Trans. Graph..

[7]  Daniel Cremers,et al.  Dense visual SLAM for RGB-D cameras , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[8]  Cristian Sminchisescu,et al.  Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Cordelia Schmid,et al.  Learning from Synthetic Humans , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Martin Svensson,et al.  Efficient algorithms for Next Best View evaluation , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[11]  Mark E. Campbell,et al.  An Adaptable, Probabilistic, Next-Best View Algorithm for Reconstruction of Unknown 3-D Objects , 2017, IEEE Robotics and Automation Letters.

[12]  Michael Suppa,et al.  Efficient next-best-scan planning for autonomous 3D surface reconstruction of unknown objects , 2013, Journal of Real-Time Image Processing.

[13]  Alan Yuille,et al.  Active Vision , 2014, Computer Vision, A Reference Guide.

[14]  Rafael Murrieta-Cid,et al.  Volumetric Next-best-view Planning for 3D Object Reconstruction with Positioning Error , 2014 .

[15]  Hans-Peter Seidel,et al.  Performance capture from sparse multi-view video , 2008, ACM Trans. Graph..

[16]  C. Ian Connolly,et al.  The determination of next best views , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[17]  Jinxiang Chai,et al.  Accurate realtime full-body motion capture using a single depth camera , 2012, ACM Trans. Graph..

[18]  Yiannis Aloimonos,et al.  Active vision , 2004, International Journal of Computer Vision.

[19]  Wolfram Burgard,et al.  OctoMap: an efficient probabilistic 3D mapping framework based on octrees , 2013, Autonomous Robots.

[20]  Ligang Liu,et al.  Scanning 3D Full Human Bodies Using Kinects , 2012, IEEE Transactions on Visualization and Computer Graphics.

[21]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[22]  Brian Yamauchi,et al.  A frontier-based approach for autonomous exploration , 1997, Proceedings 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation CIRA'97. 'Towards New Computational Principles for Robotics and Automation'.

[23]  Shengyong Chen,et al.  Vision sensor planning for 3-D model acquisition , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[24]  G. Roth,et al.  View planning for automated three-dimensional object reconstruction and inspection , 2003, CSUR.

[25]  Hans-Peter Seidel,et al.  Markerless Motion Capture with unsynchronized moving cameras , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Michael J. Black,et al.  SMPL: A Skinned Multi-Person Linear Model , 2023 .

[27]  Qionghai Dai,et al.  FlyCap: Markerless Motion Capture Using Multiple Autonomous Flying Cameras , 2016, IEEE Transactions on Visualization and Computer Graphics.

[28]  Hans P. Moravec,et al.  High resolution maps from wide angle sonar , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[29]  Stefan Leutenegger,et al.  ElasticFusion: Real-time dense SLAM and light source estimation , 2016, Int. J. Robotics Res..

[30]  Davide Scaramuzza,et al.  An information gain formulation for active volumetric 3D reconstruction , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[31]  Fei Gao,et al.  Online quadrotor trajectory generation and autonomous navigation on point clouds , 2016, 2016 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR).

[32]  Leonidas J. Guibas,et al.  Robust single-view geometry and motion reconstruction , 2009, ACM Trans. Graph..

[33]  Hans-Peter Seidel,et al.  A data-driven approach for real-time full body pose reconstruction from a depth camera , 2011, 2011 International Conference on Computer Vision.

[34]  Shengyong Chen,et al.  Active vision in robotic systems: A survey of recent developments , 2011, Int. J. Robotics Res..

[35]  B. Prabhakaran,et al.  Learning-based objective evaluation of 3D human open meshes , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[36]  Davide Scaramuzza,et al.  A comparison of volumetric information gain metrics for active 3D object reconstruction , 2017, Autonomous Robots.

[37]  Richard Pito,et al.  A Solution to the Next Best View Problem for Automated Surface Acquisition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Qionghai Dai,et al.  Performance Capture of Interacting Characters with Handheld Kinects , 2012, ECCV.

[39]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.