3D Human Reconstruction from an Image for Mobile Telepresence Systems

In remote collaboration using a telepresence system, viewing a 3D-reconstructed workspace can help a person understand the workspace situation. This is because, by viewing the workspace from birds-eye view, it is possible to both understand the positional relationship between the workspace and the worker and see the movements of workers. However, when reconstructing the workspace in 3D using depth cameras mounted on a telepresence robot or worker, the back of workers or objects in front of the depth cameras becomes a blind spot and cannot be captured. Therefore, when a person in a remote location views the 3D-reconstructed workspace, the person cannot see workers or objects in the blind spot. This makes it difficult to know the positional relationship between the workspace and the worker and to see the movement of the workers. Here, we propose a method to reduce the blind spots of a 3D-reconstructed person by reconstructing a 3D model of the person from an RGB-D image taken with a depth camera and changing the pose of the 3D model according to the movement of the persons body.

[1]  Dieter Fox,et al.  DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Chu-Song Chen,et al.  MVC: A Dataset for View-Invariant Clothing Retrieval and Attribute Prediction , 2016, ICMR.

[3]  Ran He,et al.  Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[4]  Jun Rekimoto,et al.  JackIn space: designing a seamless transition between first and third person view for effective telepresence collaborations , 2017, AH.

[5]  Marcus A. Magnor,et al.  Tex2Shape: Detailed Full Human Body Geometry From a Single Image , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[6]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).