Generating High-Definition Facial Video for Shared Mixed Reality

We propose a scheme to recover the eye-contact between multiple users in a Shared Mixed-Reality space. The eye-contact in a collaborating mixed-reality space is lost as a side efect of wearing head-mounted displays (HMDs). We developed a method to synthesize 3D facial video in order to recover the eye-contact lost by an HMD. Our method can synthesize facial video with arbitrary posture and arbitrary sight-line including moving eyelids in real time by using a 3D scanner and a high-resolution digital camera.