Virtual Space Teleconferencing: Real-Time Reproduction of 3D Human Images

Abstract Real-time reproduction of 3D human images is realized by an experimental system recently built as a prototype for virtual space teleconferencing, in which participants at different sites can feel as if they are colocated and can work cooperatively. At each sending and receiving site of the teleconferencing system, a 3D model of each participant is constructed from a wire frame model mapped by color texture and is rendered on a 3D display. In the current experimental system, real-time detection of facial features at the sending site is achieved by visually tracking tape marks pasted to the participant's face. Movements of the head, body, hands, and fingers are detected in real time using magnetic sensors and data gloves. At the receiving site, the detected motion parameters are used to move nodes in the wire frame model to reproduce the movements of the participants at each sending site. Realistic facial expressions are reproduced through the use of simple motion rules applied to the tape mark tracking information. Through experimental evaluation, the optimal number of nodes for best quality has been obtained. Reproduction of facial expressions and synthesis of arm movements are examined. The reproduction speed using the optimal human model is approximately 6 frames/s. Examples of cooperative work by participants using the experimental system illustrate the effectiveness of virtual space teleconferencing.