Skeleton-Based Data Compression for Multi-camera Tele-Immersion System

Image-based full body 3D reconstruction for tele-immersive applications generates large amount of data points, which have to be sent through the network in real-time. In this paper we introduce a skeleton-based compression method using motion estimation where kinematic parameters of the human body are extracted from the point cloud data in each frame. First we address the issues regarding the data capturing and transfer to a remote site for the tele-immersive collaboration. We compare the results of the existing compression methods and the proposed skeleton-based compression technique. We examine robustness and efficiency of the algorithm through experimental results with our multicamera tele-immersion system. The proposed skeleton-based method provides high and flexible compression ratios (from 50:1 to 5000:1) with reasonable reconstruction quality (peak signal-to-noise ratio from 28 to 31 dB).

[1]  Thomas Malzbender,et al.  The Coliseum Immersive Teleconferencing System , 2002 .

[2]  Markus H. Gross,et al.  3D video fragments: dynamic point samples for real-time free-viewpoint video , 2004, Comput. Graph..

[3]  Dariu Gavrila,et al.  The Visual Analysis of Human Movement: A Survey , 1999, Comput. Vis. Image Underst..

[4]  Massimo Piccardi,et al.  Background subtraction techniques: a review , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[5]  Kostas Daniilidis,et al.  Real time trinocular stereo for tele-immersion , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[6]  Yoshihiko Nomura,et al.  Error analysis and optimization of camera calibration , 1991, Proceedings IROS '91:IEEE/RSJ International Workshop on Intelligent Robots and Systems '91.

[7]  Daniel Thalmann,et al.  Real-Time Animation of Realistic Virtual Humans , 1998, IEEE Computer Graphics and Applications.

[8]  Wenyi Zhao,et al.  Effects of camera alignment errors on stereoscopic depth estimates , 1996, Pattern Recognit..

[9]  Ruzena Bajcsy,et al.  A Framework for Constructing Real-time Immersive Environments for Training Physical Activities , 2006, J. Multim..

[10]  Ajay Luthra,et al.  Overview of the H.264/AVC video coding standard , 2003, IEEE Trans. Circuits Syst. Video Technol..

[11]  Ruzena Bajcsy,et al.  The Effects of Fully Immersive Virtual Reality on the Learning of Physical Tasks , 2006 .

[12]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  J. Lanier.  Virtually there. , 2001, Scientific American.

[14]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[15]  Pascal Fua,et al.  Skeleton-based motion capture for robust reconstruction of human motion , 2000, Proceedings Computer Animation 2000.

[16]  Ruzena Bajcsy,et al.  Skeleton-Based Compression of 3-D Tele-Immersion Data , 2007, 2007 First ACM/IEEE International Conference on Distributed Smart Cameras.

[17]  Jake K. Aggarwal,et al.  Human Motion Analysis: A Review , 1999, Comput. Vis. Image Underst..

[18]  William Yurcik,et al.  Real-time 3D video compression for tele-immersive environments , 2006, Electronic Imaging.

[19]  Hans-Peter Seidel,et al.  Multi-Layer Skeleton Fitting for Online Human Motion Capture , 2002, VMV.