A Multiple Camera System with Real-Time Volume Reconstruction for Articulated Skeleton Pose Tracking

We present a multi-camera system for recovering skeleton body pose, by performing real-time volume reconstruction and using a hierarchical stochastic pose search algorithm. Different from many multicamera systems that require a few connected workstations, our system only uses a signle PC to control 8 cameras for synchronous image acquisition. Silhouettes of the 8 cameras are extracted via a color-based background subtraction algorithm, and set as input to the 3D volume reconstruction. Our system can perform real-time volume reconstruction rendered in point clouds, voxels as well as voxels with texturing. The full-body skeleton pose (29-D vector) is then recovered by fitting an articulated body model to the volume sequences. The pose estimation is performed in a hierarchical manner, by using a particle swarm optmization (PSO) based search strategy combined with soft constraints. 3D distance transform (DT) is used for reducing the computing time of objective evaluations.

[1]  Luc Van Gool,et al.  Full body tracking from multiple views using stochastic sampling , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[2]  Nassir Navab,et al.  Efficient visual hull computation for real-time 3D reconstruction using CUDA , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[3]  Toby Howard,et al.  Real-Time 3-D Human Body Tracking using Variable Length Markov Models , 2005, BMVC.

[4]  Hans-Peter Seidel,et al.  Performance capture from sparse multi-view video , 2008, SIGGRAPH 2008.

[5]  Mohan M. Trivedi,et al.  Human Body Model Acquisition and Tracking Using Voxel Data , 2003, International Journal of Computer Vision.

[6]  Wojciech Matusik,et al.  Polyhedral Visual Hulls for Real-Time Rendering , 2001, Rendering Techniques.

[7]  Katsushi Ikeuchi,et al.  Marker-less Human Motion Estimation using Articulated Deformable Model , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[8]  Emanuele Trucco,et al.  Human Body Pose Estimation with Particle Swarm Optimisation , 2008, Evolutionary Computation.

[9]  Neil A. Thacker,et al.  Real-time Body Tracking Using a Gaussian Process Latent Variable Model , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[10]  Saïda Bouakaz,et al.  Real-Time Marker-free Motion Capture from multiple cameras , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[11]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[12]  Adrian Hilton,et al.  Model-based multiple view reconstruction of people , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[13]  Adrian Hilton,et al.  Surface Capture for Performance-Based Animation , 2007, IEEE Computer Graphics and Applications.

[14]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Michael N. Vrahatis,et al.  Recent approaches to global optimization problems through Particle Swarm Optimization , 2002, Natural Computing.

[16]  Takeo Kanade,et al.  A real time system for robust 3D voxel reconstruction of human motions , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[17]  Hans-Peter Seidel,et al.  A Flexible and Versatile Studio for Synchronized Multi-View Video Recording , 2003, VVG.

[18]  A. Laurentini,et al.  The Visual Hull Concept for Silhouette-Based Image Understanding , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Takeo Kanade,et al.  Shape-from-silhouette of articulated objects and its use for human body kinematics estimation and motion capture , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[20]  Luc Van Gool,et al.  Markerless tracking of complex human motions from multiple views , 2006, Comput. Vis. Image Underst..

[21]  Emanuele Trucco,et al.  Human Body Pose Estimation with PSO , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[22]  Richard Szeliski,et al.  Rapid octree construction from image sequences , 1993 .

[23]  Hans-Peter Seidel,et al.  Combining 2D feature tracking and volume reconstruction for online video-based human motion capture , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[24]  Hans-Peter Seidel,et al.  Free-viewpoint video of human actors , 2003, ACM Trans. Graph..

[25]  Kazuhiko Takahashi,et al.  Remarks on 3D human body’s feature extraction from voxel reconstruction of human body posture , 2007, 2007 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[26]  Julius Ziegler,et al.  Tracking of the Articulated Upper Body on Multi-View Stereo Image Sequences , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[27]  Stefano Corazza,et al.  Accurately measuring human movement using articulated ICP with soft-joint constraints and a repository of articulated models , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Radu Horaud,et al.  Tracking with the Kinematics of Extremal Contours , 2006, ACCV.