Three-dimensional tracking of multiple skin-colored regions by a moving stereoscopic system.

A system that performs three-dimensional (3D) tracking of multiple skin-colored regions (SCRs) in images acquired by a calibrated, possibly moving stereoscopic rig is described. The system consists of a collection of techniques that permit the modeling and detection of SCRs, the determination of their temporal association in monocular image sequences, the establishment of their correspondence between stereo images, and the extraction of their 3D positions in a world-centered coordinate system. The development of these techniques has been motivated by the need for robust, near-real-time tracking performance. SCRs are detected by use of a Bayesian classifier that is trained with the aid of a novel technique. More specifically, the classifier is bootstrapped with a small set of training data. Then, as new images are being processed, an iterative training procedure is employed to refine the classifier. Furthermore, a technique is proposed to enable the classifier to cope with changes in illumination. Tracking of SCRs in time as well as matching of SCRs in the images of the employed stereo rig is performed through computationally inexpensive and robust techniques. One of the main characteristics of the skin-colored region tracker (SCRT) instrument is its ability to report the 3D positions of SCRs in a world-centered coordinate system by employing a possibly moving stereo rig with independently verging CCD cameras. The system operates on images of dimensions 640 x 480 pixels at a rate of 13 Hz on a conventional Pentium 4 processor at 1.8 GHz. Representative experimental results from the application of the SCRT to image sequences are also provided.

[1]  Hiromasa Nakatani,et al.  Efficient Search Technique for Hand Gesture Tracking in Three Dimensions , 2000, Biologically Motivated Computer Vision.

[2]  Alex Pentland,et al.  Mixtures of eigenfeatures for real-time structure from texture , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[3]  Patrick Pérez,et al.  Color-Based Probabilistic Tracking , 2002, ECCV.

[4]  Michael J. Black,et al.  Automatic Detection and Tracking of Human Motion with a View-Based Representation , 2002, ECCV.

[5]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[6]  Heiko Hirschm.  Improvements in Real-Time Correlation-Based Stereo Vision , 2001 .

[7]  James M. Rehg,et al.  Statistical Color Models with Application to Skin Detection , 2004, International Journal of Computer Vision.

[8]  Richard A. Foulds,et al.  Toward robust skin identification in video images , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[9]  Narendra Ahuja,et al.  Face Detection and Gesture Recognition for Human-Computer Interaction , 2001, The International Series in Video Computing.

[10]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[11]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Michael Isard,et al.  ICONDENSATION: Unifying Low-Level and High-Level Tracking in a Stochastic Framework , 1998, ECCV.

[13]  Olivier D. Faugeras,et al.  3D Articulated Models and Multiview Tracking with Physical Forces , 2001, Comput. Vis. Image Underst..

[14]  Narendra Ahuja,et al.  Detecting Faces in Images: A Survey , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Dariu Gavrila,et al.  The Visual Analysis of Human Movement: A Survey , 1999, Comput. Vis. Image Underst..

[16]  Mubarak Shah,et al.  Tracking and Object Classification for Automated Surveillance , 2002, ECCV.

[17]  Michael Isard,et al.  BraMBLe: a Bayesian multiple-blob tracker , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[18]  Alexandru Tupan,et al.  Triangulation , 1997, Comput. Vis. Image Underst..

[19]  Ron Goldman,et al.  Intersection of two lines in three-space , 1990 .

[20]  A. Ardeshir Goshtasby,et al.  Detecting human faces in color images , 1998, Image Vis. Comput..

[21]  Stephen J. Maybank,et al.  Fusion of Multiple Tracking Algorithms for Robust People Tracking , 2002, ECCV.

[22]  Zhengyou Zhang,et al.  Determining the Epipolar Geometry and its Uncertainty: A Review , 1998, International Journal of Computer Vision.

[23]  Shaogang Gong,et al.  Tracking colour objects using adaptive mixture models , 1999, Image Vis. Comput..

[24]  Bernt Schiele,et al.  Multi-object Tracking Based on a Modular Knowledge Hierarchy , 2003, ICVS.

[25]  Keith Jack,et al.  Video Demystified: A Handbook for the Digital Engineer , 1993 .

[26]  Yi Li,et al.  A relaxation algorithm for real-time multiple view 3D-tracking , 2002, Image Vis. Comput..

[27]  Sang Chul Ahn,et al.  Object oriented face detection using range and color information , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[28]  Alex Pentland,et al.  Parametrized structure from motion for 3D adaptive feedback tracking of faces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[29]  Esther Koller-Meier,et al.  Tracking multiple objects using the Condensation algorithm , 2001, Robotics Auton. Syst..

[30]  Patrick Pérez,et al.  Towards Improved Observation Models for Visual Tracking: Selective Adaptation , 2002, ECCV.

[31]  Shaogang Gong,et al.  Tracking and segmenting people in varying lighting conditions using colour , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[32]  Jochen Triesch,et al.  Democratic Integration: Self-Organized Integration of Adaptive Cues , 2001, Neural Computation.

[33]  Shigeru Akamatsu,et al.  Comparative performance of different skin chrominance models and chrominance spaces for the automatic detection of human faces in color images , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[34]  Patrick Pérez,et al.  Sequential Monte Carlo methods for multiple target tracking and data fusion , 2002, IEEE Trans. Signal Process..

[35]  Dorin Comaniciu,et al.  Real-time tracking of non-rigid objects using mean shift , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[36]  Frank Biocca,et al.  A Survey of Position Trackers , 1992, Presence: Teleoperators & Virtual Environments.