A Vision System for Personal Service Robots : Resilient Detection and Tracking of People

This report describes the design and implementation of a computer vision system that performs realtime people detection and tracking from stereo color video. The system, developed in the context of a project that aims to empower personal robots with advanced vision capabilities, illustrates a systematic approach to combining efficient but brittle algorithms into robust systems. The system’s tracking core builds upon established monocular algorithms for face detection and color-based tracking to perform stereo multi-target people (head) tracking. These algorithms interact in the dynamic system according to a general tracking pattern whose core is a feedback loop. Systems designed according to this pattern exhibit resilience properties, such as bootstrapping capability, robustness to input data noise, adaptability to changing environment. As a stand-alone integrated demonstration, the system incorporates video capture, processing and result visualization for evaluation purposes. It was designed using the Software Architecture for Immersipresence model. Parallel implementations on both Windows and Linux platforms, using the open source MFSM architectural middleware and the Intel OpenCV library, reliably detect and track people at interactive rates.

[1]  Harry Shum,et al.  Statistical Learning of Multi-view Face Detection , 2002, ECCV.

[2]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[3]  Alexandre R. J. François,et al.  CAMSHIFT Tracker Design Experiments With Intel OpenCV and SAI , 2004 .

[4]  Kazuhiko Kawamura,et al.  Design philosophy for service robots , 1995, 1995 IEEE International Conference on Systems, Man and Cybernetics. Intelligent Systems for the 21st Century.

[5]  Alex Pentland,et al.  Parametrized structure from motion for 3D adaptive feedback tracking of faces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Alexandre R. J. François,et al.  A hybrid architectural style for distributed parallel processing of generic data streams , 2004, Proceedings. 26th International Conference on Software Engineering.

[7]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[8]  Erik Hjelmås,et al.  Face Detection: A Survey , 2001, Comput. Vis. Image Underst..

[9]  Dorin Comaniciu,et al.  Real-time tracking of non-rigid objects using mean shift , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[10]  Illah R. Nourbakhsh,et al.  A survey of socially interactive robots , 2003, Robotics Auton. Syst..

[11]  Cynthia Breazeal,et al.  Socially intelligent robots: research, development, and applications , 2001, 2001 IEEE International Conference on Systems, Man and Cybernetics. e-Systems and e-Man for Cybernetics in Cyberspace (Cat.No.01CH37236).

[12]  Rainer Lienhart,et al.  Empirical Analysis of Detection Cascades of Boosted Classifiers for Rapid Object Detection , 2003, DAGM-Symposium.

[13]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Y. Bar-Shalom.  Tracking and data association , 1988 .

[15]  Alberto Sanfeliu,et al.  3D real-time head tracking fusing color histograms and stereovision , 2002, Object recognition supported by user interaction for service robots.

[16]  Gary Bradski,et al.  Computer Vision Face Tracking For Use in a Perceptual User Interface , 1998 .

[17]  Narendra Ahuja,et al.  Detecting Faces in Images: A Survey , 2002, IEEE Trans. Pattern Anal. Mach. Intell..