Head tracking using stereo

Abstract. Head tracking is an important primitive for smart environments and perceptual user interfaces where the poses and movements of body parts need to be determined. Most previous solutions to this problem are based on intensity images and, as a result, suffer from a host of problems including sensitivity to background clutter and lighting variations. Our approach avoids these pitfalls by using stereo depth data together with a simple human-torso model to create a head-tracking system that is both fast and robust. We use stereo data (Commercial equipment and materials are identified in order to adequately specify certain procedures. In no case does such identification imply recommendation or endorsement by the National Institute of Standards and Technology, nor does it imply that the materials or equipment identified are necessarily the best available for the purpose.) to derive a depth model of the background that is then employed to provide accurate foreground segmentation. We then use directed local edge detectors on the foreground to find occluding edges that are used as features to fit to a torso model. Once we have the model parameters, the location and orientation of the head can be easily estimated. A useful side effect from using stereo data is the ability to track head movement through a room in three dimensions. Experimental results on real image sequences are given.

[1]  Daniel P. Huttenlocher,et al.  Image segmentation using local variation , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[2]  Edward J. Wegman.  Affordable environments for 3D collaborative data visualization , 2000, Comput. Sci. Eng..

[3]  Marco La Cascia,et al.  Fast, Reliable Head Tracking under Varying Illumination: An Approach Based on Registration of Texture-Mapped 3D Models , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Henry Sowizral,et al.  The Java 3D API and Virtual Reality , 1999, IEEE Computer Graphics and Applications.

[5]  Alex Pentland,et al.  Coding, Analysis, Interpretation, and Recognition of Facial Expressions , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Michael T. Heath,et al.  Scientific Computing , 2018 .

[7]  Shaogang Gong,et al.  Tracking and segmenting people in varying lighting conditions using colour , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[8]  Michael Isard,et al.  Contour Tracking by Stochastic Propagation of Conditional Density , 1996, ECCV.

[9]  Alex Pentland,et al.  Modeling, tracking and interactive animation of faces and heads//using input from video , 1996, Proceedings Computer Animation '96.

[10]  Jochen Triesch.  Self-organized integration of adaptive visual cues for face tracking , 2000, SPIE Defense + Commercial Sensing.

[11]  Nebojsa Jojic,et al.  Tracking articulated self - occluding objects in dense disparity maps , 1999 .

[12]  S. Birchfield,et al.  An elliptical head tracker , 1997, Conference Record of the Thirty-First Asilomar Conference on Signals, Systems and Computers (Cat. No.97CB36136).

[13]  Alex Pentland,et al.  Motion regularization for model-based head tracking , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[14]  Dohyung Kim,et al.  An optical tracker for augmented reality and wearable computers , 1997, Proceedings of IEEE 1997 Annual International Symposium on Virtual Reality.

[15]  Robert C. Bolles,et al.  Background modeling for segmentation of video-rate stereo sequences , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[16]  Larry S. Davis,et al.  Computing 3-D head orientation from a monocular image sequence , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[17]  Alex Pentland,et al.  Parametrized structure from motion for 3D adaptive feedback tracking of faces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  William T. Freeman,et al.  Example-based head tracking , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[19]  Pertti Roivainen,et al.  3-D Motion Estimation in Model-Based Facial Image Coding , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Kurt Konolige,et al.  Small Vision Systems: Hardware and Implementation , 1998 .

[21]  Marco La Cascia,et al.  Head tracking via robust registration in texture map images , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[22]  Laurent Moll,et al.  Real time correlation-based stereo: algorithm, implementations and applications , 1993 .

[23]  José Santos-Victor,et al.  Active face and feature tracking , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[24]  M. Ibrahim Sezan,et al.  A robust real-time face tracking algorithm , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[25]  Michael J. Black,et al.  Recognizing Facial Expressions in Image Sequences Using Local Parameterized Models of Image Motion , 1997, International Journal of Computer Vision.

[26]  Demetri Terzopoulos,et al.  Analysis and Synthesis of Facial Image Sequences Using Physical and Anatomical Models , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Raphaël Féraud,et al.  LISTEN: a system for locating and tracking individual speakers , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[28]  Tsuhan Chen,et al.  Tracking of multiple faces for human-computer interfaces and virtual environments , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[29]  James L. Crowley,et al.  Multi-modal tracking of faces for video communications , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[30]  Trevor Darrell,et al.  Integrated Person Tracking Using Stereo, Color, and Pattern Detection , 2000, International Journal of Computer Vision.

[31]  William G. Gardner.  Head tracked 3-D audio using loudspeakers , 1997, Proceedings of 1997 Workshop on Applications of Signal Processing to Audio and Acoustics.

[32]  Hiroshi Mizoguchi,et al.  Virtual wireless microphone-a novel application of real-time visual tracking and sound signal processing , 1998, Proceedings of the 37th SICE Annual Conference. International Session Papers.

[33]  Dimitris N. Metaxas,et al.  The integration of optical flow and deformable models with applications to human face shape and motion estimation , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[34]  Oliver Bimber,et al.  Real mirrors reflecting virtual worlds , 2000, Proceedings IEEE Virtual Reality 2000 (Cat. No.00CB37048).

[35]  Carlo Tomasi,et al.  Depth Discontinuities by Pixel-to-Pixel Stereo , 1999, International Journal of Computer Vision.

[36]  Paul W. Fieguth,et al.  Color-based tracking of heads and other mobile objects at video frame rates , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[37]  Trevor Darrell,et al.  Active face tracking and pose estimation in an interactive room , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[38]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[39]  Alex Pentland,et al.  LAFTER: Lips and Face Real Time Tracker with Facial Expression Recognition , 1997, CVPR 1997.

[40]  Christopher R. Wren,et al.  Real-Time 3-D Tracking of the Human Body , 1996 .

[41]  Alex Pentland,et al.  Visually Controlled Graphics , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  Takeo Kanade,et al.  Visual Tracking of High DOF Articulated Structures: an Application to Human Hand Tracking , 1994, ECCV.

[43]  Michael T. Heath,et al.  Scientific Computing: An Introductory Survey , 1996 .

[44]  Donald C. Wunsch,et al.  Predictive head tracking for virtual reality , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[45]  Martin Herman,et al.  Head tracking using stereo , 2000, Proceedings Fifth IEEE Workshop on Applications of Computer Vision.

[46]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[47]  Takashi Matsuyama,et al.  Human head tracking using adaptive appearance models with a fixed-viewpoint pan-tilt-zoom camera , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[48]  Susumu Tachi,et al.  A method of constructing a telexistence visual system using fixed screens , 2000, Proceedings IEEE Virtual Reality 2000 (Cat. No.00CB37048).

[49]  Timothy F. Cootes,et al.  Learning to identify and track faces in image sequences , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[50]  Steven K. Feiner,et al.  A touring machine: Prototyping 3D mobile augmented reality systems for exploring the urban environment , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[51]  G. Sommer,et al.  Affine real-time face tracking using a wavelet network , 1999, Proceedings International Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems. In Conjunction with ICCV'99 (Cat. No.PR00378).

[52]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[53]  Stanley T. Birchfield,et al.  Elliptical head tracking using intensity gradients and color histograms , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[54]  Matthew Turk,et al.  Visual interaction with lifelike characters , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[55]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Other Conferences.

[56]  Ioannis Pitas,et al.  Segmentation and tracking of faces in color images , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.