Time-Varying Image Processing for 3D Model-Based Video Coding

Publisher Summary This chapter presents a robust and novel approach to real-time feature tracking using a 3D model-based framework. Image understanding and motion analysis of video sequences are two aspects of computer vision that connect and relate to a number of applications ranging from video annotation to video coding to human computer interfaces. In most scenarios, objects are not rigid because their motion, with respect to the scene, is combined with deformations and changes in light conditions. A framework for automatic detection and tracking of moving objects in such complex conditions is very important in time-varying imagery, and would provide a tool for further analysis and understanding of the higher level aspects of the observed scene. A small set of facial features are tracked successfully over a large range of head motion. Nonrigid points are also tracked successfully after compensating for the computed global motion. The combination of the 3D model, head pose estimation, and texture mapping avoids the error accumulation problem and allows better localization of the features. Results confirm that the system is robust, accurate, and can be implemented in real-time. This framework finds applications ranging from model-based video coding and facial expression analysis to face recognition using video sequences.