Tracking in 3D: Image Variability Decomposition for Recovering Object Pose and Illumination

As an object moves through space, it changes its orientation relative to the viewing camera and relative to light sources which illuminate it. As a consequence, the images of the object produced by the viewing camera may change dramatically. Thus, to successfully track a moving object, image changes due to varying pose and illumination must be accounted for. In this paper, we develop a method for object tracking that can not only accommodate large changes in object pose and illumination, but can recover these parameters as well. To do this, we separately model the image variation of the object produced by changes in pose and illumination. To track the object through each image in the sequences, we then locally search the models to find the best match, recovering the object’s orientation and illumination in the process. Throughout, we present experimental results, achieved in real-time, demonstrating the effectiveness of our methods.

[1]  Alex Pentland,et al.  View-based and modular eigenspaces for face recognition , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Michael J. Black,et al.  EigenTracking: Robust Matching and Tracking of Articulated Objects Using a View-Based Representation , 1996, International Journal of Computer Vision.

[3]  L Sirovich,et al.  Low-dimensional procedure for the characterization of human faces. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[4]  A. Shashua.  Geometry and Photometry in 3D Visual Recognition , 1992 .

[5]  Michael J. Black,et al.  Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion , 1995, Proceedings of IEEE International Conference on Computer Vision.

[6]  Larry S. Davis,et al.  Tracking of humans in action: a 3-D model-based approach , 1996 .

[7]  N BelhumeurPeter.  A Bayesian approach to binocular stereopsis , 1996 .

[8]  James M. Rehg,et al.  Visual tracking with deformation models , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[9]  D. Jacobs.  Space Efficient 3D Model Indexing , 1992 .

[10]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[11]  Nikolaos Papanikolopoulos,et al.  Adaptive robotic visual tracking: theory and experiments , 1993, IEEE Trans. Autom. Control..

[12]  Shimon Ullman,et al.  Face Recognition: The Problem of Compensating for Changes in Illumination Direction , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[14]  Gregory D. Hager,et al.  X Vision: A Portable Substrate for Real-Time Vision Applications , 1998, Comput. Vis. Image Underst..

[15]  Hiroshi Murase,et al.  Visual learning and recognition of 3-d objects from appearance , 2005, International Journal of Computer Vision.

[16]  Michael J. Black,et al.  Recognizing Facial Expressions in Image Sequences Using Local Parameterized Models of Image Motion , 1997, International Journal of Computer Vision.

[17]  S. Ullman,et al.  Geometry and photometry in three-dimensional visual recognition , 1993 .

[18]  Rachid Deriche,et al.  Tracking line segments , 1990, Image Vis. Comput..

[19]  Donald B. Gennery,et al.  Visual tracking of known three-dimensional objects , 1992, International Journal of Computer Vision.

[20]  M. Goresky,et al.  Stratified Morse theory , 1988 .

[21]  Ronen Basri,et al.  Recognition by Linear Combinations of Models , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  J J Koenderink,et al.  Affine structure from motion. , 1991, Journal of the Optical Society of America. A, Optics and image science.

[23]  Andrew Blake,et al.  Learning Dynamics of Complex Motions from Image Sequences , 1996, ECCV.

[24]  Michael Isard,et al.  Active Contours , 2000, Springer London.

[25]  Gregory D. Hager,et al.  Efficient Region Tracking With Parametric Models of Geometry and Illumination , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Amnon Shashua,et al.  Direct estimation of motion and extended scene structure from a moving stereo rig , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[27]  David G. Lowe,et al.  Robust model-based motion tracking through the integration of search and estimation , 1992, International Journal of Computer Vision.

[28]  Christian Smith,et al.  Robot manipulators , 2009, IEEE Robotics & Automation Magazine.

[29]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[31]  Michael Isard,et al.  Contour Tracking by Stochastic Propagation of Conditional Density , 1996, ECCV.

[32]  Tomaso A. Poggio,et al.  Example-Based Learning for View-Based Human Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  David J. Kriegman,et al.  What Is the Set of Images of an Object Under All Possible Illumination Conditions? , 1998, International Journal of Computer Vision.