The Asymmetry of Image Registration and Its Application to Face Tracking

Most image registration problems are formulated in an asymmetric fashion. Given a pair of images, one is implicitly or explicitly regarded as a template and warped onto the other to match as well as possible. In this paper, we focus on this seemingly arbitrary choice of the roles and reveal how it may lead to biased warp estimates in the presence of relative scaling. We present a principled way of selecting the template and explain why only the correct asymmetric form, with the potential inclusion of a blurring step, can yield an unbiased estimator. We validate our analysis in the domain of model-based face tracking. We show how the usual active appearance model (AAM) formulation overlooks the asymmetry issue, causing the fitting accuracy to degrade quickly when the observed objects are smaller than their model. We formulate a novel, "resolution-aware fitting" (RAF) algorithm that respects the asymmetry and incorporates an explicit model of the blur caused by the camera's sensing elements into the fitting formulation. We compare the RAF algorithm against a state-of-the-art tracker across a variety of resolutions and AAM complexity levels. Experimental results show that RAF significantly improves the estimation accuracy of both shape and appearance parameters when fitting to low-resolution data. Recognizing and accounting for the asymmetry of image registration leads to tangible accuracy improvements in analyzing low-resolution imagery

[1]  Peter Meer,et al.  Registration via direct methods: a statistical approach , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[2]  David Rey,et al.  Symmetrization of the Non-rigid Registration Problem Using Inversion-Invariant Energies: Application to Multiple Sclerosis , 2000, MICCAI.

[3]  Simon Baker,et al.  Active Appearance Models Revisited , 2004, International Journal of Computer Vision.

[4]  Yiannis Aloimonos,et al.  The Statistics of Optical Flow , 2001, Comput. Vis. Image Underst..

[5]  Lisa M. Brown,et al.  A survey of image registration techniques , 1992, CSUR.

[6]  Jan Flusser,et al.  Image registration methods: a survey , 2003, Image Vis. Comput..

[7]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[8]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[9]  J. Brandt.  Analysis of bias in gradient-based optical-flow estimation , 1994, Proceedings of 1994 28th Asilomar Conference on Signals, Systems and Computers.

[10]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.

[11]  P. Felber.  CHARGE-COUPLED DEVICES , 2002 .

[12]  P. Anandan,et al.  A computational framework and an algorithm for the measurement of visual motion , 1987, International Journal of Computer Vision.

[13]  G. Casella,et al.  Statistical Inference , 2003, Encyclopedia of Social Network Analysis and Mining.

[14]  Ralph Gross,et al.  Generic vs. person specific active appearance models , 2005, Image Vis. Comput..

[15]  Karl J. Friston,et al.  High-Dimensional Image Registration Using Symmetric Priors , 1999, NeuroImage.

[16]  Joseph K. Kearney,et al.  Optical Flow Estimation: An Error Analysis of Gradient-Based Methods with Local Optimization , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Jan Modersitzki,et al.  Numerical Methods for Image Registration , 2004 .

[18]  Takeo Kanade,et al.  Resolution-Aware Fitting of Active Appearance Models to Low Resolution Images , 2006, ECCV.

[19]  BrownLisa Gottesfeld.  A survey of image registration techniques , 1992 .

[20]  Hyeonjoon Moon,et al.  The FERET evaluation methodology for face-recognition algorithms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  I. Du,et al.  Direct Methods , 1998 .

[22]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[23]  Gary E. Christensen,et al.  Consistent Linear-Elastic Transformations for Image Matching , 1999, IPMI.

[24]  Cordelia Schmid,et al.  Matching images with different resolutions , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[25]  Simon Baker,et al.  Lucas-Kanade 20 Years On: A Unifying Framework , 2004, International Journal of Computer Vision.

[26]  金谷 健一.  Statistical optimization for geometric computation : theory and practice , 2005 .

[27]  Hemant D. Tagare,et al.  Symmetric, transitive, geometric deformation and intensity variation invariant nonrigid image registration , 2004, 2004 2nd IEEE International Symposium on Biomedical Imaging: Nano to Macro (IEEE Cat No. 04EX821).

[28]  Andrew Zisserman,et al.  Multiple view geometry in computer visiond , 2001 .

[29]  Bryan S. Morse,et al.  Multiscale image registration using scale trace correlation , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[30]  Timothy F. Cootes,et al.  Interpreting face images using active appearance models , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[31]  Russell C. Hardie,et al.  Joint MAP registration and high-resolution image estimation using a sequence of undersampled images , 1997, IEEE Trans. Image Process..

[32]  Hans-Hellmut Nagel,et al.  Bias-corrected optical flow estimation for road vehicle tracking , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[33]  T. Kanade,et al.  On the Source of Asymmetry in Image Registration Problems CMU-RI-TR-05-17 , 2005 .

[34]  Stanislav Kovacic,et al.  Symmetric image registration , 2003, SPIE Medical Imaging.