A Review of Active Appearance Models

Active appearance model (AAM) is a powerful generative method for modeling deformable objects. The model decouples the shape and the texture variations of objects, which is followed by an efficient gradient-based model fitting method. Due to the flexible and simple framework, AAM has been widely applied in the fields of computer vision. However, difficulties are met when it is applied to various practical issues, which lead to a lot of prominent improvements to the model. Nevertheless, these difficulties and improvements have not been studied systematically. This motivates us to review the recent advances of AAM. This paper focuses on the improvements in the literature in turns of the problems suffered by AAM in practical applications. Therefore, these algorithms are summarized from three aspects, i.e., efficiency, discrimination, and robustness. Additionally, some applications and implementations of AAM are also enumerated. The main purpose of this paper is to serve as a guide for further research.

[1]  Luiz Velho,et al.  Expression Transfer between Photographs through Multilinear AAM's , 2006, 2006 19th Brazilian Symposium on Computer Graphics and Image Processing.

[2]  Mikkel B. Stegmann,et al.  Active appearance models: Theory and cases , 2000 .

[3]  Milan Sonka,et al.  Time-Continuous Segmentation of Cardiac Image Sequences Using Active Appearance Motion Models , 2001, IPMI.

[4]  Xuelong Li,et al.  Patch Alignment for Dimensionality Reduction , 2009, IEEE Transactions on Knowledge and Data Engineering.

[5]  Xuelong Li,et al.  Bayesian Tensor Approach for 3-D Face Modeling , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[6]  Timothy F. Cootes,et al.  Automatic Interpretation and Coding of Face Images Using Flexible Models , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Rasmus Larsen,et al.  Multi-band modelling of appearance , 2003, Image Vis. Comput..

[8]  Timothy F. Cootes,et al.  An Algorithm for Tuning an Active Appearance Model to New Data , 2006, BMVC.

[9]  Takeo Kanade,et al.  Resolution-Aware Fitting of Active Appearance Models to Low Resolution Images , 2006, ECCV.

[10]  Jøger Hansegård,et al.  Constrained Active Appearance Models for Segmentation of Triplane Echocardiograms , 2007, IEEE Transactions on Medical Imaging.

[11]  Timothy F. Cootes,et al.  Interpreting face images using active appearance models , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[12]  John K. Tsotsos,et al.  A Novel Algorithm for Fitting 3-D Active Appearance Models: Applications to Cardiac MRI Segmentation , 2005, SCIA.

[13]  H. Torp,et al.  Detection of the myocardial boundary in the left ventricle from simultaneously acquired triplane ultrasound images using multi view active appearance motion models , 2005, IEEE Ultrasonics Symposium, 2005..

[14]  Shaogang Gong,et al.  On Utilising Template and Feature-Based Correspondence in Multi-view Appearance Models , 2000, ECCV.

[15]  Timothy F. Cootes,et al.  Feature Detection and Tracking with Constrained Local Models , 2006, BMVC.

[16]  David Salesin,et al.  Image-Based Hair Capture by Inverse Lighting , 2002, Graphics Interface.

[17]  Bjarne K. Ersbøll,et al.  FAME-a flexible appearance modeling environment , 2003, IEEE Transactions on Medical Imaging.

[18]  Ralph Gross,et al.  Constructing and Fitting Active Appearance Models With Occlusion , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[19]  Jing Xiao,et al.  Fitting a Single Active Appearance Model Simultaneously to Multiple Images , 2004, BMVC.

[20]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[21]  Daijin Kim,et al.  Staam: Fitting a 2D+3D AAM to Stereo Images , 2006, 2006 International Conference on Image Processing.

[22]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[23]  Xuelong Li,et al.  A new approach for face recognition by sketches in photos , 2009, Signal Process..

[24]  Aziz Umit Batur,et al.  A Novel Convergence Scheme for Active Appearance Models , 2003, CVPR.

[25]  Léon J. M. Rothkrantz,et al.  Facial expression recognition in still pictures and videos using active appearance models: a comparison approach , 2007, CompSysTech '07.

[26]  Timothy F. Cootes,et al.  Coupled-View Active Appearance Models , 2000, BMVC.

[27]  Barry-John Theobald,et al.  Real-time expression cloning using appearance models , 2007, ICMI '07.

[28]  Daijin Kim,et al.  A Real-Time Face Tracking using the Stereo Active Appearance Model , 2006, 2006 International Conference on Image Processing.

[29]  Mikkel B. Stegmann,et al.  Object tracking using active appearance models , 2001 .

[30]  Fernando De la Torre,et al.  Bilinear Active Appearance Models , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[31]  Jing Xiao,et al.  Robust AAM Fitting by Fusion of Images and Disparity Data , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[32]  Takeo Kanade,et al.  The Asymmetry of Image Registration and Its Application to Face Tracking , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Muhittin Gökmen,et al.  Illumination Invariant Three-Stage Approach for Face Alignment , 2006, 2006 International Conference on Image Processing.

[34]  Timothy F. Cootes,et al.  A Unified Framework for Atlas Matching Using Active Appearance Models , 1999, IPMI.

[35]  Takeo Kanade,et al.  Real-time combined 2D+3D active appearance models , 2004, CVPR 2004.

[36]  D. Donoho.  Wedgelets: nearly minimax estimation of edges , 1999 .

[37]  Muhittin Gökmen,et al.  Robust Face Alignment for Illumination and Pose Invariant Face Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[38]  Timothy F. Cootes,et al.  Advances in active appearance models , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[39]  Bjarne Kjær Ersbøll,et al.  Extending and Applying Active Appearance Models for Automated, High Precision Segmentation* , 2001 .

[40]  Timothy F. Cootes,et al.  Using parts and geometry models to initialise Active Appearance Models for automated segmentation of 3D medical images , 2010, 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[41]  Trevor Darrell,et al.  On modelling nonlinear shape-and-texture appearance manifolds , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[42]  Jian Wan,et al.  Automatic red-eyes detection based on AAM , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[43]  Horst Bischof,et al.  Fast Active Appearance Model Search Using Canonical Correlation Analysis , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Ralph Gross,et al.  Generic vs. person specific active appearance models , 2005, Image Vis. Comput..

[45]  Simon Baker,et al.  Active Appearance Models Revisited , 2004, International Journal of Computer Vision.

[46]  Yunus Saatci,et al.  Cascaded classification of gender and facial expression using active appearance models , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[47]  Roland Göcke,et al.  Monocular and Stereo Methods for AAM Learning from Video , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  Milan Sonka,et al.  Segmentation of cardiac MR images: an active appearance model approach , 2000, Medical Imaging: Image Processing.

[49]  Takeo Kanade,et al.  A Unified Gradient-Based Approach for Combining ASM into AAM , 2006, International Journal of Computer Vision.

[50]  Timothy F. Cootes,et al.  Constrained active appearance models , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[51]  Milan Sonka,et al.  Multistage hybrid active appearance model matching: segmentation of left and right ventricles in cardiac MR images , 2001, IEEE Transactions on Medical Imaging.

[52]  Milan Sonka,et al.  Robust active appearance models and their application to medical image analysis , 2005, IEEE Transactions on Medical Imaging.

[53]  Simon Baker,et al.  Lucas-Kanade 20 Years On: A Unifying Framework , 2004, International Journal of Computer Vision.

[54]  Franck Davoine,et al.  Facial expression recognition and synthesis based on an appearance model , 2004, Signal Process. Image Commun..

[55]  Xiaoming Liu,et al.  Generic Face Alignment using Boosted Appearance Model , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[56]  Chun Chen,et al.  Subspace analysis and optimization for AAM based face alignment , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[57]  Daijin Kim,et al.  Estimating 3D Facial Shape and Motion from Stereo Image Using Active Appearance Models with Stereo Constraints , 2006, ICIAR.

[58]  Milan Sonka,et al.  Multi-view Active Appearance Models: Application to X-Ray LV Angiography and Cardiac MRI , 2003, IPMI.

[59]  Horst-Michael Groß,et al.  Classification of Face Images for Gender, Age, Facial Expression, and Identity , 2005, ICANN.

[60]  Roland Göcke,et al.  A Nonlinear Discriminative Approach to AAM Fitting , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[61]  Timothy F. Cootes,et al.  A Comparative Evaluation of Active Appearance Model Algorithms , 1998, BMVC.

[62]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[63]  Takahiro Ishikawa,et al.  The template update problem , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[64]  Milan Sonka,et al.  Automatic segmentation of echocardiographic sequences by active appearance motion models , 2002, IEEE Transactions on Medical Imaging.

[65]  Ralph R. Martin,et al.  Merging and Splitting Eigenspace Models , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[66]  Daijin Kim,et al.  Pose-Robust Facial Expression Recognition Using View-Based 2D $+$ 3D AAM , 2008, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[67]  Daijin Kim,et al.  Large Motion Object Tracking using Active Contour Combined Active Appearance Model , 2006, Fourth IEEE International Conference on Computer Vision Systems (ICVS'06).

[68]  Stan Z. Li,et al.  Direct appearance models , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[69]  Gerhard Rigoll,et al.  A Hierarchical ASM/AAM Approach in a Stochastic Framework for Fully Automatic Tracking and Recognition , 2006, 2006 International Conference on Image Processing.

[70]  Rasmus Larsen,et al.  An Active Illumination and Appearance (AIA) Model for Face Alignment , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[71]  Bjarne K. Ersbøll,et al.  Wedgelet Enhanced Appearance Models , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[72]  Xuelong Li,et al.  Gait Components and Their Application to Gender Recognition , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[73]  Timothy F. Cootes,et al.  Improving Appearance Model Matching Using Local Image Structure , 2003, IPMI.

[74]  Timothy F. Cootes,et al.  Groupwise Construction of Appearance Models using Piece-wise Affine Deformations , 2005, BMVC.

[75]  Timothy F. Cootes,et al.  Comparing Active Shape Models with Active Appearance Models , 1999, BMVC.

[76]  Timothy F. Cootes,et al.  On representing edge structure for model matching , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[77]  Timothy F. Cootes,et al.  A comparison of shape constrained facial feature detectors , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[78]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[79]  Milan Sonka,et al.  Shape- and appearance-based segmentation of volumetric medical images , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[80]  Milan Sonka,et al.  3-D active appearance models: segmentation of cardiac MR and ultrasound images , 2002, IEEE Transactions on Medical Imaging.

[81]  Aziz Umit Batur,et al.  Adaptive active appearance models , 2005, IEEE Transactions on Image Processing.

[82]  Alejandro F. Frangi,et al.  A Framework for Weighted Fusion of Multiple Statistical Models of Shape and Appearance , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[83]  Daijin Kim,et al.  A Real-Time Facial Expression Recognition using the STAAM , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[84]  P. Kittipanya-ngam,et al.  The effect of texture representations on AAM performance , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[85]  Christopher J. Taylor,et al.  Wavelet Compression of Active Appearance Models , 1999, MICCAI.

[86]  Timothy F. Cootes,et al.  Facial feature detection and tracking with automatic template selection , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[87]  Daijin Kim,et al.  Expression-invariant face recognition by facial expression transformations , 2008, Pattern Recognit. Lett..

[88]  Muhittin Gökmen,et al.  Illumination Invariant Face Alignment Using Multi-band Active Appearance Model , 2005, PReMI.

[89]  Harry Shum,et al.  Face alignment using texture-constrained active shape models , 2003, Image Vis. Comput..

[90]  Hao Wu,et al.  Face alignment via boosted ranking model , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[91]  Milan Sonka,et al.  Multiview Active Appearance Models for Simultaneous Segmentation of Cardiac 2- and 4-Chamber Long-Axis Magnetic Resonance Images , 2005, Investigative radiology.

[92]  Jeff G. Schneider,et al.  Automatic construction of active appearance models as an image coding problem , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[93]  Rasmus Larsen,et al.  A Noise Robust Statistical Texture Model , 2002, MICCAI.

[94]  Simon Baker,et al.  Equivalence and efficiency of image alignment algorithms , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[95]  Milan Sonka,et al.  Diaphragm dome surface segmentation in CT data sets: a 3D active appearance model approach , 2002, SPIE Medical Imaging.

[96]  C. Goodall.  Procrustes methods in the statistical analysis of shape , 1991 .

[97]  K. Walker,et al.  View-based active appearance models , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[98]  Timothy F. Cootes,et al.  Comparing Variations on the Active Appearance Model Algorithm , 2002, BMVC.

[99]  Milan Sonka,et al.  Left ventricle contour detection in x-ray angiograms using multi-view active appearance models , 2003, SPIE Medical Imaging.

[100]  Jing Xiao,et al.  Multi-view AAM fitting and camera calibration , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[101]  Stefano Soatto,et al.  Layered active appearance models , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[102]  Iasonas Kokkinos,et al.  Unsupervised Learning of Object Deformation Models , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[103]  Timothy F. Cootes,et al.  Automatically building appearance models from image sequences using salient features , 2002, Image Vis. Comput..

[104]  Xuelong Li,et al.  Gabor-based texture representation in AAMs , 2008, 2008 IEEE International Conference on Systems, Man and Cybernetics.

[105]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[106]  Timothy F. Cootes,et al.  Linking Sequences of Active Appearance Sub-Models via Constraints: An Application in Automated Vertebral Morphometry , 2003, BMVC.

[107]  Franck Davoine,et al.  Facial action tracking using an AAM-based CONDENSATION approach , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[108]  Yang Wang,et al.  Enforcing convexity for improved alignment with constrained local models , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.