A Fast 2D Shape Recovery Approach by Fusing Features and Appearance

In this paper, we present a fusion approach to solve the nonrigid shape recovery problem, which takes advantage of both the appearance information and the local features. We have two major contributions. First, we propose a novel progressive finite Newton optimization scheme for the feature-based nonrigid surface detection problem, which is reduced to only solving a set of linear equations. The key is to formulate the nonrigid surface detection as an unconstrained quadratic optimization problem that has a closed-form solution for a given set of observations. Second, we propose a deformable Lucas-Kanade algorithm that triangulates the template image into small patches and constrains the deformation through the second-order derivatives of the mesh vertices. We formulate it into a sparse regularized least squares problem, which is able to reduce the computational cost and the memory requirement. The inverse compositional algorithm is applied to efficiently solve the optimization problem. We have conducted extensive experiments for performance evaluation on various environments, whose promising results show that the proposed algorithm is both efficient and effective.

[1]  Dimitris N. Metaxas,et al.  Optical Flow Constraints on Deformable Models with Applications to Face Tracking , 2000, International Journal of Computer Vision.

[2]  Simon Baker,et al.  Active Appearance Models Revisited , 2004, International Journal of Computer Vision.

[3]  John C. Hart,et al.  RotoTexture: Automated Tools for Texturing Raw Video , 2006, IEEE Transactions on Visualization and Computer Graphics.

[4]  Michael R. Lyu,et al.  Real-Time Non-rigid Shape Recovery Via Active Appearance Models for Augmented Reality , 2006, ECCV.

[5]  David A. Forsyth,et al.  Retexturing Single Views Using Texture and Shading , 2006, ECCV.

[6]  Xavier Pennec,et al.  Multi-scale EM-ICP: A Fast and Robust Approach for Surface Registration , 2002, ECCV.

[7]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[8]  Lorenzo Torresani,et al.  Tracking and modeling non-rigid objects with rank constraints , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[9]  Hany Farid,et al.  Medical image registration with partial data , 2006, Medical Image Anal..

[10]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[11]  David A. Forsyth,et al.  Combining Cues: Shape from Shading and Texture , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[12]  Yanxi Liu,et al.  Tracking Dynamic Near-Regular Texture Under Occlusion and Rapid Movements , 2006, ECCV.

[13]  Adrien Bartoli,et al.  Direct Estimation of Non-Rigid Registration , 2004, BMVC.

[14]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[15]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[16]  Vincent Lepetit,et al.  Keypoint recognition using randomized trees , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Michael J. Black,et al.  Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion , 1995, Proceedings of IEEE International Conference on Computer Vision.

[18]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[19]  Dmitry B. Goldgof,et al.  Nonrigid motion analysis based on dynamic refinement of finite element models , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[20]  Pascal Fua,et al.  Implicit Meshes for Effective Silhouette Handling , 2007, International Journal of Computer Vision.

[21]  Jiri Matas,et al.  Matching with PROSAC - progressive sample consensus , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[22]  Simon Baker,et al.  Lucas-Kanade 20 Years On: A Unifying Framework , 2004, International Journal of Computer Vision.

[23]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  S. Sathiya Keerthi,et al.  A Modified Finite Newton Method for Fast Solution of Large Scale Linear SVMs , 2005, J. Mach. Learn. Res..

[25]  Pascal Fua,et al.  Object-centered surface reconstruction: Combining multi-image stereo and shading , 1995, International Journal of Computer Vision.

[26]  Matthew Brand,et al.  Flexible flow for 3D nonrigid tracking and shape recovery , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[27]  Lisa M. Brown,et al.  A survey of image registration techniques , 1992, CSUR.

[28]  Sethu Vijayakumar,et al.  Part-based Probabilistic Point Matching using Equivalence Constraints , 2006, NIPS.

[29]  Vincent Lepetit,et al.  Real-time nonrigid surface detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[30]  Anand Rangarajan,et al.  A new point matching algorithm for non-rigid registration , 2003, Comput. Vis. Image Underst..

[31]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[32]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[33]  Vincent Lepetit,et al.  Fast Non-Rigid Surface Detection, Registration and Realistic Augmentation , 2008, International Journal of Computer Vision.

[34]  Cordelia Schmid,et al.  A performance evaluation of local descriptors , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Jitendra Malik,et al.  Shape matching and object recognition using shape contexts , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[36]  John Hart,et al.  Textureshop: texture synthesis as a photograph editing tool , 2004, SIGGRAPH 2004.

[37]  Florent Brunet,et al.  Feature-Driven Direct Non-Rigid Image Registration , 2010, International Journal of Computer Vision.

[38]  Haibin Ling,et al.  Deformation invariant image matching , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[39]  Michael R. Lyu,et al.  Progressive Finite Newton Approach To Real-time Nonrigid Surface Detection , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Pascal Fua,et al.  Surface Deformation Models for Nonrigid 3D Shape Recovery , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Stan Sclaroff,et al.  Active blobs: region-based, deformable appearance models , 2003, Computer Vision and Image Understanding.

[42]  Olvi L. Mangasarian,et al.  A finite newton method for classification , 2002, Optim. Methods Softw..

[43]  Selim Benhimane,et al.  Real-time image-based tracking of planes using efficient second-order minimization , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[44]  Adrien Bartoli,et al.  Groupwise Geometric and Photometric Direct Image Registration , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.