Fast Non-Rigid Surface Detection, Registration and Realistic Augmentation

We present a real-time method for detecting deformable surfaces, with no need whatsoever for a priori pose knowledge.Our method starts from a set of wide baseline point matches between an undeformed image of the object and the image in which it is to be detected. The matches are used not only to detect but also to compute a precise mapping from one to the other. The algorithm is robust to large deformations, lighting changes, motion blur, and occlusions. It runs at 10 frames per second on a 2.8 GHz PC.We demonstrate its applicability by using it to realistically modify the texture of a deforming surface and to handle complex illumination effects.Combining deformable meshes with a well designed robust estimator is key to dealing with the large number of parameters involved in modeling deformable surfaces and rejecting erroneous matches for error rates of more than 90%, which is considerably more than what is required in practice.

[1]  Vincent Lepetit,et al.  Monocular Model-Based 3D Tracking of Rigid Objects: A Survey , 2005, Found. Trends Comput. Graph. Vis..

[2]  Larry S. Davis,et al.  Structure of Applicable Surfaces from Single Views , 2004, ECCV.

[3]  Adrien Bartoli,et al.  Direct Estimation of Non-Rigid Registration , 2004, BMVC.

[4]  Serge J. Belongie,et al.  A Feature-Based Approach for Determining Dense Long Range Correspondences , 2004, ECCV.

[5]  Andrew Zisserman,et al.  Multiple view geometry in computer visiond , 2001 .

[6]  Vincent Lepetit,et al.  Randomized trees for real-time keypoint recognition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[7]  David A. Forsyth,et al.  Retexturing Single Views Using Texture and Shading , 2006, ECCV.

[8]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[9]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[10]  Radu Horaud,et al.  Hand Motion from 3D Point Trajectories and a Smooth Surface Model , 2004, ECCV.

[11]  Simon Baker.  Real-time non-rigid driver head tracking for driver mental state estimation , 2004 .

[12]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[13]  Dimitris N. Metaxas,et al.  Deformable model-based shape and motion analysis from images using motion residual error , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[14]  Yanxi Liu,et al.  Tracking Dynamic Near-Regular Texture Under Occlusion and Rapid Movements , 2006, ECCV.

[15]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[16]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[17]  Xavier Pennec,et al.  Multi-scale EM-ICP: A Fast and Robust Approach for Surface Registration , 2002, ECCV.

[18]  Tom Drummond,et al.  Fusing points and lines for high performance tracking , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[19]  Luc Van Gool,et al.  Edinburgh Research Explorer Simultaneous Object Recognition and Segmentation by Image Exploration , 2022 .

[20]  Haibin Ling,et al.  Deformation invariant image matching , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[21]  Pascal Fua,et al.  Model-Based Optimization: An Approach to Fast, Accurate, and Consistent Site Modeling from Imagery , 1997 .

[22]  Anand Rangarajan,et al.  A new point matching algorithm for non-rigid registration , 2003, Comput. Vis. Image Underst..

[23]  Stan Sclaroff,et al.  Active blobs: region-based, deformable appearance models , 2003, Computer Vision and Image Understanding.