Feature guided non-rigid image/surface deformation via moving least squares with manifold regularization

In this paper, a novel closed-form transformation estimation method based on feature guided moving least squares together with manifold regularization is proposed for nonrigid image/surface deformation. The method takes the user-controlled point-offset-vectors and the feature points of the image/surface as input, and estimates the spatial transformation between the two control point sets for each pixel/voxel. To achieve a detail-preserving and realistic deformation, the transformation estimation is formulated as a vector-field interpolation problem using a feature guided moving least squares method, where a manifold regularization is imposed as a prior on the transformation to capture the underlying intrinsic geometry of the input image/surface. The non-rigid transformation is specified in a reproducing kernel Hilbert space. We derive a closed-form solution of the transformation and adopt a sparse approximation to achieve a fast implementation, which largely reduces the computation complexity without performance sacrifice. In addition, the proposed method can give a wonderful user experience, fast and convenient manipulating. Extensive experiments on both 2D and 3D data demonstrate that the proposed method can produce more natural deformations compared with other state-of-the-art methods.

[1]  Zhuowen Tu,et al.  Robust Estimation of Nonrigid Transformation for Point Set Registration , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Radu Horaud,et al.  Surface feature detection and description with applications to mesh matching , 2009, CVPR.

[3]  Alan L. Yuille,et al.  Semi-Supervised Sparse Representation Based Classification for Face Recognition With Insufficient Labeled Samples , 2016, IEEE Transactions on Image Processing.

[4]  Markus H. Gross,et al.  Adaptive Space Deformations Based on Rigid Cells , 2007, Comput. Graph. Forum.

[5]  Zhuowen Tu,et al.  Regularized vector field learning with sparse approximation for mismatch removal , 2013, Pattern Recognit..

[6]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[7]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[8]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[9]  Yuan Gao,et al.  Exploiting Symmetry and/or Manhattan Properties for 3D Object Structure Estimation from Single and Multiple Images , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  N. Aronszajn Theory of Reproducing Kernels. , 1950 .

[11]  Zheng Wang,et al.  SRLSP: A Face Image Super-Resolution Algorithm Using Smooth Regression With Local Structure Prior , 2017, IEEE Transactions on Multimedia.

[12]  Sung Yong Shin,et al.  Scattered Data Interpolation with Multilevel B-Splines , 1997, IEEE Trans. Vis. Comput. Graph..

[13]  Ruimin Hu,et al.  Facial Image Hallucination Through Coupled-Layer Neighbor Embedding , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Junjun Jiang,et al.  Non-Rigid Point Set Registration with Robust Transformation Estimation under Manifold Regularization , 2017, AAAI.

[15]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[16]  Scott Schaefer,et al.  Image deformation using moving least squares , 2006, ACM Trans. Graph..

[17]  R. Horaud,et al.  Surface feature detection and description with applications to mesh matching , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  David Levin,et al.  The approximation power of moving least-squares , 1998, Math. Comput..

[19]  Tao Ju,et al.  A geometric database for gene expression data , 2003, Symposium on Geometry Processing.

[20]  Zhuowen Tu,et al.  Robust Point Matching via Vector Field Consensus , 2014, IEEE Transactions on Image Processing.

[21]  Fred L. Bookstein,et al.  Principal Warps: Thin-Plate Splines and the Decomposition of Deformations , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Kenneth I. Joy,et al.  Free-form deformations with lattices of arbitrary topology , 1996, SIGGRAPH.

[23]  Hans-Peter Seidel,et al.  Meshless Shape and Motion Design for Multiple Deformable Objects , 2010, Comput. Graph. Forum.

[24]  Ji Zhao,et al.  Nonrigid Image Deformation Using Moving Regularized Least Squares , 2013, IEEE Signal Processing Letters.

[25]  Fred L. Bookstein,et al.  Thin-Plate Splines and the Atlas Problem for Biomedical Images , 1991, IPMI.

[26]  Yuan Gao,et al.  Symmetric Non-rigid Structure from Motion for Category-Specific Object Structure Estimation , 2016, ECCV.

[27]  Takeo Igarashi,et al.  As-rigid-as-possible shape manipulation , 2005, ACM Trans. Graph..

[28]  Thaddeus Beier,et al.  Feature-based image metamorphosis , 1998 .