Image Deformation With Vector-Field Interpolation Based on MRLS-TPS

Image deformation has been successfully applied in many different kinds of fields. However, how to get an approach with high efficiency and perfect visual effect remains a challenging task. In this paper, we present a vector-field interpolation method for non-rigid image deformation, which is based on moving regularized least squares (MRLS) optimization with a thin-plate spline (TPS). The proposed approach takes user-controlled points as input data and estimates the spatial transformation for each pixel by the control points. In order to achieve a realistic deformation, we formulate the deformation as a novel closed-form transformation estimation problem by MRLS. Unlike moving least squares (MLS), we model the mapping function by a non-rigid TPS function with a regularization coefficient. Therefore, the deformation not only satisfies the global linear affine transformation but also adapts to local non-rigid deformation. In terms of the transformation, we derive a closed-form solution and achieve a fast implementation. Furthermore, the approach can show us a wonderful user experience and can give us a fast and convenient manipulating. Extensive experiments on 2D images and 3D surfaces demonstrated that the proposed method performs better than other state-of-the-art methods like MLS and the commercial software as Adobe PhotoShop CS 6, especially in the case of flexible object motion.

[1]  John P. Lewis,et al.  Pose Space Deformation: A Unified Approach to Shape Interpolation and Skeleton-Driven Deformation , 2000, SIGGRAPH.

[2]  Scott Schaefer,et al.  Image deformation using moving least squares , 2006, ACM Trans. Graph..

[3]  Yongjun Zhang,et al.  Large-Scale Remote Sensing Image Retrieval by Deep Hashing Neural Networks , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Hans-Peter Seidel,et al.  Meshless Shape and Motion Design for Multiple Deformable Objects , 2010, Comput. Graph. Forum.

[5]  Heinrich Müller,et al.  Image warping with scattered data interpolation , 1995, IEEE Computer Graphics and Applications.

[6]  Junjun Jiang,et al.  Guided Locality Preserving Feature Matching for Remote Sensing Image Registration , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[7]  Takeo Igarashi,et al.  As-rigid-as-possible shape manipulation , 2005, ACM Trans. Graph..

[8]  David Levin,et al.  The approximation power of moving least-squares , 1998, Math. Comput..

[9]  Sung Yong Shin,et al.  Image metamorphosis using snakes and free-form deformations , 1995, SIGGRAPH.

[10]  Junjun Jiang,et al.  Robust Feature Matching for Remote Sensing Image Registration via Locally Linear Transforming , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Junjun Jiang,et al.  A non-parametric depth modification model for registration between color and depth images , 2018, Multidimensional Systems and Signal Processing.

[12]  Tao Ju,et al.  A geometric database for gene expression data , 2003, Symposium on Geometry Processing.

[13]  Wen Gao,et al.  Image interpolation via regularized local linear regression , 2011, 28th Picture Coding Symposium.

[14]  Grace Wahba,et al.  Spline Models for Observational Data , 1990 .

[15]  Saeid Nahavandi,et al.  A morphing technique for facial image representation , 2000, Smc 2000 conference proceedings. 2000 ieee international conference on systems, man and cybernetics. 'cybernetics evolving to systems, humans, organizations, and their complex interactions' (cat. no.0.

[16]  Sung Yong Shin,et al.  Scattered Data Interpolation with Multilevel B-Splines , 1997, IEEE Trans. Vis. Comput. Graph..

[17]  Yansheng Li,et al.  Feature guided Gaussian mixture model with semi-supervised EM and local geometric constraint for retinal image registration , 2017, Inf. Sci..

[18]  Dan Xu,et al.  Multiresolution image morphing in wavelet domain , 2000, 2000 IEEE Conference on Information Visualization. An International Conference on Computer Visualization and Graphics.

[19]  Steven M. Seitz,et al.  View morphing , 1996, SIGGRAPH.

[20]  Zhuowen Tu,et al.  Regularized vector field learning with sparse approximation for mismatch removal , 2013, Pattern Recognit..

[21]  Marc Alexa,et al.  As-rigid-as-possible shape interpolation , 2000, SIGGRAPH.

[22]  Ji Zhao,et al.  Nonrigid Image Deformation Using Moving Regularized Least Squares , 2013, IEEE Signal Processing Letters.

[23]  Masahiro Fujita,et al.  Multiresolution interpolation meshes , 2001, Proceedings Ninth Pacific Conference on Computer Graphics and Applications. Pacific Graphics 2001.

[24]  Junjun Jiang,et al.  Locality Preserving Matching , 2018, International Journal of Computer Vision.

[25]  Junchi Yan,et al.  Adaptive Discrete Hypergraph Matching , 2018, IEEE Transactions on Cybernetics.

[26]  Ji Zhao,et al.  Nonrigid Feature Matching for Remote Sensing Images via Probabilistic Inference With Global and Local Regularizations , 2016, IEEE Geoscience and Remote Sensing Letters.

[27]  Thaddeus Beier,et al.  Feature-based image metamorphosis , 1998 .

[28]  Li Ma,et al.  Non-rigid point set registration via coherent spatial mapping , 2015, Signal Process..

[29]  Riqing Chen,et al.  Non-rigid point set registration via global and local constraints , 2018, Multimedia Tools and Applications.

[30]  Hai Liu,et al.  Fast Blind Instrument Function Estimation Method for Industrial Infrared Spectrometers , 2018, IEEE Transactions on Industrial Informatics.

[31]  Tao Lu,et al.  A Robust Method for Estimating Image Geometry With Local Structure Constraint , 2018, IEEE Access.

[32]  Sung Yong Shin,et al.  Image Morphing Using Deformation Techniques , 1996, Comput. Animat. Virtual Worlds.

[33]  Zhenghong Yu,et al.  Non-rigid image deformation algorithm based on MRLS-TPS , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[34]  Sung Yong Shin,et al.  Polymorph: Morphing Among Multiple Images , 1998, IEEE Computer Graphics and Applications.

[35]  Craig Gotsman,et al.  Intrinsic Morphing of Compatible Triangulations , 2003, Int. J. Shape Model..

[36]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[37]  Franco Dassi,et al.  A Novel Surface Remeshing Scheme via Radial Basis Functions and Higher-Dimensional Embedding , 2017, SIAM J. Sci. Comput..

[38]  Zhenghong Yu,et al.  Fast non-rigid image feature matching for agricultural UAV via probabilistic inference with regularization techniques , 2017, Comput. Electron. Agric..