Schwarps: Locally Projective Image Warps Based on 2D Schwarzian Derivatives

Image warps -or just warps- capture the geometric deformation existing between two images of a deforming surface. The current approach to enforce a warp’s smoothness is to penalize its second order partial derivatives (Bookstein in IEEE Trans Pattern Anal Mach Intell 11:567–585, 1989; Rueckert et al. in IEEE Trans Med Imaging 18:712–721, 1999). Because this favors locally affine warps, this fails to capture the local projective component of the image deformation. This may have a negative impact on applications such as image registration and deformable 3D reconstruction. We propose a novel penalty designed to smooth the warp while capturing the deformation’s local projective structure. Our penalty is based on equivalents to the Schwarzian derivatives, which are projective differential invariants exactly preserved by homographies. We propose a methodology to derive a set of partial differential equations with only homographies as solutions. We call this system the Schwarzian equations and we explicitly derive them for 2D functions using differential properties of homographies. We name as Schwarp a warp which is estimated by penalizing the residual of Schwarzian equations. Experimental evaluation shows that Schwarps outperform existing warps in modeling and extrapolation power, and lead to better results in three deformable reconstruction methods, namely, shape reconstruction in shape-from-template, camera calibration in Shape-from-Template and Non-Rigid Structure-from-Motion.