Robust Isometric Non-Rigid Structure-from-Motion

Non-Rigid Structure-from-Motion (NRSfM) reconstructs a deformable 3D object from the correspondences establishedbetween monocular 2D images. Current NRSfM methods lack statistical robustness, which is the ability to cope with correspondenceerrors. This prevents one to use automatically established correspondences, which are prone to errors, thereby strongly limiting thescope of NRSfM. We propose a three-step automatic pipeline to solve NRSfM robustly by exploiting isometry. Step(i)computes theoptical flow from correspondences, step(ii)reconstructs each 3D points normal vector using multiple reference images and integratesthem to form surfaces with the best reference and step(iii)rejects the 3D points that break isometry in their local neighborhood.Importantly, each step is designed to discard or flag erroneous correspondences. Our contributions include the robustification of opticalflow by warp estimation, new fast analytic solutions to local normal reconstruction and their robustification, and a newscale-independent measure of 3D local isometric coherence. Experimental results show that our robust NRSfM method consistentlyoutperforms existing methods on both synthetic and real datasets.