A Variational Framework for Structure from Motion in Omnidirectional Image Sequences

We address the problem of depth and ego-motion estimation from omnidirectional images. We propose a correspondence-free structure-from-motion problem for sequences of images mapped on the 2-sphere. A novel graph-based variational framework is first proposed for depth estimation between pairs of images. The estimation is cast as a TV-L1 optimization problem that is solved by a fast graph-based algorithm. The ego-motion is then estimated directly from the depth information without explicit computation of the optical flow. Both problems are finally addressed together in an iterative algorithm that alternates between depth and ego-motion estimation for fast computation of 3D information from motion in image sequences. Experimental results demonstrate the effective performance of the proposed algorithm for 3D reconstruction from synthetic and natural omnidirectional images.

[1]  Pascal Frossard,et al.  Optical flow and depth from motion for omnidirectional images using a TV-L1 variational framework on graphs , 2009, ICIP.

[2]  A. Jepson,et al.  A fast subspace algorithm for recovering rigid motion , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[3]  Andrew Blake,et al.  Robust estimation of egomotion from normal flow , 1994, International Journal of Computer Vision.

[4]  Guy Gilboa,et al.  Nonlocal Operators with Applications to Image Processing , 2008, Multiscale Model. Simul..

[5]  Berthold K. P. Horn,et al.  Direct methods for recovering motion , 1988, International Journal of Computer Vision.

[6]  A. Chambolle.  Practical, Unified, Motion and Missing Data Treatment in Degraded Video , 2004, Journal of Mathematical Imaging and Vision.

[7]  Andrew J. Davison,et al.  Active Matching , 2008, ECCV.

[8]  Shree K. Nayar,et al.  Ego-motion and omnidirectional cameras , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[9]  Kostas Daniilidis,et al.  Correspondence-free Structure from Motion , 2007, International Journal of Computer Vision.

[10]  Rémi Gribonval,et al.  Proceedings of EUSIPCO , 2011 .

[11]  Berthold K. P. Horn,et al.  Passive navigation , 1982, Comput. Vis. Graph. Image Process..

[12]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[13]  Dana H. Ballard,et al.  Computer Vision , 1982 .

[14]  A. Makadia,et al.  Image processing in catadioptric planes: spatiotemporal derivatives and optical flow computation , 2002, Proceedings of the IEEE Workshop on Omnidirectional Vision 2002. Held in conjunction with ECCV'02.

[15]  K. Hanna.  Direct multi-resolution estimation of ego-motion and structure from motion , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[16]  ANTONIN CHAMBOLLE,et al.  An Algorithm for Total Variation Minimization and Applications , 2004, Journal of Mathematical Imaging and Vision.

[17]  Mila Nikolova,et al.  Regularizing Flows for Constrained Matrix-Valued Images , 2004, Journal of Mathematical Imaging and Vision.

[18]  B. Schölkopf,et al.  A Regularization Framework for Learning from Graph Data , 2004, ICML 2004.

[19]  Olivier D. Faugeras,et al.  The geometry of multiple images - the laws that govern the formation of multiple images of a scene and some of their applications , 2001 .

[20]  Allan D. Jepson,et al.  Subspace methods for recovering rigid motion I: Algorithm and implementation , 2004, International Journal of Computer Vision.

[21]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[22]  Amit K Agruwul.  ROBUST EGO-MOTION ESTIMATION AND 3D MODEL REFINEMENT USING DEPTH BASED PARALLAX MODEL , 2004 .

[23]  Shree K. Nayar,et al.  A Theory of Single-Viewpoint Catadioptric Image Formation , 1999, International Journal of Computer Vision.

[24]  Carlo Tomasi,et al.  Comparison of approaches to egomotion computation , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[25]  Steven S. Beauchemin,et al.  The computation of optical flow , 1995, CSUR.

[26]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[27]  P. Vandergheynst,et al.  Optical flow and depth from motion for omnidirectional images using a TV-L1 variational framework on graphs , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[28]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[29]  Pascal Frossard,et al.  Multiresolution motion estimation for omnidirectional images , 2005, 2005 13th European Signal Processing Conference.

[30]  O. Faugeras,et al.  The Geometry of Multiple Images , 1999 .

[31]  J HeegerDavid,et al.  Subspace methods for recovering rigid motion I , 1992 .

[32]  Horst Bischof,et al.  A Duality Based Approach for Realtime TV-L1 Optical Flow , 2007, DAGM-Symposium.

[33]  Laurent D. Cohen,et al.  Non-local Regularization of Inverse Problems , 2008, ECCV.

[34]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[35]  Farhad Soleimanian Gharehchopogh,et al.  A Novel Approach for Edge Detection in Images Based on Cellular Learning Automata , 2012, Int. J. Comput. Vis. Image Process..

[36]  Tony F. Chan,et al.  Structure-Texture Image Decomposition—Modeling, Algorithms, and Parameter Selection , 2006, International Journal of Computer Vision.