Dense Semi-rigid Scene Flow Estimation from RGBD Images

Scene flow is defined as the motion field in 3D space, and can be computed from a single view when using an RGBD sensor. We propose a new scene flow approach that exploits the local and piecewise rigidity of real world scenes. By modeling the motion as a field of twists, our method encourages piecewise smooth solutions of rigid body motions. We give a general formulation to solve for local and global rigid motions by jointly using intensity and depth data. In order to deal efficiently with a moving camera, we model the motion as a rigid component plus a non-rigid residual and propose an alternating solver. The evaluation demonstrates that the proposed method achieves the best results in the most commonly used scene flow benchmark. Through additional experiments we indicate the general applicability of our approach in a variety of different scenarios.

[1]  Daniel Cremers,et al.  Anisotropic Huber-L1 Optical Flow , 2009, BMVC.

[2]  Geoffrey E. Hinton,et al.  Learning Generative Texture Models with extended Fields-of-Experts , 2009, BMVC.

[3]  Konrad Schindler,et al.  3D scene flow estimation with a rigid motion prior , 2011, 2011 International Conference on Computer Vision.

[4]  Wolfram Burgard,et al.  A benchmark for the evaluation of RGB-D SLAM systems , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Alexander M. Bronstein,et al.  Group-Valued Regularization for Analysis of Articulated Motion , 2012, ECCV Workshops.

[6]  James L. Crowley,et al.  Local/global scene flow estimation , 2013, 2013 IEEE International Conference on Image Processing.

[7]  James L. Crowley,et al.  Scene flow by tracking in intensity and depth data , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[8]  Daniel Cremers,et al.  The Natural Vectorial Total Variation Which Arises from Geometric Measure Theory , 2012, SIAM J. Imaging Sci..

[9]  Richard Szeliski,et al.  High-accuracy stereo depth maps using structured light , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[10]  Dieter Fox,et al.  RGB-D flow: Dense 3-D motion estimation using color and depth , 2013, 2013 IEEE International Conference on Robotics and Automation.

[11]  Bernd Jähne,et al.  Dense range flow from depth and intensity data , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[12]  Bodo Rosenhahn,et al.  Statistical and Geometrical Approaches to Visual Motion Analysis, International Dagstuhl Seminar, Dagstuhl Castle, Germany, July 13-18, 2008. Revised Papers , 2009, Lecture Notes in Computer Science.

[13]  Daniel Cremers,et al.  Efficient Dense Scene Flow from Sparse or Dense Stereo Data , 2008, ECCV.

[14]  Takeo Kanade,et al.  Three-dimensional scene flow , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Daniel Cremers,et al.  Robust odometry estimation for RGB-D cameras , 2013, 2013 IEEE International Conference on Robotics and Automation.

[16]  Matthieu Guillaumin,et al.  Segmentation Propagation in ImageNet , 2012, ECCV.

[17]  Junfeng Yang,et al.  A New Alternating Minimization Algorithm for Total Variation Image Reconstruction , 2008, SIAM J. Imaging Sci..

[18]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[19]  Edmond Boyer,et al.  Scene Flow from Depth and Color Images , 2011, BMVC.

[20]  Andrew J. Davison,et al.  Active Matching , 2008, ECCV.

[21]  Richard M. Murray,et al.  A Mathematical Introduction to Robotic Manipulation , 1994 .

[22]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[23]  Jitendra Malik,et al.  Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Yael Moses,et al.  Multi-view Scene Flow Estimation: A View Centered Variational Approach , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[25]  Frederic Devernay,et al.  A Variational Method for Scene Flow Estimation from Stereo Sequences , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[26]  Daniel Cremers,et al.  An Improved Algorithm for TV-L 1 Optical Flow , 2009, Statistical and Geometrical Approaches to Visual Motion Analysis.

[27]  Richard Bowden,et al.  Scene Particles: Unregularized Particle-Based Scene Flow Estimation , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.