Structure and motion from road-driving stereo sequences

We introduce a unified framework for scene structure and motion estimation on road-driving stereo sequences. This framework is based on the slanted-plane scene model that has become widely popular in the stereo vision community. Our algorithm iteratively and alternately solves for scene structure and motion. Surface estimation is done using our own slanted-plane stereo algorithm. Motion estimation is achieved by solving a MRF labeling problem using Loopy Belief Propagation. We show that with some specific assumptions about the motion of the camera and the scene, the motion estimation problem can be reduced to a 1D search problem along the epipolar lines. We also propose a novel evaluation metrics, based on the notion of view prediction error. This metrics can be used to evaluate the performance of structure and motion estimation algorithms on stereo sequences without ground truth data. Experimental results on road-driving stereo sequences demonstrate that our algorithm successfully improves the view prediction error although it was not designed to directly optimize this quantity.

[1]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[2]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[3]  P. F. Felzenzwalb Efficiently computing a good segmentation , 1998 .

[4]  Pedro F. Felzenszwalb,et al.  Efficiently computing a good segmentation , 1998 .

[5]  Dragomir Anguelov,et al.  A General Algorithm for Approximate Inference and Its Application to Hybrid Bayes Nets , 1999, UAI.

[6]  Carlo Tomasi,et al.  Multiway cut for stereo and motion with slanted surfaces , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[7]  Ye Zhang,et al.  On 3D scene flow and structure estimation , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[8]  Pedro F. Felzenszwalb,et al.  Efficient belief propagation for early vision , 2004, CVPR 2004.

[9]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[10]  Ashutosh Saxena,et al.  Learning Depth from Single Monocular Images , 2005, NIPS.

[11]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[12]  Andreas Klaus,et al.  Segment-Based Stereo Matching Using Belief Propagation and a Self-Adapting Dissimilarity Measure , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[13]  D. Nistér,et al.  Stereo Matching with Color-Weighted Correlation, Hierarchical Belief Propagation, and Occlusion Handling , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Kwanghoon Sohn,et al.  Edge-preserving Simultaneous Joint Motion-Disparity Estimation , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[15]  Ashutosh Saxena,et al.  3-D Depth Reconstruction from a Single Still Image , 2007, International Journal of Computer Vision.

[16]  Ashutosh Saxena,et al.  Depth Estimation Using Monocular and Stereo Cues , 2007, IJCAI.

[17]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[18]  Frederic Devernay,et al.  A Variational Method for Scene Flow Estimation from Stereo Sequences , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[19]  David A. McAllester,et al.  Unsupervised Learning of Stereo Vision with Monocular Depth Cues , 2009, BMVC.

[20]  Zheng Zhi A Region Based Stereo Matching Algorithm Using Cooperative Optimization , 2009 .

[21]  David A. McAllester,et al.  Particle Belief Propagation , 2009, AISTATS.