Two-view multibody structure-and-motion with outliers through model selection

Multibody structure-and-motion (MSaM) is the problem in establishing the multiple-view geometry of several views of a 3D scene taken at different times, where the scene consists of multiple rigid objects moving relative to each other. We examine the case of two views. The setting is the following: Given are a set of corresponding image points in two images, which originate from an unknown number of moving scene objects, each giving rise to a motion model. Furthermore, the measurement noise is unknown, and there are a number of gross errors, which are outliers to all models. The task is to find an optimal set of motion models for the measurements. It is solved through Monte-Carlo sampling, careful statistical analysis of the sampled set of motion models, and simultaneous selection of multiple motion models to best explain the measurements. The framework is not restricted to any particular model selection mechanism because it is developed from a Bayesian viewpoint: different model selection criteria are seen as different priors for the set of moving objects, which allow one to bias the selection procedure for different purposes.

[1]  Gérard G. Medioni,et al.  Simultaneous two-view epipolar geometry estimation and motion segmentation by 4D tensor voting , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[3]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[4]  S. Shankar Sastry,et al.  Two-View Segmentation of Dynamic Scenes from the Multibody Fundamental Matrix , 2002 .

[5]  G. L. Bretthorst.  AN INTRODUCTION TO MODEL SELECTION USING PROBABILITY THEORY AS LOGIC , 1996 .

[6]  P. Torr.  Geometric motion segmentation and model selection , 1998, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[7]  S. Shankar Sastry,et al.  Optimal segmentation of dynamic scenes from two perspective views , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[8]  Kenichi Kanatani,et al.  Calibration of a Moving Camera Using a Planar Pattern: Optimal Computation, Relibility Evaluation, and Stabilization by Model Selection , 1999, ECCV.

[9]  René Vidal,et al.  A Unified Algebraic Approach to 2-D and 3-D Motion Segmentation , 2004, ECCV.

[10]  Kenichi Kanatani,et al.  Geometric Information Criterion for Model Selection , 1998, International Journal of Computer Vision.

[11]  Jorma Rissanen,et al.  Universal coding, information, prediction, and estimation , 1984, IEEE Trans. Inf. Theory.

[12]  K. Schindler.  Simultaneous, robust fitting of multiple 3D motion models , 2004 .

[13]  C. S. Wallace,et al.  An Information Measure for Classification , 1968, Comput. J..

[14]  Richard I. Hartley,et al.  Estimation of Relative Camera Positions for Uncalibrated Cameras , 1992, ECCV.

[15]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[16]  Amnon Shashua,et al.  Multi-frame infinitesimal motion model for the reconstruction of (dynamic) scenes with multiple linearly moving objects , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[17]  Peter J. Rousseeuw,et al.  Robust Regression and Outlier Detection , 2005, Wiley Series in Probability and Statistics.

[18]  Konrad Schindler.  Spatially consistent 3D motion segmentation , 2005, IEEE International Conference on Image Processing 2005.

[19]  P. Anandan,et al.  A unified approach to moving object detection in 2D and 3D scenes , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[20]  Jiri Matas,et al.  Robust wide-baseline stereo from maximally stable extremal regions , 2004, Image Vis. Comput..

[21]  Dean Phillips Foster,et al.  Calibration and Empirical Bayes Variable Selection , 1997 .

[22]  Philip H. S. Torr,et al.  Bayesian Model Estimation and Selection for Epipolar Geometry and Generic Manifold Fitting , 2002, International Journal of Computer Vision.

[23]  Peter F. Sturm.  Structure and Motion for Dynamic Scenes - The Case of Points Moving in Planes , 2002, ECCV.

[24]  S. Shankar Sastry,et al.  An Invitation to 3-D Vision , 2004 .

[25]  Richard I. Hartley,et al.  Projective Reconstruction and Invariants from Multiple Images , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[27]  O. Faugeras,et al.  The Geometry of Multiple Images , 1999 .

[28]  C. Reeves.  Modern heuristic techniques for combinatorial problems , 1993 .

[29]  Zhengyou Zhang,et al.  Parameter estimation techniques: a tutorial with application to conic fitting , 1997, Image Vis. Comput..

[30]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[31]  Lior Wolf,et al.  Two-body segmentation from two perspective views , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[32]  David Suter,et al.  MDPE: A Very Robust Estimator for Model Fitting and Range Image Segmentation , 2004, International Journal of Computer Vision.

[33]  Mei Han,et al.  Reconstruction of a Scene with Multiple Linearly Moving Objects , 2004, International Journal of Computer Vision.

[34]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[35]  David Suter,et al.  Robust Fitting by Adaptive-Scale Residual Consensus , 2004, ECCV.

[36]  Amnon Shashua,et al.  Trajectory Triangulation: 3D Reconstruction of Moving Points from a Monocular Image Sequence , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  P. Torr.  Model Selection for Structure and Motion Recovery from Multiple Images , 2000 .

[38]  Peter J. Rousseeuw,et al.  Robust regression and outlier detection , 1987 .

[39]  Kun Huang,et al.  Minimum effective dimension for mixtures of subspaces: a robust GPCA algorithm and its applications , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[40]  Kenichi Kanatani,et al.  Uncertainty modeling and model selection for geometric inference , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[42]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  Ales Leonardis,et al.  ExSel++: A General Framework to Extract Parametric Models , 1995, CAIP.

[44]  H. C. Longuet-Higgins,et al.  A computer algorithm for reconstructing a scene from two projections , 1981, Nature.

[45]  Ruzena Bajcsy,et al.  Segmentation of range images as the search for geometric parametric models , 1995, International Journal of Computer Vision.