MECSE-12-2004 Simultaneous , robust fitting of multiple 3 D motion models

Multi-body structure-and-motion (MSaM) is the problem to establish the multiple-view geometry of several views of a 3D scene taken at different times, where the scene is not static, but consists of multiple rigid object moving relative to each other. In this report the case of two views is examined. The general setting is the following: given are a set of corresponding image points in two images, which originate from an unknown number of moving scene objects, each giving rise to a motion model. The correspondences can be constrained by either a fundamental matrix (non-planar object, general motion) or a homography (planar object or pure rotation). Furthermore, the measurement noise is unknown, and there are a number of gross errors, which are outliers to all models. The task is to find an optimal set of motion models to explain the measurements. The problem is a special case of the general problem of robust model selection and fitting. The proposed solution follows the recover-and-select paradigm: it randomly creates a large number of candidate models, estimates the scale of the noise for each model, and computes its goodness-of-fit. Then model selection is used to prune the redundant collection of models to an optimal set, including an outlier model.

[1]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[2]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[3]  H. C. Longuet-Higgins,et al.  A computer algorithm for reconstructing a scene from two projections , 1981, Nature.

[4]  Jorma Rissanen,et al.  Universal coding, information, prediction, and estimation , 1984, IEEE Trans. Inf. Theory.

[5]  Peter J. Rousseeuw,et al.  Robust Regression and Outlier Detection , 2005, Wiley Series in Probability and Statistics.

[6]  Richard I. Hartley,et al.  Estimation of Relative Camera Positions for Uncalibrated Cameras , 1992, ECCV.

[7]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.

[8]  C. Reeves.  Modern heuristic techniques for combinatorial problems , 1993 .

[9]  Ales Leonardis,et al.  ExSel++: A General Framework to Extract Parametric Models , 1995, CAIP.

[10]  Kenichi Kanatani,et al.  Statistical optimization for geometric computation - theory and practice , 1996, Machine intelligence and pattern recognition.

[11]  G. L. Bretthorst.  AN INTRODUCTION TO MODEL SELECTION USING PROBABILITY THEORY AS LOGIC , 1996 .

[12]  B. Triggs.  A New Approach to Geometric Fitting , 1996 .

[13]  Zhengyou Zhang,et al.  Parameter estimation techniques: a tutorial with application to conic fitting , 1997, Image Vis. Comput..

[14]  P. Torr.  Geometric motion segmentation and model selection , 1998, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[15]  O. Faugeras,et al.  The Geometry of Multiple Images , 1999 .

[16]  Stephen J. Maybank,et al.  MDL, Colllineations and the Fundamental Matrix , 1999, BMVC.

[17]  Amnon Shashua,et al.  Trajectory Triangulation: 3D Reconstruction of Moving Points from a Monocular Image Sequence , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Mei Han,et al.  Reconstruction of a Scene with Multiple Linearly Moving Objects , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[19]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[20]  Amnon Shashua,et al.  Multi-frame infinitesimal motion model for the reconstruction of (dynamic) scenes with multiple linearly moving objects , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[21]  Lior Wolf,et al.  Two-body segmentation from two perspective views , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[22]  Luc Van Gool,et al.  Surviving Dominant Planes in Uncalibrated Structure and Motion Recovery , 2002, ECCV.

[23]  S. Shankar Sastry,et al.  Two-View Segmentation of Dynamic Scenes from the Multibody Fundamental Matrix , 2002 .

[24]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[26]  Peter F. Sturm.  Structure and Motion for Dynamic Scenes - The Case of Points Moving in Planes , 2002, ECCV.

[27]  S. Shankar Sastry,et al.  Optimal segmentation of dynamic scenes from two perspective views , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[28]  Kenichi Kanatani,et al.  Geometric Information Criterion for Model Selection , 1998, International Journal of Computer Vision.

[29]  Philip H. S. Torr,et al.  Bayesian Model Estimation and Selection for Epipolar Geometry and Generic Manifold Fitting , 2002, International Journal of Computer Vision.

[30]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[31]  The multibody trifocal tensor: motion segmentation from 3 perspective views , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[32]  David Suter,et al.  MDPE: A Very Robust Estimator for Model Fitting and Range Image Segmentation , 2004, International Journal of Computer Vision.

[33]  David Suter,et al.  Robust Fitting by Adaptive-Scale Residual Consensus , 2004, ECCV.

[34]  Ruzena Bajcsy,et al.  Segmentation of range images as the search for geometric parametric models , 1995, International Journal of Computer Vision.

[35]  S. Sastry,et al.  Lecture Notes 2. an Invitation to 3-d Vision: from Images to Models (in Preparation) Image Formation Background , 2022 .