Outlier correction in image sequences for the affine camera

It is widely known that, for the affine camera model, both shape and motion can be factorized directly from the so-called image measurement matrix constructed from image point coordinates. The ability to extract both shape and motion from this matrix by a single SVD operation makes this shape-from-motion approach attractive; however, it can not deal with missing feature points and, in the presence of outliers, a direct SVD to the matrix would yield highly unreliable shape and motion components. Here, we present an outlier correction scheme that iteratively updates the elements of the image measurement matrix. The magnitude and sign of the update to each element is dependent upon the residual robustly estimated in each iteration. The result is that outliers are corrected and retained, giving improved reconstruction and smaller reprojection errors. Our iterative outlier correction scheme has been applied to both synthesized and real video sequences. The results obtained are remarkably good.

[1]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[2]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[3]  C. Jennison,et al.  Robust Statistics: The Approach Based on Influence Functions , 1987 .

[4]  Andrew Zisserman,et al.  Appendix—projective geometry for machine vision , 1992 .

[5]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Rachid Deriche,et al.  A Robust Technique for Matching two Uncalibrated Images Through the Recovery of the Unknown Epipolar Geometry , 1995, Artif. Intell..

[7]  Gunnar Sparr,et al.  Simultaneous reconstruction of scene structure and camera locations from uncalibrated image sequences , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[8]  Peter F. Sturm,et al.  A Factorization Based Algorithm for Multi-Image Projective Structure and Motion , 1996, ECCV.

[9]  Anders Heyden,et al.  Recursive structure and motion from image sequences using shape and depth spaces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Anders Heyden.  Projective Structure and Motion from Image Sequences using Subspace Methods , 1997 .

[11]  David W. Jacobs,et al.  Linear fitting with missing data: applications to structure-from-motion and to characterizing intensity images , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  Anders Heyden,et al.  Reconstruction from affine cameras using closure constraints , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[13]  Martial Hebert,et al.  Iterative projective reconstruction from multiple views , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[14]  Andrew Zisserman,et al.  A Six Point Solution for Structure and Motion , 2000, ECCV.

[15]  Anders Heyden,et al.  Outlier detection in video sequences under affine projection , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[16]  George Q. Chen,et al.  Robust point feature matching in projective space , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[17]  Henrik Aanæs,et al.  Robust Factorization , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  R. Hartley,et al.  PowerFactorization : 3D reconstruction with missing or uncertain data , 2003 .

[19]  B. Ripley,et al.  Robust Statistics , 2018, Wiley Series in Probability and Statistics.

[20]  P. Anandan,et al.  Factorization with Uncertainty , 2000, International Journal of Computer Vision.

[21]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.