Theorems and algorithms for multiple view geometry with applications to electron tomography

The thesis considers both theory and algorithms for geometric computer vision. The framework of the work is built around the application of autonomous transmission electron microscope image registration. The theoretical part of the thesis first develops a consistent robust estimator that is evaluated in estimating two view geometry with both affine and projective camera models. The uncertainty of the fundamental matrix is similarly estimated robustly, and the previous observation whether the covariance matrix of the fundamental matrix contains disparity information of the scene is explained and its utilization in matching is discussed. For point tracking purposes, a reliable wavelet-based matching technique and two EM algorithms for the maximum likelihood affine reconstruction under missing data are proposed. The thesis additionally discusses identification of degeneracy as well as affine bundle adjustment. The application part of the thesis considers transmission electron microscope image registration, first with fiducial gold markers and thereafter without markers. Both methods utilize the techniques proposed in the theoretical part of the thesis and, in addition, a graph matching method is proposed for matching gold markers. Conversely, alignment without markers is disposed by tracking interest points of the intensity surface of the images. At the present level of development, the former method is more accurate but the latter is appropriate for situations where fiducial markers cannot be used. Perhaps the most significant result of the thesis is the proposed robust estimator because of consistence proof and its many application areas, which are not limited to the computer vision field. The other algorithms could be found useful in multiple view applications in computer vision that have to deal with uncertainty, matching, tracking, and reconstruction. From the viewpoint of image registration, the thesis further achieved its aims since two accurate image alignment methods are suggested for obtaining the most exact reconstructions in electron tomography.

[1]  Joachim Frank,et al.  Electron Tomography , 1992, Springer US.

[2]  H. C. Longuet-Higgins,et al.  A computer algorithm for reconstructing a scene from two projections , 1981, Nature.

[3]  O. Faugeras,et al.  The Geometry of Multiple Images , 1999 .

[4]  Andrew Zisserman,et al.  MLESAC: A New Robust Estimator with Application to Estimating Image Geometry , 2000, Comput. Vis. Image Underst..

[5]  Gang Xu,et al.  Epipolar Geometry in Stereo, Motion and Object Recognition , 1996, Computational Imaging and Vision.

[6]  D Ress,et al.  Automatic acquisition of fiducial markers and alignment of images in tilt series for electron tomography. , 1999, Journal of electron microscopy.

[7]  J. Frank,et al.  Double-tilt electron tomography. , 1995, Ultramicroscopy.

[8]  R. K. Shyamasundar,et al.  Introduction to algorithms , 1996 .

[9]  Andrew Zisserman,et al.  Motion From Point Matches Using Affine Epipolar Geometry , 1994, ECCV.

[10]  Narendra Ahuja,et al.  Motion and Structure From Two Perspective Views: Algorithms, Error Analysis, and Error Estimation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Narendra Ahuja,et al.  Surfaces from Stereo: Integrating Feature Matching, Disparity Estimation, and Contour Detection , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[13]  Rachid Deriche,et al.  Dense Disparity Map Estimation Respecting Image Discontinuities: A PDE and Scale-Space BasedApproach , 2002, MVA.

[14]  D. Mastronarde.  Dual-axis tomography: an approach with alignment methods that preserve resolution. , 1997, Journal of structural biology.

[15]  Peng Zhang,et al.  A Highly Robust Estimator Through Partially Likelihood Function Modeling and Its Application in Computer Vision , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Richard I. Hartley,et al.  Euclidean Reconstruction from Uncalibrated Views , 1993, Applications of Invariance in Computer Vision.

[17]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[18]  G. Celeux,et al.  A Classification EM algorithm for clustering and two stochastic versions , 1992 .

[19]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[20]  Bill Triggs,et al.  Joint feature distributions for image correspondence , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[21]  Peter J. Rousseeuw,et al.  Robust Regression and Outlier Detection , 2005, Wiley Series in Probability and Statistics.

[22]  J. Frank,et al.  Alignment by Cross-Correlation , 1992 .

[23]  Narendra Ahuja,et al.  Motion and Structure from Line Correspondences; Closed-Form Solution, Uniqueness, and Optimization , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[25]  David W. Jacobs,et al.  Linear Fitting with Missing Data for Structure-from-Motion , 2001, Comput. Vis. Image Underst..

[26]  Philip H. S. Torr,et al.  The Development and Comparison of Robust Methods for Estimating the Fundamental Matrix , 1997, International Journal of Computer Vision.

[27]  Jukka Heikkonen,et al.  On the alignment of transmission electron microscope images without fiducial markers , 2002, Object recognition supported by user interaction for service robots.

[28]  C. J. Stone,et al.  Adaptive Maximum Likelihood Estimators of a Location Parameter , 1975 .

[29]  Olivier D. Faugeras,et al.  Characterizing the Uncertainty of the Fundamental Matrix , 1997, Comput. Vis. Image Underst..

[30]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[31]  Anders Heyden,et al.  Using conic correspondences in two images to estimate the epipolar geometry , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[32]  Hirotugu Akaike,et al.  On entropy maximization principle , 1977 .

[33]  Jukka Heikkonen,et al.  A Fully Automatic Alignment of Electron Tomography Images without Fiducial Markers , 2000, MVA.

[34]  Wilfried Brauer,et al.  Intensity- and Gradient-Based Stereo Matching Using Hierarchical Gaussian Basis Functions , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  M Marko,et al.  Three-dimensional transmission electron microscopy and its application to mitosis research. , 1999, Methods in cell biology.

[36]  Zhengyou Zhang,et al.  On the Optimization Criteria Used in Two-View Motion Analysis , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Richard I. Hartley,et al.  Projective Reconstruction and Invariants from Multiple Images , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  R. Guckenberger.  Determination of a common origin in the micrographs of tilt series in three-dimensional electron microscopy , 1982 .

[39]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[40]  D. Marquardt.  An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[41]  Haiying Liu,et al.  Uncalibrated stereo matching using DWT , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[42]  Olivier D. Faugeras,et al.  The fundamental matrix: Theory, algorithms, and stability analysis , 2004, International Journal of Computer Vision.

[43]  Wojciech Chojnacki,et al.  On the Fitting of Surfaces to Data with Covariances , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[44]  David W. Jacobs,et al.  Linear fitting with missing data: applications to structure-from-motion and to characterizing intensity images , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[45]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[46]  Anders Heyden,et al.  Affine Structure and Motion from Points, Lines and Conics , 1999, International Journal of Computer Vision.

[47]  D A Agard,et al.  Toward fully automated high-resolution electron tomography. , 1996, Journal of structural biology.

[48]  Harry Shum,et al.  Principal Component Analysis with Missing Data and Its Application to Polyhedral Object Modeling , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  S. Ullman,et al.  The interpretation of visual motion , 1977 .

[50]  Olaf Kübler,et al.  Simulation of neural contour mechanisms: from simple to end-stopped cells , 1992, Vision Research.

[51]  Jukka Heikkonen,et al.  A Bayesian weighting principle for the fundamental matrix estimation , 2000, Pattern Recognit. Lett..

[52]  Richard I. Hartley,et al.  In Defense of the Eight-Point Algorithm , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[53]  Jorma Rissanen,et al.  Stochastic Complexity in Statistical Inquiry , 1989, World Scientific Series in Computer Science.

[54]  Sami S. Brandt,et al.  Use of Shape Features in Content-Based Image Retrieval , 1999 .

[55]  R. Viswanathan,et al.  An introduction to statistical signal processing with applications , 1979 .

[56]  J Heikkonen,et al.  Automatic alignment of transmission electron microscope tilt series without fiducial markers. , 2001, Journal of structural biology.

[57]  R. Hogg.  Adaptive Robust Procedures: A Partial Review and Some Suggestions for Future Applications and Theory , 1974 .

[58]  Jukka Heikkonen,et al.  Multi-resolution matching of uncalibrated images utilizing epipolar geometry and its uncertainty , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[59]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[60]  D Marr,et al.  A computational theory of human stereo vision. , 1979, Proceedings of the Royal Society of London. Series B, Biological sciences.

[61]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[62]  Takeo Kanade,et al.  A unified factorization algorithm for points, line segments and planes with uncertainty models , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[63]  Takeo Kanade,et al.  A factorization method for affine structure from line correspondences , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[64]  Anders Heyden,et al.  Outlier detection in video sequences under affine projection , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[65]  Sami S. Brandt.  Maximum Likelihood Robust Regression with Known and Unknown Residual Models , 2002 .

[66]  Charles T. Loop,et al.  Estimating the Fundamental Matrix by Transforming Image Points in Projective Space , 2001, Comput. Vis. Image Underst..

[67]  New York Dover,et al.  ON THE CONVERGENCE PROPERTIES OF THE EM ALGORITHM , 1983 .

[68]  O. Faugeras,et al.  On determining the fundamental matrix : analysis of different methods and experimental results , 1993 .

[69]  Philip H. S. Torr,et al.  Outlier detection and motion segmentation , 1993, Other Conferences.

[70]  M. Brady,et al.  Rejecting outliers and estimating errors in an orthogonal-regression framework , 1995, Philosophical Transactions of the Royal Society of London. Series A: Physical and Engineering Sciences.

[71]  Jerome Sacks,et al.  AN ASYMPTOTICALLY EFFICIENT SEQUENCE OF ESTIMATORS OF A LOCATION PARAMETER , 1975 .

[72]  J. Rissanen.  A UNIVERSAL PRIOR FOR INTEGERS AND ESTIMATION BY MINIMUM DESCRIPTION LENGTH , 1983 .

[73]  Andrew Zisserman,et al.  Concerning Bayesian Motion Segmentation, Model, Averaging, Matching and the Trifocal Tensor , 1998, ECCV.

[74]  J Heikkonen,et al.  Multiphase method for automatic alignment of transmission electron microscope images using markers. , 2001, Journal of structural biology.

[75]  C. Burrus,et al.  Introduction to Wavelets and Wavelet Transforms: A Primer , 1997 .

[76]  Vladimir Cherkassky,et al.  Learning from Data: Concepts, Theory, and Methods , 1998 .

[77]  Takeo Kanade,et al.  A Paraperspective Factorization Method for Shape and Motion Recovery , 1994, ECCV.

[78]  Jukka Heikkonen,et al.  Automatic alignment of electron tomography images using markers , 2000, SPIE Optics East.

[79]  P. Anandan,et al.  Factorization with Uncertainty , 2000, ECCV.

[80]  Rachid Deriche,et al.  A Robust Technique for Matching two Uncalibrated Images Through the Recovery of the Unknown Epipolar Geometry , 1995, Artif. Intell..

[81]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[82]  Zhengyou Zhang,et al.  Determining the Epipolar Geometry and its Uncertainty: A Review , 1998, International Journal of Computer Vision.

[83]  Jacques Cohen,et al.  On the implementation of Strassen's fast multiplication algorithm , 2004, Acta Informatica.

[84]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[85]  B. Sewell,et al.  The application of the maximum entropy method to electron microscopic tomography. , 1989, Ultramicroscopy.

[86]  Jukka Heikkonen,et al.  Optimal method for the affine F-matrix and its uncertainty estimation in the sense of both noise and outliers , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[87]  Thomas S. Huang,et al.  Optimal multi-scale matching , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[88]  Jukka Heikkonen,et al.  A New Robust Bayesian Method for the Affine F-Matrix Estimation , 2000, VMV.

[89]  Jong Chan Lee,et al.  Image clustering using self-organizing feature map with refinement , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[90]  Cordelia Schmid,et al.  The Geometry and Matching of Lines and Curves Over Multiple Views , 2000, International Journal of Computer Vision.

[91]  Kenneth Levenberg.  A METHOD FOR THE SOLUTION OF CERTAIN NON – LINEAR PROBLEMS IN LEAST SQUARES , 1944 .

[92]  Hans P. Moravec.  Visual Mapping by a Robot Rover , 1979, IJCAI.

[93]  Azriel Rosenfeld,et al.  Gray-level corner detection , 1982, Pattern Recognit. Lett..

[94]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[95]  P. Rousseeuw.  Least Median of Squares Regression , 1984 .

[96]  Wojciech Chojnacki,et al.  A new approach to constrained parameter estimation applicable to some computer vision problems , 2002 .

[97]  Roger Mohr,et al.  Epipole and fundamental matrix estimation using virtual parallax , 1995, Proceedings of IEEE International Conference on Computer Vision.

[98]  Rudolf Beran,et al.  Asymptotically Efficient Adaptive Rank Estimates in Location Models , 1974 .

[99]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[100]  M. J. Hannah.  A system for digital stereo image matching , 1989 .

[101]  Ian D. Reid,et al.  Active tracking of foveated feature clusters using affine structure , 1996, International Journal of Computer Vision.

[102]  Adrian E. Raftery,et al.  How Many Clusters? Which Clustering Method? Answers Via Model-Based Cluster Analysis , 1998, Comput. J..

[103]  Cordelia Schmid,et al.  Evaluation of Interest Point Detectors , 2000, International Journal of Computer Vision.

[104]  Yakup Genc,et al.  Epipolar Geometry and Linear Subspace Methods: A New Approach to Weak Calibration , 2004, International Journal of Computer Vision.