Automatic TEM image alignment by trifocal geometry

Here we propose a novel method for automatic, markerless, feature‐based alignment of TEM images suitable for electron tomography. The proposed method, termed trifocal alignment, is more accurate than the previous markerless methods. The key components developed are: (1) a reliable multi‐resolution algorithm for matching feature points between images; (2) a robust, maximum‐likelihood‐based estimator for determining the geometry of three views – the trifocal constraint – required for validating the correctness of the matches; and (3) a robust, large‐scale optimization framework to compute the alignment parameters from hundreds of thousands of feature point measurements from a few hundred images. The ability to utilize such a large number of measurements successfully compensates for point localization errors. The method was experimentally confirmed with electron tomography tilt series of biological and material sciences samples, consisting of from 40 to 150 images. The results show that, with this feature‐based alignment approach, a level of accuracy comparable with fiducial marker alignment can be achieved.

[1]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[2]  S Jonić,et al.  Spectral signal-to-noise ratio and resolution assessment of 3D reconstructions. , 2005, Journal of structural biology.

[3]  Lawrence A. Ray.  Epipolar Geometry in Stereo, Motion and Object Recognition, by Gang Xu and Zhengyou Zhang , 1999 .

[4]  D. DeRosier,et al.  The reconstruction of a three-dimensional structure from projections and its application to electron microscopy , 1970, Proceedings of the Royal Society of London. A. Mathematical and Physical Sciences.

[5]  K. Taylor,et al.  Focus gradient correction applied to tilt series image data used in electron tomography. , 2003, Journal of structural biology.

[6]  Heymann Jb,et al.  Bsoft: Image and Molecular Processing in Electron Microscopy , 2001 .

[7]  Cordelia Schmid,et al.  Evaluation of Interest Point Detectors , 2000, International Journal of Computer Vision.

[8]  P. Anandan,et al.  Factorization with Uncertainty , 2000, International Journal of Computer Vision.

[9]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[10]  Sami S. Brandt,et al.  Theorems and algorithms for multiple view geometry with applications to electron tomography , 2002 .

[11]  U Ziese,et al.  Automated electron tomography of the septal pore cap in Rhizoctonia solani. , 2000, Journal of structural biology.

[12]  D A Agard,et al.  Toward fully automated high-resolution electron tomography. , 1996, Journal of structural biology.

[13]  J Heikkonen,et al.  Multiphase method for automatic alignment of transmission electron microscope images using markers. , 2001, Journal of structural biology.

[14]  Chia-Min Yang,et al.  Gold nanoparticles in SBA-15 showing catalytic activity in CO oxidation , 2003 .

[15]  J B Heymann,et al.  Bsoft: image and molecular processing in electron microscopy. , 2001, Journal of structural biology.

[16]  K A Taylor,et al.  The use of electron tomography for structural analysis of disordered protein arrays. , 1997, Journal of structural biology.

[17]  J. Frank,et al.  Alignment by Cross-Correlation , 1992 .

[18]  Rudolf Mester,et al.  Subspace Methods and Equilibration in Computer Vision , 1999 .

[19]  Peter Engelhardt,et al.  Electron Tomography of Chromosome Structure , 2006 .

[20]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[21]  Peter J. Huber,et al.  Robust Statistics , 2005, Wiley Series in Probability and Statistics.

[22]  Bruce F. McEwen,et al.  Three-dimensional tomographic reconstruction in high voltage electron microscopy , 1987 .

[23]  Rachid Deriche,et al.  A Robust Technique for Matching two Uncalibrated Images Through the Recovery of the Unknown Epipolar Geometry , 1995, Artif. Intell..

[24]  Thomas F. Coleman,et al.  An Interior Trust Region Approach for Nonlinear Minimization Subject to Bounds , 1993, SIAM J. Optim..

[25]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[26]  D. Mastronarde.  Dual-axis tomography: an approach with alignment methods that preserve resolution. , 1997, Journal of structural biology.

[27]  A. Verkleij,et al.  Automated high‐throughput electron tomography by pre‐calibration of image shifts , 2002, Journal of microscopy.

[28]  Richard H. Byrd,et al.  Approximate solution of the trust region problem by minimization over two-dimensional subspaces , 1988, Math. Program..

[29]  Joachim Frank,et al.  Electron Tomography , 1992, Springer US.

[30]  P. Midgley,et al.  3D electron microscopy in the physical sciences: the development of Z-contrast and EFTEM tomography. , 2003, Ultramicroscopy.

[31]  R. Guckenberger.  Determination of a common origin in the micrographs of tilt series in three-dimensional electron microscopy , 1982 .

[32]  D Ress,et al.  Automatic acquisition of fiducial markers and alignment of images in tilt series for electron tomography. , 1999, Journal of electron microscopy.

[33]  Giovanni Cardone,et al.  A resolution criterion for electron tomography based on cross-validation. , 2005, Journal of structural biology.

[34]  J Heikkonen,et al.  Automatic alignment of transmission electron microscope tilt series without fiducial markers. , 2001, Journal of structural biology.