Automatic mosaicking and volume assembly for high-throughput serial-section transmission electron microscopy

We describe a computationally efficient and robust, fully-automatic method for large-scale electron microscopy image registration. The proposed method is able to construct large image mosaics from thousands of smaller, overlapping tiles with unknown or uncertain positions, and to align sections from a serial section capture into a common coordinate system. The method also accounts for nonlinear deformations both in constructing sections and in aligning sections to each other. The underlying algorithms are based on the Fourier shift property which allows for a computationally efficient and robust method. We demonstrate results on two electron microscopy datasets. We also quantify the accuracy of the algorithm through a simulated image capture experiment. The publicly available software tools include the algorithms and a Graphical User Interface for easy access to the algorithms.

[1]  N. Kasthuri,et al.  Automating the Collection of Ultrathin Serial Sections for Large Volume TEM Reconstructions , 2006, Microscopy and Microanalysis.

[2]  F. Amthor,et al.  Synaptic input to the on–off directionally selective ganglion cell in the rabbit retina , 2003, The Journal of comparative neurology.

[3]  Shmuel Peleg,et al.  Mosaicing on Adaptive Manifolds , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  A. Khotanzad,et al.  A physics-based coordinate transformation for 3-D image matching , 1997, IEEE Transactions on Medical Imaging.

[5]  Yuriy Mishchenko,et al.  Automation of 3D reconstruction of neural tissue from large volume of conventional serial section transmission electron micrographs , 2009, Journal of Neuroscience Methods.

[6]  Peter Sterling,et al.  Parallel Circuits from Cones to the On‐Beta Ganglion Cell , 1992, The European journal of neuroscience.

[7]  Takeo Kanade,et al.  Virtualized Reality: Constructing Virtual Worlds from Real Scenes , 1997, IEEE Multim..

[8]  Neill W Campbell,et al.  IEEE International Conference on Computer Vision and Pattern Recognition , 2008 .

[9]  Constantin Virgil Negoita,et al.  CYBERNETICS AND SOCIETY , 1982 .

[10]  Fred L. Bookstein,et al.  Principal Warps: Thin-Plate Splines and the Decomposition of Deformations , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  F. Bookstein,et al.  Morphometric Tools for Landmark Data: Geometry and Biology , 1999 .

[12]  Simon K. Warfield,et al.  Improved registration for large electron microscopy images , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[13]  Shmuel Peleg,et al.  Seamless Image Stitching in the Gradient Domain , 2004, ECCV.

[14]  C. D. Kuglin,et al.  The phase correlation image alignment method , 1975 .

[15]  Olaf Sporns,et al.  The Human Connectome: A Structural Description of the Human Brain , 2005, PLoS Comput. Biol..

[16]  Gregory Randall,et al.  Neuro3D: an interactive 3D reconstruction system of serial sections using automatic registration , 1998, Photonics West - Biomedical Optics.

[17]  W Hoppe,et al.  Three-dimensional electron microscopy. , 1981, Annual review of biophysics and bioengineering.

[18]  Kevin L. Briggman,et al.  Towards neural circuit reconstruction with volume electron microscopy techniques , 2006, Current Opinion in Neurobiology.

[19]  Stephan Saalfeld,et al.  Globally optimal stitching of tiled 3D microscopic image acquisitions , 2009, Bioinform..

[20]  David N Mastronarde,et al.  Automated electron microscope tomography using robust prediction of specimen movements. , 2005, Journal of structural biology.

[21]  Paul S. Heckbert,et al.  Graphics gems IV , 1994 .

[22]  Harry Shum,et al.  Correction to Construction of Panoramic Image Mosaics with Global and Local Alignment , 2001, International Journal of Computer Vision.

[23]  DH Hall,et al.  The posterior nervous system of the nematode Caenorhabditis elegans: serial reconstruction of identified neurons and complete pattern of synaptic interactions , 1991, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[24]  Kristen M Harris,et al.  Structural changes at dendritic spine synapses during long-term potentiation. , 2003, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[25]  C. Morandi,et al.  Registration of Translated and Rotated Images Using Finite Fourier Transforms , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Alan C. Evans,et al.  Anatomical-Functional Correlation Using an Adjustable MRI-Based Region of Interest Atlas with Positron Emission Tomography , 1988, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[27]  S. Brenner,et al.  The structure of the nervous system of the nematode Caenorhabditis elegans. , 1986, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[28]  Karel J. Zuiderveld,et al.  Contrast Limited Adaptive Histogram Equalization , 1994, Graphics Gems.

[29]  G. Knott,et al.  Serial Section Scanning Electron Microscopy of Adult Brain Tissue Using Focused Ion Beam Milling , 2008, The Journal of Neuroscience.

[30]  Kristen M. Harris,et al.  Synthesis of Research: Extending Unbiased Stereology of Brain Ultrastructure to Three-dimensional Volumes , 2001, J. Am. Medical Informatics Assoc..

[31]  Demetri Terzopoulos,et al.  Computer-assisted registration, segmentation, and 3D reconstruction from images of neuronal tissue sections , 1994, IEEE Trans. Medical Imaging.

[32]  D. Chklovskii,et al.  Wiring optimization can relate neuronal structure and function. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[33]  P. Anandan,et al.  Mosaic based representations of video sequences and their applications , 1995, Proceedings of IEEE International Conference on Computer Vision.

[34]  Karl Rohr,et al.  Approximating Thin-Plate Splines for Elastic Registration: Integration of Landmark Errors and Orientation Attributes , 1999, IPMI.

[35]  Michael I. Miller,et al.  Deformable templates using large deformation kinematics , 1996, IEEE Trans. Image Process..

[36]  D. Mastronarde,et al.  A Computational Framework for Ultrastructural Mapping of Neural Circuitry , 2009, PLoS biology.

[37]  Ruzena Bajcsy,et al.  Multiresolution elastic matching , 1989, Comput. Vis. Graph. Image Process..

[38]  Karl Rohr,et al.  Spline-based elastic image registration: integration of landmark errors and orientation attributes , 2003, Comput. Vis. Image Underst..

[39]  W. Denk,et al.  Serial Block-Face Scanning Electron Microscopy to Reconstruct Three-Dimensional Tissue Nanostructure , 2004, PLoS biology.

[40]  Jean-Philippe Thirion,et al.  New feature points based on geometric invariants for 3D image registration , 1996, International Journal of Computer Vision.

[41]  F. Bookstein,et al.  Morphometric Tools for Landmark Data: Geometry and Biology , 1999 .

[42]  James Davis,et al.  Mosaics of scenes with moving objects , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[43]  Jean-Philippe Thirion,et al.  Extremal points: definition and application to 3D image registration , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[44]  Mark H. Ellisman,et al.  Correlated three-dimensional light and electron microscopy reveals transformation of mitochondria during apoptosis , 2007, Nature Cell Biology.

[45]  Nicholas Ayache,et al.  Robust mosaicing with correction of motion distortions and tissue deformations for in vivo fibered microscopy , 2006, Medical Image Anal..