Scalable and Interactive Segmentation and Visualization of Neural Processes in EM Datasets

Recent advances in scanning technology provide high resolution EM (electron microscopy) datasets that allow neuro-scientists to reconstruct complex neural connections in a nervous system. However, due to the enormous size and complexity of the resulting data, segmentation and visualization of neural processes in EM data is usually a difficult and very time-consuming task. In this paper, we present NeuroTrace, a novel EM volume segmentation and visualization system that consists of two parts: a semi-automatic multiphase level set segmentation with 3D tracking for reconstruction of neural processes, and a specialized volume rendering approach for visualization of EM volumes. It employs view-dependent on-demand filtering and evaluation of a local histogram edge metric, as well as on-the-fly interpolation and ray-casting of implicit surfaces for segmented neural structures. Both methods are implemented on the GPU for interactive performance. NeuroTrace is designed to be scalable to large datasets and data-parallel hardware architectures. A comparison of NeuroTrace with a commonly used manual EM segmentation tool shows that our interactive workflow is faster and easier to use for the reconstruction of complex neural processes.

[1]  Laurent D. Cohen,et al.  Fast extraction of minimal paths in 3D images and applications to virtual endoscopy , 2001, Medical Image Anal..

[2]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[3]  Jitendra Malik,et al.  Learning to detect natural image boundaries using local brightness, color, and texture cues , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Jitendra Malik,et al.  Using contours to detect and localize junctions in natural images , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Olaf Sporns,et al.  The Human Connectome: A Structural Description of the Human Brain , 2005, PLoS Comput. Biol..

[6]  Dirk Bartz,et al.  Virtual voyage: interactive navigation in the human colon , 1997, SIGGRAPH.

[7]  John Keyser,et al.  Visualization of Cellular and Microvascular Relationships , 2008, IEEE Transactions on Visualization and Computer Graphics.

[8]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  J C Fiala,et al.  Reconstruct: a free editor for serial section microscopy , 2005, Journal of microscopy.

[10]  Suyash P. Awate,et al.  MRI Tissue Classification with Neighborhood Statistics: A Nonparametric, Entropy-Minimizing Approach , 2005, MICCAI.

[11]  Eric L. Miller,et al.  Multiphase geometric couplings for the segmentation of neural processes , 2009, CVPR.

[12]  Thomas Ertl,et al.  GPU-Based Hyperstreamlines for Diffusion Tensor Imaging , 2006, EuroVis.

[13]  James A. Sethian,et al.  Level Set Methods and Fast Marching Methods , 1999 .

[14]  U. Clarenz,et al.  Towards fast non-rigid registration , 2003 .

[15]  Markus Hadwiger,et al.  Smooth Mixed-Resolution GPU Volume Rendering , 2008, VG/PBG@SIGGRAPH.

[16]  Ross T. Whitaker,et al.  A Fast Iterative Method for Eikonal Equations , 2008, SIAM J. Sci. Comput..

[17]  Guillermo Sapiro,et al.  An energy-based three-dimensional segmentation approach for the quantitative interpretation of electron tomograms , 2005, IEEE Transactions on Image Processing.

[18]  L. R. Dice.  Measures of the Amount of Ecologic Association Between Species , 1945 .

[19]  Vid Petrovic,et al.  Visualizing Whole-Brain DTI Tractography with GPU-based Tuboids and LoD Management , 2007, IEEE Transactions on Visualization and Computer Graphics.

[20]  Markus Hadwiger,et al.  Perspective Isosurface and Direct Volume Rendering for Virtual Endoscopy Applications , 2006, EuroVis.

[21]  Francis K. H. Quek,et al.  A review of vessel extraction techniques and algorithms , 2004, CSUR.

[22]  Dirk Bartz,et al.  Interactive exploration of extra- and interacranial blood vessels , 1999, Proceedings Visualization '99 (Cat. No.99CB37067).

[23]  Ross T. Whitaker,et al.  Axon tracking in serial block-face scanning electron microscopy , 2009, Medical Image Anal..

[24]  Alexander Borst,et al.  Contour-propagation algorithms for semi-automated reconstruction of neural processes , 2008, Journal of Neuroscience Methods.

[25]  Ross T. Whitaker,et al.  Interactive deformation and visualization of level set surfaces using graphics hardware , 2003, IEEE Visualization, 2003. VIS 2003..

[26]  Ross T. Whitaker,et al.  Enhancement of cell boundaries in transmission electron microscopy images , 2005, IEEE International Conference on Image Processing 2005.

[27]  Gordon L. Kindlmann,et al.  Semi-Automatic Generation of Transfer Functions for Direct Volume Rendering , 1998, VVS.

[28]  Markus Hadwiger,et al.  Real‐Time Ray‐Casting and Advanced Shading of Discrete Isosurfaces , 2005, Comput. Graph. Forum.

[29]  Otmar Scherzer,et al.  Inverse Problems, Image Analysis, and Medical Imaging , 2002 .

[30]  Yuriy Mishchenko,et al.  Automation of 3D reconstruction of neural tissue from large volume of conventional serial section transmission electron micrographs , 2009, Journal of Neuroscience Methods.

[31]  P. Anandan,et al.  A computational framework and an algorithm for the measurement of visual motion , 1987, International Journal of Computer Vision.

[32]  Stephen J Smith,et al.  Circuit reconstruction tools today , 2007, Current Opinion in Neurobiology.

[33]  Penny Rheingans,et al.  Texture-based Transfer Functions for Direct Volume Rendering , 2008, IEEE Transactions on Visualization and Computer Graphics.