Feature‐based multi‐resolution registration of immunostained serial sections

&NA; The form and exact function of the blood vessel network in some human organs, like spleen and bone marrow, are still open research questions in medicine. In this paper, we propose a method to register the immunohistological stainings of serial sections of spleen and bone marrow specimens to enable the visualization and visual inspection of blood vessels. As these vary much in caliber, from mesoscopic (millimeter‐range) to microscopic (few micrometers, comparable to a single erythrocyte), we need to utilize a multi‐resolution approach. Our method is fully automatic; it is based on feature detection and sparse matching. We utilize a rigid alignment and then a non‐rigid deformation, iteratively dealing with increasingly smaller features. Our tool pipeline can already deal with series of complete scans at extremely high resolution, up to 620 megapixels. The improvement presented increases the range of represented details up to smallest capillaries. This paper provides details on the multi‐resolution non‐rigid registration approach we use. Our application is novel in the way the alignment and subsequent deformations are computed (using features, i.e. “sparse”). The deformations are based on all images in the stack (“global”). We also present volume renderings and a 3D reconstruction of the vascular network in human spleen and bone marrow on a level not possible before. Our registration makes easy tracking of even smallest blood vessels possible, thus granting experts a better comprehension. A quantitative evaluation of our method and related state of the art approaches with seven different quality measures shows the efficiency of our method. We also provide z‐profiles and enlarged volume renderings from three different registrations for visual inspection. HighlightsWe non‐rigidly register immunohistological serial sections of human specimen.Using feature detection and matching we iteratively compute non‐rigid deformations.Vascular networks in spleen and bone marrow are shown on a level not possible before.A quantitative evaluation of our method shows its efficiency. Graphical abstract Figure. No caption available.

[1]  Birte Steiniger,et al.  The Three-dimensional Structure of Human Splenic White Pulp Compartments , 2003, The journal of histochemistry and cytochemistry : official journal of the Histochemistry Society.

[2]  Nicholas Ayache,et al.  Rigid registration of 3-D ultrasound with MR images: a new approach combining intensity and gradient information , 2001, IEEE Transactions on Medical Imaging.

[3]  P. Wesseling,et al.  Three-dimensional reconstruction of tumor microvasculature: Simultaneous visualization of multiple components in paraffin-embedded tissue , 2006, Angiogenesis.

[4]  Birte Steiniger,et al.  Phenotypic differences between red pulp capillary and sinusoidal endothelia help localizing the open splenic circulation in humans , 2007, Histochemistry and Cell Biology.

[5]  Verena Wilhelmi,et al.  Heterogeneity of stromal cells in the human splenic white pulp. Fibroblastic reticulum cells, follicular dendritic cells and a third superficial stromal cell type , 2014, Immunology.

[6]  Li Bai,et al.  Smoothness-guided 3-D reconstruction of 2-D histological images , 2011, NeuroImage.

[7]  Adrien Bartoli,et al.  KAZE Features , 2012, ECCV.

[8]  Laurent Risser,et al.  Hybrid Feature-Based Diffeomorphic Registration for Tumor Tracking in 2-D Liver Ultrasound Images , 2013, IEEE Transactions on Medical Imaging.

[9]  Aaron D. Ward,et al.  A Method for 3D Histopathology Reconstruction Supporting Mouse Microvasculature Analysis , 2015, PloS one.

[10]  Pierre Vandergheynst,et al.  FREAK: Fast Retina Keypoint , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Stephan Saalfeld,et al.  As-rigid-as-possible mosaicking and serial section registration of large ssTEM datasets , 2010, Bioinform..

[12]  Oliver Schmitt,et al.  Robust and staining-invariant elastic registration of a series of images from histologic slices , 2005, SPIE Medical Imaging.

[13]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[14]  J. E. Glynn,et al.  Numerical Recipes: The Art of Scientific Computing , 1989 .

[15]  Brian B. Avants,et al.  Evaluation of Registration Methods on Thoracic CT: The EMPIRE10 Challenge , 2011, IEEE Transactions on Medical Imaging.

[16]  Bostjan Likar,et al.  A hierarchical approach to elastic registration based on mutual information , 2001, Image Vis. Comput..

[17]  R. Lenkinski,et al.  Elastic registration of multimodal prostate MRI and histology via multiattribute combined mutual information. , 2011, Medical physics.

[18]  Jan Flusser,et al.  Image registration methods: a survey , 2003, Image Vis. Comput..

[19]  Fabrice Heitz,et al.  A robust statistics-based global energy function for the alignment of serially acquired autoradiographic sections , 2003, Journal of Neuroscience Methods.

[20]  Tao Ju,et al.  3D volume reconstruction of a mouse brain from histological sections using warp filtering , 2006, Journal of Neuroscience Methods.

[21]  Stephen M. Smith,et al.  A global optimisation method for robust affine registration of brain images , 2001, Medical Image Anal..

[22]  Karl J. Friston,et al.  Spatial registration and normalization of images , 1995 .

[23]  Jean-Philippe Thirion,et al.  Image matching as a diffusion process: an analogy with Maxwell's demons , 1998, Medical Image Anal..

[24]  G. Marchal,et al.  Multi-modal volume registration by maximization of mutual information , 1997 .

[25]  Gunnar Farnebäck,et al.  Two-Frame Motion Estimation Based on Polynomial Expansion , 2003, SCIA.

[26]  Xiao Han.  Feature-constrained Nonlinear Registration of Lung CT Images , 2010 .

[27]  Nassir Navab,et al.  Deformable Registration of Multi-modal Microscopic Images Using a Pyramidal Interactive Registration-Learning Methodology , 2014, WBIR.

[28]  Nasir M. Rajpoot,et al.  A Nonlinear Mapping Approach to Stain Normalization in Digital Histopathology Images Using Image-Specific Color Deconvolution , 2014, IEEE Transactions on Biomedical Engineering.

[29]  Verena Wilhelmi,et al.  B lymphocyte compartments in the human splenic red pulp: capillary sheaths and periarteriolar regions , 2013, Histochemistry and Cell Biology.

[30]  Jan-Michael Frahm,et al.  Comparative Evaluation of Binary Features , 2012, ECCV.

[31]  Antonio Ruiz,et al.  Non-rigid Registration for Large Sets of Microscopic Images on Graphics Processors , 2009, J. Signal Process. Syst..

[32]  Anant Madabhushi,et al.  A novel point-based nonrigid image registration scheme based on learning optimal landmark configurations , 2013, Medical Imaging.

[33]  Ghassan Hamarneh,et al.  N-Sift: N-Dimensional Scale Invariant Feature Transform for Matching Medical Images , 2007, ISBI.

[34]  Li Bai,et al.  Registration of standardized histological images in feature space , 2008, SPIE Medical Imaging.

[35]  B. S. Manjunath,et al.  Biological imaging software tools , 2012, Nature Methods.

[36]  Johannes E. Schindelin,et al.  Fiji: an open-source platform for biological-image analysis , 2012, Nature Methods.

[37]  Charles V. Stewart,et al.  A Feature-Based, Robust, Hierarchical Algorithm for Registering Pairs of Images of the Curved Human Retina , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[39]  Sébastien Ourselin,et al.  Reconstructing a 3D structure from serial histological sections , 2001, Image Vis. Comput..

[40]  Xinjian Chen,et al.  Hierarchical Scale-Based Multiobject Recognition of 3-D Anatomical Structures , 2012, IEEE Transactions on Medical Imaging.

[41]  Nikos Paragios,et al.  Deformable Medical Image Registration: A Survey , 2013, IEEE Transactions on Medical Imaging.

[42]  Li Bai,et al.  Automatic Best Reference Slice Selection for Smooth Volume Reconstruction of a Mouse Brain From Histological Images , 2010, IEEE Transactions on Medical Imaging.

[43]  Haiying Liu,et al.  A Generic Framework for Non-rigid Registration Based on Non-uniform Multi-level Free-Form Deformations , 2001, MICCAI.

[44]  Tatsuo Ushiki,et al.  Three-dimensional reconstruction of serial sections for analysis of the microvasculature of the white pulp and the marginal zone in the human spleen. , 2015, Biomedical research.

[45]  Eliana Vasquez Osorio.  Non-rigid Registration , 2009 .

[46]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[47]  Ghassan Hamarneh,et al.  N-SIFT: N-DIMENSIONAL SCALE INVARIANT FEATURE TRANSFORM FOR MATCHING MEDICAL IMAGES , 2007, 2007 4th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[48]  Oleg Lobachev,et al.  Imaging the Vascular Network of the Human Spleen from Immunostained Serial Sections , 2014, VCBM.

[49]  Horst Bischof,et al.  SIFT and Shape Context for Feature-Based Nonlinear Registration of Thoracic CT Images , 2006, CVAMIA.

[50]  Pieter Wesseling,et al.  Three-dimensional (3D) reconstruction and quantitative analysis of the microvasculature in medulloblastoma and ependymoma subtypes , 2006, Angiogenesis.

[51]  A. Cardona,et al.  Elastic volume reconstruction from series of ultra-thin microscopy sections , 2012, Nature Methods.

[52]  Michael A. Saunders,et al.  LSMR: An Iterative Algorithm for Sparse Least-Squares Problems , 2011, SIAM J. Sci. Comput..

[53]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[54]  Richard A. Baldock,et al.  Robust Point Correspondence Applied to Two-and Three-Dimensional Image Registration , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[55]  Edward J. Delp,et al.  A multi-resolution approach to non-rigid registration of microscopy images , 2012, 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI).

[56]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[57]  Daniel Rueckert,et al.  Non-rigid registration using free-form deformations , 2015 .

[58]  J. Hermans,et al.  The splenic red pulp; a histomorphometrical study in splenectomy specimens embedded in methylmethacrylate , 1985, Histopathology.

[59]  Xuejun Ren,et al.  Rigid Registration , 2014, Computer Vision, A Reference Guide.

[60]  Michael A. Saunders,et al.  LSMR: An Iterative Algorithm for Sparse Least-Squares Problems , 2010, SIAM J. Sci. Comput..

[61]  Vitus Stachniss,et al.  Immunostaining of Pulpal Nerve Fibre Bundle/Arteriole Associations in Ground Serial Sections of Whole Human Teeth Embedded in Technovit® 9100 , 2013, Cells Tissues Organs.

[62]  Derek R. Magee,et al.  Unsupervised content classification based nonrigid registration of differently stained histology images , 2014, IEEE Transactions on Biomedical Engineering.

[63]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[64]  Erik Reinhard,et al.  Color Transfer between Images , 2001, IEEE Computer Graphics and Applications.

[65]  Simon R. Arridge,et al.  A survey of hierarchical non-linear medical image registration , 1999, Pattern Recognit..

[66]  Dan E. Tamir,et al.  Non-Rigid Image Registration , 2008, FLAIRS Conference.

[67]  Lisa M. Brown,et al.  A survey of image registration techniques , 1992, CSUR.

[68]  Rodney A. Kennedy,et al.  A Survey of Medical Image Registration on Multicore and the GPU , 2010, IEEE Signal Processing Magazine.

[69]  D. Hill,et al.  Non-rigid image registration: theory and practice. , 2004, The British journal of radiology.

[70]  Birte Steiniger,et al.  The Open Microcirculation in Human Spleens , 2011, The journal of histochemistry and cytochemistry : official journal of the Histochemistry Society.

[71]  Max A. Viergever,et al.  elastix: A Toolbox for Intensity-Based Medical Image Registration , 2010, IEEE Transactions on Medical Imaging.