Comparative analysis of tissue reconstruction algorithms for 3D histology

Abstract Motivation Digital pathology enables new approaches that expand beyond storage, visualization or analysis of histological samples in digital format. One novel opportunity is 3D histology, where a three-dimensional reconstruction of the sample is formed computationally based on serial tissue sections. This allows examining tissue architecture in 3D, for example, for diagnostic purposes. Importantly, 3D histology enables joint mapping of cellular morphology with spatially resolved omics data in the true 3D context of the tissue at microscopic resolution. Several algorithms have been proposed for the reconstruction task, but a quantitative comparison of their accuracy is lacking. Results We developed a benchmarking framework to evaluate the accuracy of several free and commercial 3D reconstruction methods using two whole slide image datasets. The results provide a solid basis for further development and application of 3D histology algorithms and indicate that methods capable of compensating for local tissue deformation are superior to simpler approaches. Availability and implementation Code: https://github.com/BioimageInformaticsTampere/RegBenchmark. Whole slide image datasets: http://urn.fi/urn: nbn: fi: csc-kata20170705131652639702. Supplementary information Supplementary data are available at Bioinformatics online.

[1]  Arrate Muñoz-Barrutia,et al.  3D reconstruction of histological sections: Application to mammary gland tissue , 2010, Microscopy research and technique.

[2]  Heikki Lehväslaiho,et al.  Three‐dimensional immersive virtual reality for studying cellular compartments in 3D models from EM preparations of neural tissues , 2015, The Journal of comparative neurology.

[3]  Michael Unser,et al.  A pyramid approach to subpixel registration based on intensity , 1998, IEEE Trans. Image Process..

[4]  Andrew Evans,et al.  Digital imaging in pathology: whole-slide imaging and beyond. , 2013, Annual review of pathology.

[5]  Donald R. Jones,et al.  A Taxonomy of Global Optimization Methods Based on Response Surfaces , 2001, J. Glob. Optim..

[6]  Erik Meijering,et al.  Imagining the future of bioimage analysis , 2016, Nature Biotechnology.

[7]  Rui Xu,et al.  Three-dimensional reconstruction of light microscopy image sections: present and future , 2015, Frontiers of Medicine.

[8]  Maik Stille,et al.  3D reconstruction of 2D fluorescence histology images and registration with in vivo MR images: Application in a rodent stroke model , 2013, Journal of Neuroscience Methods.

[9]  Christos Davatzikos,et al.  Methodology to study the three-dimensional spatial distribution of prostate cancer and their dependence on clinical parameters , 2015, Journal of medical imaging.

[10]  Jay B. West,et al.  Predicting error in rigid-body point-based registration , 1998, IEEE Transactions on Medical Imaging.

[11]  Vicente Grau,et al.  Transformation diffusion reconstruction of three-dimensional histology volumes from two-dimensional image stacks , 2017, Medical Image Anal..

[12]  Yukako Yagi,et al.  A Role of Three-Dimensional (3D)-Reconstruction in the Classification of Lung Adenocarcinoma , 2011, Analytical cellular pathology.

[13]  Rémy Prost,et al.  Robust Alignment of Prostate Histology Slices With Quantified Accuracy , 2013, IEEE Transactions on Biomedical Engineering.

[14]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[15]  Aaron Fenster,et al.  3D prostate histology image reconstruction: Quantifying the impact of tissue deformation and histology section location , 2013, Journal of pathology informatics.

[16]  Tao Ju,et al.  3D volume reconstruction of a mouse brain from histological sections using warp filtering , 2006, Journal of Neuroscience Methods.

[17]  Markus Löffler,et al.  Three-dimensional reconstruction and quantification of cervical carcinoma invasion fronts from histological serial sections , 2005, IEEE Transactions on Medical Imaging.

[18]  Omer Ishaq,et al.  Bridging Histology and Bioinformatics—Computational Analysis of Spatially Resolved Transcriptomics , 2017, Proceedings of the IEEE.

[19]  N. Ayache,et al.  Three-dimensional reconstruction of stained histological slices and 3D non-linear registration with in-vivo MRI for whole baboon brain , 2007, Journal of Neuroscience Methods.

[20]  Ben Loos,et al.  Virtual reality assisted microscopy data visualization and colocalization analysis , 2017, BMC Bioinformatics.

[21]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[22]  Carlos Ortiz-de-Solorzano,et al.  Consistent and Elastic Registration of Histological Sections Using Vector-Spline Regularization , 2006, CVAMIA.

[23]  Andreas K. Maier,et al.  A Gauss-Seidel Iteration Scheme for Reference-Free 3-D Histological Image Reconstruction , 2015, IEEE Transactions on Medical Imaging.

[24]  Allan R. Jones,et al.  Genome-wide atlas of gene expression in the adult mouse brain , 2007, Nature.

[25]  Darren Treanor,et al.  Histopathology in 3D: From three-dimensional reconstruction to multi-stain and multi-modal analysis , 2015, Journal of pathology informatics.

[26]  Derek R. Magee,et al.  3D reconstruction of multiple stained histology images , 2013, Journal of pathology informatics.

[27]  Jasper Snoek,et al.  Practical Bayesian Optimization of Machine Learning Algorithms , 2012, NIPS.

[28]  Nando de Freitas,et al.  Taking the Human Out of the Loop: A Review of Bayesian Optimization , 2016, Proceedings of the IEEE.

[29]  Li Bai,et al.  Smoothness-guided 3-D reconstruction of 2-D histological images , 2011, NeuroImage.

[30]  Johannes E. Schindelin,et al.  TrakEM2 Software for Neural Circuit Reconstruction , 2012, PloS one.

[31]  Colin Studholme,et al.  An overlap invariant entropy measure of 3D medical image alignment , 1999, Pattern Recognit..

[32]  Torsten Rohlfing,et al.  Image Similarity and Tissue Overlaps as Surrogates for Image Registration Accuracy: Widely Used but Unreliable , 2012, IEEE Transactions on Medical Imaging.

[33]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[34]  Aaron D. Ward,et al.  A Method for 3D Histopathology Reconstruction Supporting Mouse Microvasculature Analysis , 2015, PloS one.

[35]  Jon Griffin,et al.  Digital pathology in clinical use: where are we now and what is holding us back? , 2017, Histopathology.

[36]  Heidi Ledford.  The race to map the human body — one cell at a time , 2017, Nature.

[37]  Darren Treanor,et al.  Toward routine use of 3D histopathology as a research tool. , 2012, The American journal of pathology.

[38]  Alan C. Evans,et al.  BigBrain: An Ultrahigh-Resolution 3D Human Brain Model , 2013, Science.

[39]  Yi Gao,et al.  Algorithm sensitivity analysis and parameter tuning for tissue image segmentation pipelines , 2017, Bioinform..

[40]  Kevin W. Eliceiri,et al.  ImageJ‐MATLAB: a bidirectional framework for scientific image analysis interoperability , 2016, Bioinform..

[41]  Grégoire Malandain,et al.  Fusion of autoradiographs with an MR volume using 2-D and 3-D linear transformations , 2004, NeuroImage.

[42]  G. Allan Johnson,et al.  Waxholm Space: An image-based reference for coordinating mouse brain research , 2010, NeuroImage.

[43]  Carolina Wählby,et al.  Next-generation pathology--surveillance of tumor microecology. , 2015, Journal of molecular biology.

[44]  I. Ellis,et al.  Three-dimensional reconstruction of sentinel lymph nodes with metastatic breast cancer indicates three distinct patterns of tumour growth , 2009, Journal of Clinical Pathology.

[45]  Yukako Yagi,et al.  Evaluation of a completely automated tissue-sectioning machine for paraffin blocks. , 2012, Studies in health technology and informatics.

[46]  Patrik L. Ståhl,et al.  Visualization and analysis of gene expression in tissue sections by spatial transcriptomics , 2016, Science.

[47]  Johannes E. Schindelin,et al.  Fiji: an open-source platform for biological-image analysis , 2012, Nature Methods.

[48]  Nikos Paragios,et al.  Deformable Medical Image Registration: A Survey , 2013, IEEE Transactions on Medical Imaging.

[49]  S. Murphy,et al.  An assessment of methods for aligning two-dimensional microscope sections to create image volumes , 2008, Journal of Neuroscience Methods.

[50]  Adam D. Bull,et al.  Convergence Rates of Efficient Global Optimization Algorithms , 2011, J. Mach. Learn. Res..

[51]  A. Cardona,et al.  Elastic volume reconstruction from series of ultra-thin microscopy sections , 2012, Nature Methods.

[52]  Leena Latonen,et al.  Benchmarking of algorithms for 3D tissue reconstruction , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[53]  Genomics: Spatial transcriptomics , 2016, Nature Methods.

[54]  Kevin W Eliceiri,et al.  NIH Image to ImageJ: 25 years of image analysis , 2012, Nature Methods.