High performance image analysis for large histological datasets

The convergence of emerging challenges in biological research and developments in imaging and computing technologies suggests that image analysis will play an important role in providing a better understanding of biological phenomenon. The ability of imaging to localize molecular information is a key capability in the post-genomic era and will be critical in discovering the roles of genes and the relationships that connect them. The scale of the data in these emerging challenges is daunting; high throughput microscopy can generate hundreds of gigabytes to terabytes of high-resolution imagery even for studies limited in scope to a single gene or interaction. In addition to the scale of the data, the analysis of microscopic image content presents significant problems for the state-of-the-art in image analysis. This dissertation addresses two significant problems in the analysis of large histological images: reconstruction and tissue segmentation. The proposed methods form a framework that is intended to provide researchers with tools to explore and quantitatively analyze large image datasets. The works on reconstruction address several problems in the reconstruction of tissue from sequences of serial sections using image registration. A scalable algorithm for nonrigid registration is presented that features a novel method for the matching small nondescript anatomical features using geometric reasoning. Methods for the nonrigid registration of images with different stains are presented for two application scenarios. Correlation sharpness is proposed as a new measure for image similarity, and is used to map tumor suppressor gene expression to structure in mouse mammary tissues. An extended process of geometric reasoning based on the matching of cliques of anatomical features is presented and demonstrated for the nonrigid registration of immunohistochemical stain to hemotoxylin and eosin stain for human cancer images. Finally, a method for the incorporation of structural constraints into the reconstruction process is proposed and demonstrated on the reconstruction of ducts in mammary tissues. The work on tissue segmentation focuses on the use of statistical geometrical methods to describe the spatial distributions of biologically meaningful elements such as nuclei in tissue. The two point correlation function is demonstrated to be an effective feature for the segmentation of tissues, and is shown to possess a peculiar low-dimensional distribution in feature space that permits unsupervised segmentation by robust methods. The relationship between two-point functions for proximal image regions is derived and used to accelerate computation, resulting in a 7-68x improvement over a naive FFT-based implementation. In addition to the methods proposed for reconstruction and segmentation, a significant portion of this dissertation is devoted to applying high performance computing to enable the analysis of large datasets. In particular, multi-node parallelization as well as multi-core and general purpose computing on graphics processing are used to form a heterogeneous multiprocessor platform that is used to demonstrate the segmentation and reconstruction methods on images up to 62K × 23K in size.

[1]  C C Whitcomb,et al.  Morphological subclassification of follicular lymphoma: variability of diagnoses among hematopathologists, a collaborative study between the Repository Center and Pathology Panel for Lymphoma Clinical Studies. , 1985, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[2]  Christian Buske,et al.  Treatment strategies in follicular lymphomas: current status and future perspectives. , 2005, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[3]  Karl Rohr,et al.  Landmark-Based Image Analysis , 2001, Computational Imaging and Vision.

[4]  F. Stillinger,et al.  Modeling heterogeneous materials via two-point correlation functions. II. Algorithmic details and applications. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[5]  R. Baldock,et al.  3D modelling, gene expression mapping and post-mapping image analysis in the developing human brain , 2005, Brain Research Bulletin.

[6]  Bidyut Baran Chaudhuri,et al.  Semi-automatic segmentation of tissue cells from confocal microscope images , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[7]  Raghu Machiraju,et al.  Geometry-driven visualization of microscopic structures in biology , 2008, 2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[8]  Douglas G. Altman,et al.  Measurement in Medicine: The Analysis of Method Comparison Studies , 1983 .

[9]  Jean-Philippe Thiran,et al.  Morphological classification of cancerous cells , 1994, Proceedings of 1st International Conference on Image Processing.

[10]  Ruzena Bajcsy,et al.  Multiresolution elastic matching , 1989, Comput. Vis. Graph. Image Process..

[11]  Leonard T. Bruton,et al.  Recursive pruning of the 2D DFT with 3D signal processing applications , 1993, IEEE Trans. Signal Process..

[12]  Jens H. Krüger,et al.  A Survey of General‐Purpose Computation on Graphics Hardware , 2007, Eurographics.

[13]  Jean-Christophe Olivo-Marin,et al.  Color image segmentation based on Markov random field clustering for histological image analysis , 2002, Object recognition supported by user interaction for service robots.

[14]  G. Bearman,et al.  Biological imaging spectroscopy , 2001 .

[15]  Robert E Marc,et al.  Molecular Phenotyping of Retinal Ganglion Cells , 2002, The Journal of Neuroscience.

[16]  Laurent D. Cohen,et al.  A multiresolution algorithm for signal and image registration , 1997, Proceedings of International Conference on Image Processing.

[17]  Eugene W. Myers,et al.  Clustering gene expression patterns of fly embryos , 2006, 3rd IEEE International Symposium on Biomedical Imaging: Nano to Macro, 2006..

[18]  J. P. Lewis Fast Normalized Cross-Correlation , 2010 .

[19]  Roger Y. Tsien,et al.  Improved green fluorescence , 1995, Nature.

[20]  Ieee Xplore,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Information for Authors , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Min Gu,et al.  Penetration depth in multiphoton fluorescence microscopy , 2004, The Second Asian and Pacific Rim Symposium on Biophotonics, 2004. APBP 2004..

[22]  D. Hill,et al.  Medical image registration , 2001, Physics in medicine and biology.

[23]  Fred L. Bookstein,et al.  Principal Warps: Thin-Plate Splines and the Decomposition of Deformations , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Arnim Jenett,et al.  The Virtual Insect Brain protocol: creating and comparing standardized neuroanatomy , 2006, BMC Bioinformatics.

[25]  Joel H. Saltz,et al.  A Parallel Implementation of 4-Dimensional Haralick Texture Analysis for Disk-Resident Image Datasets , 2004, Proceedings of the ACM/IEEE SC2004 Conference.

[26]  Tony Pan,et al.  Automatic registration of large set of microscopic images using high-level features , 2006, 3rd IEEE International Symposium on Biomedical Imaging: Nano to Macro, 2006..

[27]  Teresa E. Alarcón,et al.  Blood Vessel Segmentation via Neural Network in Histological Images , 2003, J. Intell. Robotic Syst..

[28]  Peter H. Tu,et al.  Simultaneous estimation of segmentation and shape , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[29]  J M Bland,et al.  Statistical methods for assessing agreement between two methods of clinical measurement , 1986 .

[30]  Kenneth Moreland,et al.  The FFT on a GPU , 2003, HWWS '03.

[31]  Carlos Ortiz de Solorzano,et al.  Automatic segementation of histological structures in normal and neoplastic mammary gland tissue sections , 2003, SPIE BiOS.

[32]  R B Mann,et al.  Criteria for the cytologic subclassification of follicular lymphomas: A Proposed alternative method , 2007, Hematological oncology.

[33]  Tony Pan,et al.  Rb is critical in a mammalian tissue stem cell population. , 2007, Genes & development.

[34]  J. Capowski,et al.  A system for the three-dimensional reconstruction of biological structures. , 1983, Computers and biomedical research, an international journal.

[35]  R Fernandez-Gonzalez,et al.  Automatic segmentation of histological structures in mammary gland tissue sections. , 2004, Journal of biomedical optics.

[36]  Xuejun Yang,et al.  A Load-Balanced Parallel Algorithm for 2D Image Warping , 2004, ISPA.

[37]  Robert F. Murphy,et al.  Automated determination of protein subcellular locations from 3D fluorescence microscope images , 2002, Proceedings IEEE International Symposium on Biomedical Imaging.

[38]  E. Elonen,et al.  Tumours of haematopoietic and lymphoid tissues: general guidelines , 2004 .

[39]  Torsten Rohlfing,et al.  Nonrigid image registration in shared-memory multiprocessor environments with application to brains, breasts, and bees , 2003, IEEE Transactions on Information Technology in Biomedicine.

[40]  Michael Z. Lin,et al.  Mammalian Expression of Infrared Fluorescent Proteins Engineered from a Bacterial Phytochrome , 2009, Science.

[41]  Max A. Viergever,et al.  Mutual-information-based registration of medical images: a survey , 2003, IEEE Transactions on Medical Imaging.

[42]  Joel H. Saltz,et al.  Imaging and visual analysis - Large image correction and warping in a cluster environment , 2006, SC.

[43]  Anders Heyden,et al.  Segmentation of Histopathological Sections Using Snakes , 2003, SCIA.

[44]  Bill Hill,et al.  The Constrained Distance Transform : Interactive Atlas Registration With Large Deformations Through Constrained Distances , 2006 .

[45]  Anders Heyden,et al.  A Fast Snake Segmentation Method Applied to Histopathological Sections , 2003, EMMCVPR.

[46]  Chase E. Zachary,et al.  Statistical properties of determinantal point processes in high-dimensional Euclidean spaces. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[47]  F. Stillinger,et al.  Modeling heterogeneous materials via two-point correlation functions: basic principles. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[48]  Erkki Oja,et al.  Reduced Multidimensional Co-Occurrence Histograms in Texture Classification , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  Raghu Machiraju,et al.  MULTI-RESOLUTION IMAGE SEGMENTATION USING THE 2-POINT CORRELATION FUNCTIONS , 2007, 2007 4th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[50]  Thomas S. Huang,et al.  A fast two-dimensional median filtering algorithm , 1979 .

[51]  Serge Miguet,et al.  A Load-Balanced Algorithm for Parallel Digital Image Warping , 1999, Int. J. Pattern Recognit. Artif. Intell..

[52]  Nikolas P. Galatsanos,et al.  EM-based simultaneous registration, restoration, and interpolation of super-resolved images , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[53]  Haim J. Wolfson,et al.  Geometric hashing: an overview , 1997 .

[54]  Fumihiko Ino,et al.  A data distributed parallel algorithm for nonrigid image registration , 2005, Parallel Comput..

[55]  Adel Hafiane,et al.  Clustering initiated multiphase active contours and robust separation of nuclei groups for tissue segmentation , 2008, 2008 19th International Conference on Pattern Recognition.

[56]  CYRUS LEVINTHAL,et al.  Three Dimensional Reconstruction from Serial Sections , 1972, Nature.

[57]  John Wright,et al.  Segmentation of Multivariate Mixed Data via Lossy Data Coding and Compression , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  Raghu Machiraju,et al.  Image Segmentation with Tensor – Based Classification of N – Point Correlation Functions , 2006 .

[59]  R. Fernandez-Gonzalez,et al.  A tool for the quantitative spatial analysis of mammary gland epithelium , 2004, The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[60]  Kun Huang,et al.  Fast Automatic Registration Algorithm for Large Microscopy Images , 2006, 2006 IEEE/NLM Life Science Systems and Applications Workshop.

[61]  Pat Hanrahan,et al.  Understanding the efficiency of GPU algorithms for matrix-matrix multiplication , 2004, Graphics Hardware.

[62]  J. Capowski Computer-aided reconstruction of neuron trees from several serial sections. , 1977, Computers and biomedical research, an international journal.

[63]  M. Chalfie,et al.  Green fluorescent protein as a marker for gene expression. , 1994, Science.

[64]  M. Hadwiger,et al.  State of the Art Report 2004 on GPU-Based Segmentation , 2004 .

[65]  Jens Einenkel,et al.  Combined serial section‐based 3D reconstruction of cervical carcinoma invasion using H&E/p16INK4a/CD3 alternate staining , 2007, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[66]  K. Schughart,et al.  Computer-based three-dimensional visualization of developmental gene expression , 2000, Nature Genetics.

[67]  Kai Huang,et al.  Feature reduction for improved recognition of subcellular location patterns in fluorescence microscope images , 2003, SPIE BiOS.

[68]  Jake K. Aggarwal,et al.  AnEmpirical Evaluation ofGeneralized Cooccurrence Matrices , 1981 .

[69]  Salvatore Torquato,et al.  Dense sphere packings from optimized correlation functions. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[70]  Fumihiko Ino,et al.  A GPGPU Approach for Accelerating 2-D/3-D Rigid Registration of Medical Images , 2006, ISPA.

[71]  Joel H. Saltz,et al.  Tensor classification of N-point correlation function features for histology tissue segmentation , 2009, Medical Image Anal..

[72]  Bidyut Baran Chaudhuri,et al.  An efficient method based on watershed and rule-based merging for segmentation of 3-D histo-pathological images , 2001, Pattern Recognit..

[73]  Nico Scherf,et al.  Registration of Histologic Colour Images of Different Staining , 2006, Bildverarbeitung für die Medizin.

[74]  Mark W. Kieran,et al.  Identification of fibroblast heterogeneity in the tumor microenvironment , 2006, Cancer biology & therapy.

[75]  D. Hill,et al.  Non-rigid image registration: theory and practice. , 2004, The British journal of radiology.

[76]  Terry S. Yoo,et al.  Insight into Images: Principles and Practice for Segmentation, Registration, and Image Analysis , 2004 .

[77]  Ron Kikinis,et al.  A High Performance Computing Approach to the Registration of Medical Imaging Data , 1998, Parallel Comput..

[78]  P. Banks,et al.  Use of the working formulation for non-Hodgkin's lymphoma in epidemiologic studies: agreement between reported diagnoses and a panel of experienced pathologists. , 1987, Journal of the National Cancer Institute.

[79]  Arie E. Kaufman,et al.  GPU Cluster for High Performance Computing , 2004, Proceedings of the ACM/IEEE SC2004 Conference.

[80]  Kun Huang,et al.  Virtual Mouse Placenta: Tissue Layer Segmentation , 2005, 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference.

[81]  Guy Marchal,et al.  Multimodality image registration by maximization of mutual information , 1997, IEEE Transactions on Medical Imaging.

[82]  Nicholas Ayache,et al.  Medical image registration using geometric hashing , 1997 .

[83]  Joel H. Saltz,et al.  Ieee Transactions on Visualization and Computer Graphics Reconstruction of Cellular Biological Structures from Optical Microscopy Data , 2022 .

[84]  N. Otsu A threshold selection method from gray level histograms , 1979 .

[85]  Cornelis J H van de Velde,et al.  Four‐color staining combining fluorescence and brightfield microscopy for simultaneous immune cell phenotyping and localization in tumor tissue sections , 2005, Microscopy research and technique.

[86]  Nathan C Shaner,et al.  A guide to choosing fluorescent proteins , 2005, Nature Methods.

[87]  Markus Löffler,et al.  Three-dimensional reconstruction and quantification of cervical carcinoma invasion fronts from histological serial sections , 2005, IEEE Transactions on Medical Imaging.

[88]  R. Menzel,et al.  Three‐dimensional average‐shape atlas of the honeybee brain and its applications , 2005, The Journal of comparative neurology.

[89]  Sudipto Guha,et al.  Data Visualization and Mining using the GPU , 2011 .

[90]  Cenk Sokmensuer,et al.  Object-oriented texture analysis for the unsupervised segmentation of biopsy images for cancer detection , 2009, Pattern Recognit..

[91]  Taejung Kim,et al.  Automatic satellite image registration by combination of matching and random sample consensus , 2003, IEEE Trans. Geosci. Remote. Sens..

[92]  Stefan Heldmann,et al.  International Journal of Computer Vision c ○ 2006 Springer Science + Business Media, LLC. Manufactured in the United States. DOI: 10.1007/s11263-006-9780-x Image Registration of Sectioned Brains , 2004 .

[93]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[94]  Radhika Sivaramakrishna,et al.  Texture analysis of lesions in breast ultrasound images. , 2002, Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society.

[95]  Klaus Mueller,et al.  Visual Simulation of Heat Shimmering and Mirage , 2007, IEEE Transactions on Visualization and Computer Graphics.

[96]  S. Torquato Random Heterogeneous Materials , 2002 .

[97]  Hiroshi Inoue,et al.  REAL-TIME MUTUAL-INFORMATION-BASED LINEAR REGISTRATION ON THE CELL BROADBAND ENGINE PROCESSOR , 2007, 2007 4th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[98]  Jun Kong,et al.  Computerized microscopic image analysis of follicular lymphoma , 2008, SPIE Medical Imaging.

[99]  Luc Florack,et al.  The Topological Structure of Scale-Space Images , 2000, Journal of Mathematical Imaging and Vision.

[100]  J.-G. Postaire,et al.  A clustering method based on multidimensional texture analysis , 2006, Pattern Recognit..

[101]  Zhong Hu,et al.  A novel generic fast Fourier transform pruning technique and complexity analysis , 2005, IEEE Trans. Signal Process..

[102]  Pheng-Ann Heng,et al.  A hybrid condensed finite element model with GPU acceleration for interactive 3D soft tissue cutting: Research Articles , 2004 .

[103]  Charis Eng,et al.  PTEN: One Gene, Many Syndromes , 2003, Human mutation.

[104]  Linda G. Shapiro,et al.  Image Segmentation Techniques , 1984, Other Conferences.

[105]  Hang Zhang,et al.  Scaling Argument for the Amplitudes of Clustering Correlation Functions , 2000 .

[106]  Stefan Henn A Levenberg–Marquardt Scheme for Nonlinear Image Registration , 2003 .

[107]  Ross T. Whitaker,et al.  Implementation of an Automatic Slice-to-Slice Registration Tool , 2006 .

[108]  Joel H. Saltz,et al.  Histopathological Image Analysis Using Model-Based Intermediate Representations and Color Texture: Follicular Lymphoma Grading , 2009, J. Signal Process. Syst..

[109]  Franz Franchetti,et al.  Generating high performance pruned FFT implementations , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[110]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..