Visual exploration and information analytics of high-dimensional medical images

Data visualization has transformed how we analyze increasingly large and complex data sets. Advanced visual tools logically represent data in a way that communicates the most important information inherent within it and culminate the analysis with an insightful conclusion. Automated analysis disciplines—such as data mining, machine learning, and statistics—have traditionally been the most dominant fields for data analysis. It has been complemented with a near-ubiquitous adoption of specialized hardware and software environments that handle the storage, retrieval, and pre- and postprocessing of digital data. The addition of interactive visualization tools allows an active human participant in the model creation process. The advantage is a data-driven approach where the constraints and assumptions of the model can be explored and chosen based on human insight and confirmed on demand by the analytic system. This translates to a better understanding of data and a more effective knowledge discovery. This trend has become very popular across various domains, not limited to machine learning, simulation, computer vision, genetics, stock market, data mining, and geography. In this dissertation, we highlight the role of visualization within the context of medical image analysis in the field of neuroimaging. The analysis of brain images has uncovered amazing traits about its underlying dynamics. Multiple image modalities capture qualitatively different internal brain mechanisms and abstract it within the information space of that modality. Computational studies based on these modalities help correlate the high-level brain function measurements with abnormal human behavior. These functional maps are easily projected in the physical space through accurate 3-D brain reconstructions and visualized in excellent detail from different anatomical vantage points. Statistical models built for comparative analysis across subject groups test for significant variance within the features and localize abnormal behaviors contextualizing the high-level brain activity. Currently, the task of identifying the features is based on empirical evidence, and preparing data for testing is time-consuming. Correlations among features are usually ignored due to lack of insight. With a multitude of features available and with new emerging modalities appearing, the process of identifying the salient features and their interdependencies becomes more difficult to perceive. This limits the analysis only to certain discernible features, thus limiting human judgments regarding the most important process that governs the symptom and hinders prediction. These shortcomings can be addressed using an analytical system that leverages data-driven techniques for guiding the user toward discovering relevant hypotheses. The research contributions within this dissertation encompass multidisciplinary fields of study not limited to geometry processing, computer vision, and 3-D visualization. However, the principal achievement of this research is the design and development of an interactive system for multimodality integration of medical images. The research proceeds in various stages, which are important to reach the desired goal. The different stages are briefly described as follows: First, we develop a rigorous geometry computation framework for brain surface matching. The brain is a highly convoluted structure of closed topology. Surface parameterization explicitly captures the non-Euclidean geometry of the cortical surface and helps derive a more accurate registration of brain surfaces. We describe a technique based on conformal parameterization that creates a bijective mapping to the canonical domain, where surface operations can be performed with improved efficiency and feasibility. Subdividing the brain into a finite set of anatomical elements provides the structural basis for a categorical division of anatomical view points and a spatial context for statistical analysis. We present statistically significant results of our analysis into functional and morphological features for a variety of brain disorders. Second, we design and develop an intelligent and interactive system for visual analysis of brain disorders by utilizing the complete feature space across all modalities. Each subdivided anatomical unit is specialized by a vector of features that overlap within that element. The analytical framework provides the necessary interactivity for exploration of salient features and discovering relevant hypotheses. It provides visualization tools for confirming model results and an easy-to-use interface for manipulating parameters for feature selection and filtering. It provides coordinated display views for visualizing multiple features across multiple subject groups, visual representations for highlighting interdependencies and correlations between features, and an efficient data-management solution for maintaining provenance and issuing formal data queries to the back end.

[1]  Jing Hua,et al.  Brain Structure Segmentation from MRI by Geometric Surface Flow , 2006, Int. J. Biomed. Imaging.

[2]  Nicholas Ayache,et al.  Spherical Demons: Fast Diffeomorphic Landmark-Free Surface Registration , 2010, IEEE Transactions on Medical Imaging.

[3]  Andreas Buja,et al.  Grand tour and projection pursuit , 1995 .

[4]  A. Toga,et al.  Detection and mapping of abnormal brain structure with a probabilistic atlas of cortical surfaces. , 1997, Journal of computer assisted tomography.

[5]  Fillia Makedon,et al.  Surface-based Morphometric Analysis for Hippocampal Shape in Schizophrenia , .

[6]  Cynthia A. Brewer,et al.  ColorBrewer.org: An Online Tool for Selecting Colour Schemes for Maps , 2003 .

[7]  Darshan Pai,et al.  Evaluation of fiber bundles across subjects through brain mapping and registration of diffusion tensor data , 2011, NeuroImage.

[8]  Paul M. Thompson,et al.  A Framework for Registration, Statistical Characterization and Classification of Cortically Constrained Functional Imaging Data , 2005, IPMI.

[9]  H. Singer,et al.  Neurobiology of Tourette's syndrome: concepts of neuroanatomic localization and neurochemical abnormalities , 2003, Brain and Development.

[10]  Paul M. Thompson,et al.  Inferring brain variability from diffeomorphic deformations of currents: An integrative approach , 2008, Medical Image Anal..

[11]  David H. Laidlaw,et al.  Exploring 3D DTI Fiber Tracts with Linked 2D Representations , 2009, IEEE Transactions on Visualization and Computer Graphics.

[12]  Jenq-Neng Hwang,et al.  A fast minimal path active contour model , 2001, IEEE Trans. Image Process..

[13]  Martin Styner,et al.  Statistical shape analysis of neuroanatomical structures based on medial models , 2003, Medical Image Anal..

[14]  M. Behen,et al.  Abnormal Fronto-striatal Connectivity in Children with Histories of Early Deprivation: A Diffusion Tensor Imaging Study , 2009, Brain Imaging and Behavior.

[15]  T. Chan,et al.  Genus zero surface conformal mapping and its application to brain surface mapping. , 2004, IEEE transactions on medical imaging.

[16]  Derek K. Jones,et al.  Spatial Normalization and Averaging of Diffusion Tensor MRI Data Sets , 2002, NeuroImage.

[17]  David C. Van Essen,et al.  Application of Information Technology: An Integrated Software Suite for Surface-based Analyses of Cerebral Cortex , 2001, J. Am. Medical Informatics Assoc..

[18]  Robin Sibson,et al.  What is projection pursuit , 1987 .

[19]  Demetri Terzopoulos,et al.  T-snakes: Topology adaptive snakes , 2000, Medical Image Anal..

[20]  Terry M. Peters,et al.  3D statistical neuroanatomical models from 305 MRI volumes , 1993, 1993 IEEE Conference Record Nuclear Science Symposium and Medical Imaging Conference.

[21]  Laurent D. Cohen,et al.  Global Minimum for Active Contour Models: A Minimal Path Approach , 1997, International Journal of Computer Vision.

[22]  Jeffrey Heer,et al.  Wrangler: interactive visual specification of data transformation scripts , 2011, CHI.

[23]  Lok Ming Lui,et al.  Optimization of Brain Conformal Mapping with Landmarks , 2005, MICCAI.

[24]  R. Kikinis,et al.  Automated segmentation of MR images of brain tumors. , 2001, Radiology.

[25]  Helwig Hauser,et al.  Parallel Sets: interactive exploration and visual analysis of categorical data , 2006, IEEE Transactions on Visualization and Computer Graphics.

[26]  Karl J. Friston,et al.  Voxel-Based Morphometry—The Methods , 2000, NeuroImage.

[27]  Ashraf A. Kassim,et al.  Medical Image Segmentation Using Minimal Path Deformable Models With Implicit Shape Priors , 2006, IEEE Transactions on Information Technology in Biomedicine.

[28]  Peter Z. Kunszt,et al.  Indexing the Sphere with the Hierarchical Triangular Mesh , 2007, ArXiv.

[29]  Nicholas Ayache,et al.  Medical Image Analysis: Progress over Two Decades and the Challenges Ahead , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Matthew O. Ward,et al.  Exploring N-dimensional databases , 1990, Proceedings of the First IEEE Conference on Visualization: Visualization `90.

[31]  Beat Kleiner,et al.  Graphical Methods for Data Analysis , 1983 .

[32]  Stefan Berchtold,et al.  Similarity clustering of dimensions for an enhanced visualization of multidimensional data , 1998, Proceedings IEEE Symposium on Information Visualization (Cat. No.98TB100258).

[33]  I. V. Ramakrishnan,et al.  Model-driven Visual Analytics , 2008, 2008 IEEE Symposium on Visual Analytics Science and Technology.

[34]  Marcus A. Magnor,et al.  Automated Analytical Methods to Support Visual Exploration of High-Dimensional Data , 2011, IEEE Transactions on Visualization and Computer Graphics.

[35]  Philip L. Bowers,et al.  Coordinate systems for conformal cerebellar flat maps , 2000, NeuroImage.

[36]  Alain Trouvé,et al.  Measuring Brain Variability Via Sulcal Lines Registration: A Diffeomorphic Approach , 2007, MICCAI.

[37]  Xianfeng Gu,et al.  A conformal and statistical mapping method for 3D PET image analysis , 2007 .

[38]  Carl-Fredrik Westin,et al.  Spatial normalization of diffusion tensor MRI using multiple channels , 2003, NeuroImage.

[39]  Jing Hua,et al.  Quantitative analysis of diffusion tensor images across subjects using probabilistic tractography , 2008, 2008 15th IEEE International Conference on Image Processing.

[40]  M. Torrens Co-Planar Stereotaxic Atlas of the Human Brain—3-Dimensional Proportional System: An Approach to Cerebral Imaging, J. Talairach, P. Tournoux. Georg Thieme Verlag, New York (1988), 122 pp., 130 figs. DM 268 , 1990 .

[41]  P. Basser,et al.  Estimation of the effective self-diffusion tensor from the NMR spin echo. , 1994, Journal of magnetic resonance. Series B.

[42]  KJ Worsley,et al.  SurfStat: A Matlab toolbox for the statistical analysis of univariate and multivariate surface and volumetric data using linear mixed effects models and random field theory , 2009, NeuroImage.

[43]  A. Dale,et al.  Cortical Surface-Based Analysis II: Inflation, Flattening, and a Surface-Based Coordinate System , 1999, NeuroImage.

[44]  Brian B. Avants,et al.  Shape averaging with diffeomorphic flows for atlas creation , 2004, 2004 2nd IEEE International Symposium on Biomedical Imaging: Nano to Macro (IEEE Cat No. 04EX821).

[45]  Randy L. Gollub,et al.  Reproducibility of quantitative tractography methods applied to cerebral white matter , 2007, NeuroImage.

[46]  Gabriele Lohmann,et al.  Extracting line representations of sulcal and gyral patterns in MR images of the human brain , 1998, IEEE Transactions on Medical Imaging.

[47]  Max A. Viergever,et al.  A survey of medical image registration , 1998, Medical Image Anal..

[48]  Chin Seng Chua,et al.  Point Signatures: A New Representation for 3D Object Recognition , 1997, International Journal of Computer Vision.

[49]  James C. Gee,et al.  Spatial transformations of diffusion tensor magnetic resonance images , 2001, IEEE Transactions on Medical Imaging.

[50]  Jing Hua,et al.  An Approach for Intersubject Analysis of 3D Brain Images Based on Conformal Geometry , 2006, 2006 International Conference on Image Processing.

[51]  David F. Tate,et al.  A Novel Interface for Interactive Exploration of DTI Fibers , 2009, IEEE Transactions on Visualization and Computer Graphics.

[52]  Jing Hua,et al.  Electrical, molecular and behavioral effects of interictal spiking in the rat , 2012, Neurobiology of Disease.

[53]  Jerry L. Prince,et al.  Cortical Surface Alignment Using Geometry Driven Multispectral Optical Flow , 2005, IPMI.

[54]  Karl J. Friston,et al.  Identifying global anatomical differences: Deformation‐based morphometry , 1998 .

[55]  Karl J. Friston,et al.  Statistical parametric maps in functional imaging: A general linear approach , 1994 .

[56]  O. Muzik,et al.  Statistical Parametric Mapping: Assessment of Application in Children , 2000, NeuroImage.

[57]  Paul M. Thompson,et al.  A framework for computational anatomy , 2002 .

[58]  Carl-Fredrik Westin,et al.  A Bayesian approach for stochastic white matter tractography , 2006, IEEE Transactions on Medical Imaging.

[59]  Linda G. Shapiro,et al.  Image Segmentation Techniques , 1984, Other Conferences.

[60]  Scott T. Grafton,et al.  Automated image registration: I. General methods and intrasubject, intramodality validation. , 1998, Journal of computer assisted tomography.

[61]  Alfred Inselberg,et al.  Parallel coordinates: a tool for visualizing multi-dimensional geometry , 1990, Proceedings of the First IEEE Conference on Visualization: Visualization `90.

[62]  D. Louis Collins,et al.  Retrospective evaluation of intersubject brain registration , 2003, IEEE Transactions on Medical Imaging.

[63]  Fred L. Bookstein,et al.  Principal Warps: Thin-Plate Splines and the Decomposition of Deformations , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[64]  Stephen M. Smith,et al.  A global optimisation method for robust affine registration of brain images , 2001, Medical Image Anal..

[65]  Michael I. Miller,et al.  Deformable templates using large deformation kinematics , 1996, IEEE Trans. Image Process..

[66]  Martin Rumpf,et al.  An image processing approach to surface matching , 2005, SGP '05.

[67]  Paul M. Thompson,et al.  Measuring brain variability by extrapolating sparse tensor fields measured on sulcal lines , 2007, NeuroImage.

[68]  Jerry L. Prince,et al.  Mapping Techniques for Aligning Sulci across Multiple Brains , 2003, MICCAI.

[69]  Paul M. Thompson,et al.  Automated Surface Matching Using Mutual Information Applied to Riemann Surface Structures , 2005, MICCAI.

[70]  P. Sandor,et al.  Speech disfluencies in individuals with Tourette syndrome. , 2005, Journal of psychosomatic research.

[71]  Daniel A. Keim,et al.  Information Visualization and Visual Data Mining , 2002, IEEE Trans. Vis. Comput. Graph..

[72]  Ali R. Khan,et al.  A Combined Surface And VOlumetric Registration (SAVOR) Framework to Study Cortical Biomarkers and Volumetric Imaging Data , 2009, MICCAI.

[73]  A. Bye,et al.  Application of statistical parametric mapping to SPET in the assessment of intractable childhood epilepsy , 2004, European Journal of Nuclear Medicine and Molecular Imaging.

[74]  S. Johansson,et al.  Interactive Dimensionality Reduction Through User-defined Combinations of Quality Metrics , 2009, IEEE Transactions on Visualization and Computer Graphics.

[75]  P. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 1999 .

[76]  Paul M. Thompson,et al.  Surface-Constrained Volumetric Brain Registration Using Harmonic Mappings , 2007, IEEE Transactions on Medical Imaging.

[77]  A. Dale,et al.  High‐resolution intersubject averaging and a coordinate system for the cortical surface , 1999, Human brain mapping.

[78]  Martial Hebert,et al.  Harmonic shape images: a three-dimensional free-form surface representation and its applications in surface matching , 2000 .

[79]  C. Davatzikos,et al.  Finding 3D parametric representations of the deep cortical folds , 1996, Proceedings of the Workshop on Mathematical Methods in Biomedical Image Analysis.

[80]  Hans-Peter Kriegel,et al.  VisDB: database exploration using multidimensional visualization , 1994, IEEE Computer Graphics and Applications.

[81]  Helwig Hauser,et al.  An Interaction View on Information Visualization , 2003, Eurographics.

[82]  Otto Muzik,et al.  Quantitative brain surface mapping of an electrophysiologic/metabolic mismatch in human neocortical epilepsy , 2009, Epilepsy Research.

[83]  G. Wahba Spline Interpolation and Smoothing on the Sphere , 1981 .

[84]  Maria Assunta Rocca,et al.  A method for obtaining tract-specific diffusion tensor MRI measurements in the presence of disease: application to patients with clinically isolated syndromes suggestive of multiple sclerosis , 2005, NeuroImage.

[85]  R. Woods,et al.  Mathematical/computational challenges in creating deformable and probabilistic atlases of the human brain , 2000, Human brain mapping.

[86]  Jing Hua,et al.  Coclustering for Cross-Subject Fiber Tract Analysis Through Diffusion Tensor Imaging , 2010, IEEE Transactions on Information Technology in Biomedicine.

[87]  M. Miller,et al.  Landmark Matching via Large Deformation Diffeomorphisms on the Sphere , 2004 .

[88]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[89]  John D. Van Horn,et al.  Circular representation of human cortical networks for subject and population-level connectomic visualization , 2012, NeuroImage.

[90]  Ben Shneiderman,et al.  A Rank-by-Feature Framework for Unsupervised Multidimensional Data Exploration Using Low Dimensional Projections , 2004, IEEE Symposium on Information Visualization.

[91]  Szymon Rusinkiewicz,et al.  Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors , 2003, Symposium on Geometry Processing.

[92]  Daniel A. Keim,et al.  An Automated Approach for the Optimization of Pixel-Based Visualizations , 2007, Inf. Vis..

[93]  P.K Sahoo,et al.  A survey of thresholding techniques , 1988, Comput. Vis. Graph. Image Process..

[94]  John W. Tukey,et al.  A Projection Pursuit Algorithm for Exploratory Data Analysis , 1974, IEEE Transactions on Computers.

[95]  Hong Qin,et al.  Rational spherical splines for genus zero shape modeling , 2005, International Conference on Shape Modeling and Applications 2005 (SMI' 05).

[96]  James F. O'Brien,et al.  Shape transformation using variational implicit functions , 1999, SIGGRAPH Courses.

[97]  Stephen M. Smith,et al.  Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm , 2001, IEEE Transactions on Medical Imaging.

[98]  Isaac N. Bankman,et al.  Handbook of medical image processing and analysis , 2009 .

[99]  Guillermo Sapiro,et al.  Geodesic Active Contours , 1995, International Journal of Computer Vision.

[100]  Samuel R. Buss,et al.  Spherical averages and applications to spherical splines and interpolation , 2001, TOGS.

[101]  Shiyong Lu,et al.  Segmentation of brain structures using PDE-driven surface growing , 2005, NAFIPS 2005 - 2005 Annual Meeting of the North American Fuzzy Information Processing Society.

[102]  J. Sethian,et al.  Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations , 1988 .

[103]  Ben Shneiderman,et al.  Interactively Exploring Hierarchical Clustering Results , 2003 .

[104]  Michael I. Miller,et al.  Volumetric transformation of brain anatomy , 1997, IEEE Transactions on Medical Imaging.

[105]  Jerry L Prince,et al.  Current methods in medical image segmentation. , 2000, Annual review of biomedical engineering.

[106]  O. Faugeras,et al.  Statistical shape influence in geodesic active contours , 2002, 5th IEEE EMBS International Summer School on Biomedical Imaging, 2002..

[107]  Matthew O. Ward,et al.  Visual Hierarchical Dimension Reduction for Exploration of High Dimensional Datasets , 2003, VisSym.

[108]  A Horsman,et al.  Tumour volume determination from MR images by morphological segmentation , 1996, Physics in medicine and biology.

[109]  V. Caselles,et al.  A geometric model for active contours in image processing , 1993 .

[110]  John G. Csernansky,et al.  Comparing surface-based and volume-based analyses of functional neuroimaging data in patients with schizophrenia , 2008, NeuroImage.

[111]  Kecheng Liu,et al.  Shape recovery algorithms using level sets in 2-D/3-D medical imagery: a state-of-the-art review , 2002, IEEE Transactions on Information Technology in Biomedicine.

[112]  Richard M. Leahy,et al.  Comparison of landmark-based and automatic methods for cortical surface registration , 2010, NeuroImage.

[113]  Diansheng Guo,et al.  Coordinating Computational and Visual Approaches for Interactive Feature Selection and Multivariate Clustering , 2003, Inf. Vis..

[114]  Matthew O. Ward,et al.  XmdvTool: integrating multiple methods for visualizing multivariate data , 1994, Proceedings Visualization '94.

[115]  Mert R. Sabuncu,et al.  Effects of Registration Regularization and Atlas Sharpness on Segmentation Accuracy , 2007, MICCAI.

[116]  Herman Chernoff,et al.  The Use of Faces to Represent Points in k- Dimensional Space Graphically , 1973 .

[117]  K J Zuiderveld,et al.  Normal fusion for three-dimensional integrated visualization of SPECT and magnetic resonance brain images. , 1997, Journal of nuclear medicine : official publication, Society of Nuclear Medicine.

[118]  Jake K. Aggarwal,et al.  Curvature-based representation of objects from range data , 1986, Image Vis. Comput..

[119]  U. Grenander,et al.  Statistical methods in computational anatomy , 1997, Statistical methods in medical research.

[120]  Nadia Magnenat-Thalmann,et al.  MRI Bone Segmentation Using Deformable Models and Shape Priors , 2008, MICCAI.

[121]  A. Dale,et al.  Whole Brain Segmentation Automated Labeling of Neuroanatomical Structures in the Human Brain , 2002, Neuron.

[122]  Daniel Asimov,et al.  The grand tour: a tool for viewing multidimensional data , 1985 .

[123]  Dong Soo Lee,et al.  (18)F-FDG PET in localization of frontal lobe epilepsy: comparison of visual and SPM analysis. , 2002, Journal of nuclear medicine : official publication, Society of Nuclear Medicine.

[124]  Pierre Dragicevic,et al.  Rolling the Dice: Multidimensional Visual Exploration using Scatterplot Matrix Navigation , 2008, IEEE Transactions on Visualization and Computer Graphics.

[125]  Robert L. Grossman,et al.  Graph-Theoretic Scagnostics , 2005, INFOVIS.

[126]  John P. Lewis,et al.  Eurographics/ Ieee-vgtc Symposium on Visualization 2009 Selecting Good Views of High-dimensional Data Using Class Consistency , 2022 .

[127]  Alan C. Evans,et al.  Structural asymmetries in the human brain: a voxel-based statistical analysis of 142 MRI scans. , 2001, Cerebral cortex.

[128]  K. Herholz,et al.  “3D-Tool” - A Software System for Visualisation and Analysis of Coregistered Multimodality Volume Datasets of Individual Subjects , 1998, NeuroImage.

[129]  Kiralee M. Hayashi,et al.  Mapping cortical change in Alzheimer's disease, brain development, and schizophrenia , 2004, NeuroImage.

[130]  Paul M. Thompson,et al.  Volumetric harmonic brain mapping , 2004, 2004 2nd IEEE International Symposium on Biomedical Imaging: Nano to Macro (IEEE Cat No. 04EX821).

[131]  Sen Wang,et al.  3D Surface Matching and Recognition Using Conformal Geometry , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[132]  Krzysztof Z. Gajos,et al.  Evaluation of Artery Visualizations for Heart Disease Diagnosis , 2011, IEEE Transactions on Visualization and Computer Graphics.

[133]  Ramana Rao,et al.  The table lens: merging graphical and symbolic representations in an interactive focus + context visualization for tabular information , 1994, CHI '94.