Computer-aided prognosis: Predicting patient and disease outcome via quantitative fusion of multi-scale, multi-modal data

Computer-aided prognosis (CAP) is a new and exciting complement to the field of computer-aided diagnosis (CAD) and involves developing and applying computerized image analysis and multi-modal data fusion algorithms to digitized patient data (e.g. imaging, tissue, genomic) for helping physicians predict disease outcome and patient survival. While a number of data channels, ranging from the macro (e.g. MRI) to the nano-scales (proteins, genes) are now being routinely acquired for disease characterization, one of the challenges in predicting patient outcome and treatment response has been in our inability to quantitatively fuse these disparate, heterogeneous data sources. At the Laboratory for Computational Imaging and Bioinformatics (LCIB)(1) at Rutgers University, our team has been developing computerized algorithms for high dimensional data and image analysis for predicting disease outcome from multiple modalities including MRI, digital pathology, and protein expression. Additionally, we have been developing novel data fusion algorithms based on non-linear dimensionality reduction methods (such as Graph Embedding) to quantitatively integrate information from multiple data sources and modalities with the overarching goal of optimizing meta-classifiers for making prognostic predictions. In this paper, we briefly describe 4 representative and ongoing CAP projects at LCIB. These projects include (1) an Image-based Risk Score (IbRiS) algorithm for predicting outcome of Estrogen receptor positive breast cancer patients based on quantitative image analysis of digitized breast cancer biopsy specimens alone, (2) segmenting and determining extent of lymphocytic infiltration (identified as a possible prognostic marker for outcome in human epidermal growth factor amplified breast cancers) from digitized histopathology, (3) distinguishing patients with different Gleason grades of prostate cancer (grade being known to be correlated to outcome) from digitized needle biopsy specimens, and (4) integrating protein expression measurements obtained from mass spectrometry with quantitative image features derived from digitized histopathology for distinguishing between prostate cancer patients at low and high risk of disease recurrence following radical prostatectomy.

[1]  Marinette Revenu,et al.  Neighborhood graphs and image processing , 1996, Other Conferences.

[2]  Anant Madabhushi,et al.  Integrating structural and functional imaging for computer assisted detection of prostate cancer on multi-protocol in vivo 3 Tesla MRI , 2009, Medical Imaging.

[3]  Arend Heerschap,et al.  Combination of feature‐reduced MR spectroscopic and MR imaging data for improved brain tumor classification , 2005, NMR in biomedicine.

[4]  Paul L. Rosin,et al.  Selection of the optimal parameter value for the Isomap algorithm , 2006, Pattern Recognit. Lett..

[5]  Dimitris N. Metaxas,et al.  Automated detection of prostatic adenocarcinoma from high-resolution ex vivo MRI , 2005, IEEE Transactions on Medical Imaging.

[6]  S. Provencher Estimation of metabolite concentrations from localized in vivo proton NMR spectra , 1993, Magnetic resonance in medicine.

[7]  A. Madabhushi,et al.  Integrated diagnostics: a conceptual framework with examples , 2010, Clinical chemistry and laboratory medicine.

[8]  Volker Tresp,et al.  Heterogenous Data Fusion via a Probabilistic Latent-Variable Model , 2004, ARCS.

[9]  Vural Ozdemir,et al.  Mapping translational research in personalized therapeutics: from molecular markers to health policy. , 2007, Pharmacogenomics.

[10]  Arend Heerschap,et al.  A chemometric approach for brain tumor classification using magnetic resonance imaging and spectroscopy. , 2003, Analytical chemistry.

[11]  Jun Kong,et al.  Feature-based registration of histopathology images with different stains: An application for computerized follicular lymphoma prognosis , 2009, Comput. Methods Programs Biomed..

[12]  D. Grignon,et al.  Diagnostic and prognostic markers for human prostate cancer , 1997, The Prostate.

[13]  Harald Treuer,et al.  COMBINED X‐RAY AND MAGNETIC RESONANCE IMAGING FACILITY: APPLICATION TO IMAGE‐GUIDED STEREOTACTIC AND FUNCTIONAL NEUROSURGERY , 2007, Neurosurgery.

[14]  Gabriela Alexe,et al.  Towards Improved Cancer Diagnosis and Prognosis Using Analysis of Gene Expression Data and Computer Aided Imaging , 2009, Experimental biology and medicine.

[15]  Eric Bruno,et al.  Design of Multimodal Dissimilarity Spaces for Retrieval of Video Documents , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Lawrence J. Mazlack,et al.  Multi-modal Data Fusion: A Description , 2004, KES.

[17]  Atif R Mirza,et al.  An architectural selection framework for data fusion in sensor platforms , 2006 .

[18]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[19]  E. Mittendorf,et al.  High Expression of Lymphocyte-Associated Genes in Node-Negative HER2+ Breast Cancers Correlates with Lower Recurrence Rates , 2008 .

[20]  A. Stephenson,et al.  The value of radiotherapy in treating recurrent prostate cancer after radical prostatectomy , 2004, Nature Clinical Practice Urology.

[21]  D. Bostwick,et al.  Staging of early prostate cancer: a proposed tumor volume-based prognostic index. , 1993, Urology.

[22]  Jianbo Shi,et al.  Graph Embedding to Improve Supervised Classification and Novel Class Detection: Application to Prostate Cancer , 2005, MICCAI.

[23]  Gary Benson,et al.  Evaluating distance functions for clustering tandem repeats. , 2005, Genome informatics. International Conference on Genome Informatics.

[24]  William Stafford Noble,et al.  Support vector machine learning from heterogeneous data: an empirical analysis using protein sequence and structure , 2006, Bioinform..

[25]  M. Kattan,et al.  Preoperative nomogram predicting the 10-year probability of prostate cancer recurrence after radical prostatectomy. , 2006, Journal of the National Cancer Institute.

[26]  A. Shanberg,et al.  Ureteric and urethral stenosis: A complication of BK virus infection in a pediatric renal transplant patient , 2007, Pediatric transplantation.

[27]  Joseph C. Aman,et al.  An Evaluation of Information Content as a Metric for the Inference of Putative Conserved Noncoding Regions in DNA Sequences Using a Genetic Algorithms Approach , 2008, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[28]  Jing Li,et al.  Heterogeneous data fusion for alzheimer's disease study , 2008, KDD.

[29]  Lijun Jiang,et al.  Using locally estimated geodesic distance to optimize neighborhood graph for isometric data embedding , 2008, Pattern Recognit..

[30]  J. Sudbø,et al.  New Algorithms Based on the Voronoi Diagram Applied in a Pilot Study on Normal Mucosa and Carcinomas , 2000, Analytical cellular pathology : the journal of the European Society for Analytical Cellular Pathology.

[31]  Raphael Marcelpoil,et al.  Prognostic Value of Graph Theory-Based Tissue Architecture Analysis in Carcinomas of the Tongue , 2000, Laboratory Investigation.

[32]  C. Floyd,et al.  Optimized approach to decision fusion of heterogeneous data for breast cancer diagnosis. , 2006, Medical physics.

[33]  George Lee,et al.  Computer-aided prognosis: Predicting patient and disease outcome via multi-modal image analysis , 2010, 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[34]  L. Klotz Active surveillance for prostate cancer: for whom? , 2005, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[35]  Anant Madabhushi,et al.  A Boosted Bayesian Multiresolution Classifier for Prostate Cancer Detection From Digitized Needle Biopsies , 2012, IEEE Transactions on Biomedical Engineering.

[36]  Anant Madabhushi,et al.  Textural Kinetics: A Novel Dynamic Contrast-Enhanced (DCE)-MRI Feature for Breast Lesion Classification , 2011, Journal of Digital Imaging.

[37]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[38]  George Lee,et al.  An Empirical Comparison of Dimensionality Reduction Methods for Classifying Gene and Protein Expression Datasets , 2007, ISBRA.

[39]  R. Marcelpoil,et al.  Normalization of the minimum spanning tree. , 1993, Analytical cellular pathology : the journal of the European Society for Analytical Cellular Pathology.

[40]  D. Bostwick Grading prostate cancer. , 1994, American journal of clinical pathology.

[41]  Michael W Kattan,et al.  Defining biochemical recurrence of prostate cancer after radical prostatectomy: a proposal for a standardized definition. , 2006, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[42]  Anant Madabhushi,et al.  Computer-aided prognosis of ER+ breast cancer histopathology and correlating survival outcome with Oncotype DX assay , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[43]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[44]  A. Madabhushi,et al.  Investigating the Efficacy of Nonlinear Dimensionality Reduction Schemes in Classifying Gene and Protein Expression Studies , 2008, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[45]  Lei Guo,et al.  Brain tissue segmentation based on DTI data , 2007, NeuroImage.

[46]  B. Nicolas Bloch,et al.  An illustration of the potential for mapping MRI/MRS parameters with genetic over-expression profiles in human prostate cancer , 2008, Magnetic Resonance Materials in Physics, Biology and Medicine.

[47]  Michael W Kattan,et al.  Postoperative nomogram predicting the 10-year probability of prostate cancer recurrence after radical prostatectomy. , 2005, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[48]  Anant Madabhushi,et al.  A meta-classifier for detecting prostate cancer by quantitative integration of in vivo magnetic resonance spectroscopy and magnetic resonance imaging , 2008, SPIE Medical Imaging.

[49]  Andreas Rauber,et al.  Decision Manifolds—A Supervised Learning Algorithm Based on Self-Organization , 2008, IEEE Transactions on Neural Networks.

[50]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[51]  G. Bhanot,et al.  Identification of a microRNA panel for clear-cell kidney cancer. , 2010, Urology.

[52]  Christopher J. C. Burges,et al.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998, Data Mining and Knowledge Discovery.

[53]  Daniel Rueckert,et al.  Multiclassifier Fusion in Human Brain MR Segmentation: Modelling Convergence , 2006, MICCAI.

[54]  Gabriela Alexe,et al.  High expression of lymphocyte-associated genes in node-negative HER2+ breast cancers correlates with lower recurrence rates. , 2007, Cancer research.

[55]  Magnus Hellström,et al.  Proteomic analysis of protein expression in prostate cancer. , 2005, Analytical and quantitative cytology and histology.

[56]  Anant Madabhushi,et al.  A Hierarchical Unsupervised Spectral Clustering Scheme for Detection of Prostate Cancer from Magnetic Resonance Spectroscopy (MRS) , 2007, MICCAI.

[57]  Nello Cristianini,et al.  Kernel-Based Data Fusion and Its Application to Protein Function Prediction in Yeast , 2003, Pacific Symposium on Biocomputing.

[58]  A W Smeulders,et al.  An analysis of pathology knowledge and decision making for the development of artificial intelligence-based consulting systems. , 1989, Analytical and quantitative cytology and histology.

[59]  A. Khatami,et al.  Early Prostate Cancer, On prognostic markers and predictors of treatment outcome after radical prostatectomy. , 2007 .

[60]  David G. Stork,et al.  Pattern Classification , 1973 .

[61]  Karl J. Friston,et al.  Functional topography: multidimensional scaling and functional connectivity in the brain. , 1996, Cerebral cortex.

[62]  C G Roehrborn,et al.  Gleason scores from prostate biopsies obtained with 18-gauge biopsy needles poorly predict Gleason scores of radical prostatectomy specimens. , 1998, European urology.

[63]  Hans Lilja,et al.  Results of a randomized, population‐based study of biennial screening using serum prostate‐specific antigen measurement to detect prostate carcinoma , 2004, Cancer.

[64]  A. Madabhushi Digital pathology image analysis: opportunities and challenges. , 2009, Imaging in medicine.

[65]  A. W. Simonetti,et al.  The use of multivariate MR imaging intensities versus metabolic data from MR spectroscopic imaging for brain tumour classification. , 2005, Journal of magnetic resonance.

[66]  Giuseppe Lippi,et al.  Wisdom of theragnostics, other changes. , 2008, MLO: medical laboratory observer.

[67]  Torsten Rohlfing,et al.  Information Fusion in Biomedical Image Analysis: Combination of Data vs. Combination of Interpretations , 2005, IPMI.

[68]  T. Turkington,et al.  PET and brain tumor image fusion. , 2004, Cancer journal.

[69]  D. Gleason,et al.  Histologic grading of prostate cancer: a perspective. , 1992, Human pathology.

[70]  J. Sudbø,et al.  Caveats: Numerical Requirements in Graph Theory Based Quantitation of Tissue Architecture , 2000, Analytical cellular pathology : the journal of the European Society for Analytical Cellular Pathology.

[71]  Anant Madabhushi,et al.  A consensus embedding approach for segmentation of high resolution in vivo prostate magnetic resonance imagery , 2008, SPIE Medical Imaging.

[72]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[73]  Alain Cariou,et al.  Toward theragnostics , 2009, Critical care medicine.

[74]  Axel Saalbach,et al.  Image fusion for dynamic contrast enhanced magnetic resonance imaging , 2004, Biomedical engineering online.

[75]  D. Bostwick,et al.  Interobserver reproducibility of Gleason grading of prostatic carcinoma: urologic pathologists. , 2001, Human pathology.

[76]  Purang Abolmaesumi,et al.  High-throughput detection of prostate cancer in histological sections using probabilistic pairwise Markov models , 2010, Medical Image Anal..

[77]  Isabelle Bloch,et al.  Representation and fusion of heterogeneous fuzzy information in the 3D space for model-based structural recognition--Application to 3D brain imaging , 2003, Artif. Intell..

[78]  Anant Madabhushi,et al.  Spectral Embedding Based Probabilistic Boosting Tree (ScEPTre): Classifying High Dimensional Heterogeneous Biomedical Data , 2009, MICCAI.

[79]  Anant Madabhushi,et al.  AUTOMATED GRADING OF PROSTATE CANCER USING ARCHITECTURAL AND TEXTURAL IMAGE FEATURES , 2007, 2007 4th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[80]  C. Mascott,et al.  Image fusion of fluid-attenuated inversion recovery magnetic resonance imaging sequences for surgical image guidance. , 2007, Surgical neurology.

[81]  G. Bhanot,et al.  Manifold learning with graph-based features for identifying extent of lymphocytic infiltration from high grade , HER 2 + breast cancer histology , 2008 .

[82]  Jia Wei,et al.  Adaptive neighborhood selection for manifold learning , 2008, 2008 International Conference on Machine Learning and Cybernetics.

[83]  Robert E Lenkinski,et al.  Prostate cancer: accurate determination of extracapsular extension with high-spatial-resolution dynamic contrast-enhanced and T2-weighted MR imaging--initial results. , 2007, Radiology.

[84]  John K. Tsotsos,et al.  Parameterless Isomap with Adaptive Neighborhood Selection , 2006, DAGM-Symposium.

[85]  Anant Madabhushi,et al.  Automated grading of breast cancer histopathology using spectral clustering with textural and architectural image features , 2008, 2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[86]  Pierre I Karakiewicz,et al.  Can predictive models for prostate cancer patients derived in the United States of America be utilized in European patients? A validation study of the Partin tables. , 2003, European urology.

[87]  Andrew Janowczyk,et al.  Hierarchical Normalized Cuts: Unsupervised Segmentation of Vascular Biomarkers from Ovarian Cancer Tissue Microarrays , 2009, MICCAI.

[88]  Rainer Schrader,et al.  Fast and robust registration of PET and MR images of human brain , 2004, NeuroImage.

[89]  R Albert,et al.  Three-dimensional image processing for morphometric analysis of epithelium sections. , 1992, Cytometry.

[90]  George Lee,et al.  A knowledge representation framework for integration, classification of multi-scale imaging and non-imaging data: Preliminary results in predicting prostate cancer recurrence by fusing mass spectrometry and histology , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[91]  Abdul Ghapor Hussin,et al.  Computerized Medical Imaging and Graphics. , 2011 .

[92]  Amod Jog,et al.  Classifying Ayurvedic Pulse Signals Via Consensus Locally Linear Embedding , 2009, BIOSIGNALS.

[93]  Kazuyuki Aihara,et al.  Sequential Data Fusion via Vector Spaces: Fusion of Heterogeneous Data in the Complex Domain , 2007, J. VLSI Signal Process..

[94]  Hongyuan Zha,et al.  Adaptive Manifold Learning , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[95]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[96]  Jun Nakashima,et al.  Prognostic Implication of Microvascular Invasion in Biochemical Failure in Patients Treated with Radical Prostatectomy , 2003, Urologia Internationalis.

[97]  George Coukos,et al.  Intratumoral T cells, recurrence, and survival in epithelial ovarian cancer. , 2003, The New England journal of medicine.

[98]  David L. Hall,et al.  Perspectives on the fusion of image and non-image data , 2003, 32nd Applied Imagery Pattern Recognition Workshop, 2003. Proceedings..

[99]  Anant Madabhushi,et al.  Consensus-Locally Linear Embedding (C-LLE): Application to Prostate Cancer Detection on Magnetic Resonance Spectroscopy , 2008, MICCAI.

[100]  S. Dube,et al.  Content Based Image Retrieval for MR Image Studies of Brain Tumors , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[101]  Gyan Bhanot,et al.  Computerized Image-Based Detection and Grading of Lymphocytic Infiltration in HER2+ Breast Cancer Histopathology , 2010, IEEE Transactions on Biomedical Engineering.

[102]  Graeme P. Penney,et al.  Estimating and resolving uncertainty in cardiac respiratory motion modelling , 2012, 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI).

[103]  Shigeo Abe DrEng Pattern Classification , 2001, Springer London.