Towards computational fluorescence microscopy: Machine learning-based integrated prediction of morphological and molecular tumor profiles

Recent advances in cancer research largely rely on new developments in microscopic or molecular profiling techniques offering high level of detail with respect to either spatial or molecular features, but usually not both. Here, we present a novel machine learning-based computational approach that allows for the identification of morphological tissue features and the prediction of molecular properties from breast cancer imaging data. This integration of microanatomic information of tumors with complex molecular profiling data, including protein or gene expression, copy number variation, gene methylation and somatic mutations, provides a novel means to computationally score molecular markers with respect to their relevance to cancer and their spatial associations within the tumor microenvironment.

[1]  W. Hoeffding Probability Inequalities for sums of Bounded Random Variables , 1963 .

[2]  Gerard Salton,et al.  A vector space model for automatic indexing , 1975, CACM.

[3]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[4]  P. Diggle,et al.  Spatial point pattern analysis and its application in geographical epidemiology , 1996 .

[5]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[6]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[7]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[8]  Pablo Tamayo,et al.  Gene set enrichment analysis: A knowledge-based approach for interpreting genome-wide expression profiles , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[9]  Frédéric Jurie,et al.  Sampling Strategies for Bag-of-Features Image Classification , 2006, ECCV.

[10]  Cheng Soon Ong,et al.  Multiclass multiple kernel learning , 2007, ICML '07.

[11]  Cor J. Veenman,et al.  Kernel Codebooks for Scene Categorization , 2008, ECCV.

[12]  Stefanie Nowak,et al.  Overview of the CLEF 2009 Large Scale - Visual Concept Detection and Annotation Task , 2009, CLEF.

[13]  Fabio A. González,et al.  Visual Pattern Analysis in Histopathology Images Using Bag of Features , 2009, CIARP.

[14]  A. Madabhushi,et al.  Histopathological Image Analysis: A Review , 2009, IEEE Reviews in Biomedical Engineering.

[15]  Fabio A. González,et al.  Histopathology Image Classification Using Bag of Features and Kernel Functions , 2009, AIME.

[16]  Koen E. A. van de Sande,et al.  Evaluating Color Descriptors for Object and Scene Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Stefanie Nowak,et al.  New Strategies for Image Annotation: Overview of the Photo Annotation Task at ImageCLEF 2010 , 2010, CLEF.

[18]  Yihong Gong,et al.  Locality-constrained Linear Coding for image classification , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[20]  Gunnar Rätsch,et al.  The SHOGUN Machine Learning Toolbox , 2010, J. Mach. Learn. Res..

[21]  Andrew H. Beck,et al.  Systematic Analysis of Breast Cancer Morphology Uncovers Stromal Features Associated with Survival , 2011, Science Translational Medicine.

[22]  Nicholas Ayache,et al.  Retrieval Evaluation and Distance Learning from Perceived Similarity between Endomicroscopy Videos , 2011, MICCAI.

[23]  Arnold W. M. Smeulders,et al.  The Visual Extent of an Object , 2011, International Journal of Computer Vision.

[24]  Fabio A. González,et al.  Visual pattern mining in histology image collections using bag of features , 2011, Artif. Intell. Medicine.

[25]  J. Uijlings,et al.  UvA-DARE ( Digital Academic Repository ) The visual extent of an object : suppose we know the object locations , 2011 .

[26]  R. Mirimanoff,et al.  Inhibition of the Kit Ligand/c-Kit Axis Attenuates Metastasis in a Mouse Model Mimicking Local Breast Cancer Relapse after Radiotherapy , 2012, Clinical Cancer Research.

[27]  Michael Y. Gerner,et al.  Histo-cytometry: a method for highly multiplex quantitative tissue imaging analysis applied to dendritic cell subset microanatomy in lymph nodes. , 2012, Immunity.

[28]  Lei Wang,et al.  What has my classifier learned? Visualizing the classification rules of bag-of-feature model by support region detection , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Fabio A. González,et al.  A Visual Latent Semantic Approach for Automatic Analysis and Interpretation of Anaplastic Medulloblastoma Virtual Slides , 2012, MICCAI.

[30]  Justin Andrew Powell,et al.  GO2MSIG, an automated GO based multi-species gene set generator for gene set enrichment analysis , 2014, BMC Bioinformatics.

[31]  Masaru Ishii,et al.  CognitionMaster: an object-based image analysis framework , 2013, Diagnostic Pathology.

[32]  Motoaki Kawanabe,et al.  Enhanced representation and multi-task learning for image annotation , 2013, Comput. Vis. Image Underst..

[33]  Ying Jiang,et al.  Foxo3a Expression Is a Prognostic Marker in Breast Cancer , 2013, PloS one.

[34]  D. Rimm Next-gen immunohistochemistry , 2014, Nature Methods.

[35]  Alexander Binder,et al.  On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation , 2015, PloS one.

[36]  W. Weichert,et al.  Classical pathology and mutational load of breast cancer – integration of two worlds , 2015, The journal of pathology. Clinical research.

[37]  Alexander Binder,et al.  Interpreting the Predictions of Complex ML Models by Layer-wise Relevance Propagation , 2016, ArXiv.

[38]  Alexander Binder,et al.  Analyzing Classifiers: Fisher Vectors and Deep Neural Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Ce Zhang,et al.  Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features , 2016, Nature Communications.

[40]  Yinyin Yuan Spatial Heterogeneity in the Tumor Microenvironment. , 2016, Cold Spring Harbor perspectives in medicine.

[41]  J. Harrell,et al.  Estrogen induces c-Kit and an aggressive phenotype in a model of invasive lobular breast cancer , 2017, Oncogenesis.

[42]  D. Ruderman,et al.  Abstract 540: Deep learning to determine breast cancer estrogen receptor status from nuclear morphometric features in H&E images , 2017 .

[43]  M. Salido,et al.  Defective Cyclin B1 Induction in Trastuzumab-emtansine (T-DM1) Acquired Resistance in HER2-positive Breast Cancer , 2017, Clinical Cancer Research.

[44]  Alexander Binder,et al.  Evaluating the Visualization of What a Deep Neural Network Has Learned , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[45]  Alexander Binder,et al.  Explaining nonlinear classification decisions with deep Taylor decomposition , 2015, Pattern Recognit..

[46]  Thomas J. Fuchs,et al.  Terabyte-scale Deep Multiple Instance Learning for Classification and Localization in Pathology , 2018, ArXiv.