Deep Neural Network for Analysis of DNA Methylation Data

Many researches demonstrated that the DNA methylation, which occurs in the context of a CpG, has strong correlation with diseases, including cancer. There is a strong interest in analyzing the DNA methylation data to find how to distinguish different subtypes of the tumor. However, the conventional statistical methods are not suitable for analyzing the highly dimensional DNA methylation data with bounded support. In order to explicitly capture the properties of the data, we design a deep neural network, which composes of several stacked binary restricted Boltzmann machines, to learn the low dimensional deep features of the DNA methylation data. Experiments show these features perform best in breast cancer DNA methylation data cluster analysis, comparing with some state-of-the-art methods.

[1]  Jun Guo,et al.  Cross-modal subspace learning for sketch-based image retrieval: A comparative study , 2016, 2016 IEEE International Conference on Network Infrastructure and Digital Content (IC-NIDC).

[2]  N. Nasios,et al.  Variational learning for Gaussian mixture models , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[3]  Nizar Bouguila,et al.  Bayesian learning of inverted Dirichlet mixtures for SVM kernels generation , 2013, Neural Computing and Applications.

[4]  Honglak Lee,et al.  Learning and Selecting Features Jointly with Point-wise Gated Boltzmann Machines , 2013, ICML.

[5]  Andrew R. Jamieson,et al.  Exploring nonlinear feature space dimension reduction and data representation in breast Cadx with Laplacian eigenmaps and t-SNE. , 2009, Medical physics.

[6]  José M. Bioucas-Dias,et al.  Hyperspectral Unmixing Based on Mixtures of Dirichlet Components , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[7]  Olivier Thonnard,et al.  An Experimental Study of Diversity with Off-the-Shelf AntiVirus Engines , 2009, 2009 Eighth IEEE International Symposium on Network Computing and Applications.

[8]  Jun Guo,et al.  Spoofing Detection in Automatic Speaker Verification Systems Using DNN Classifiers and Dynamic Acoustic Features , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Honggang Zhang,et al.  Variational Bayesian Matrix Factorization for Bounded Support Data , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  B. Everitt,et al.  Finite Mixture Distributions , 1981 .

[11]  Xiao Zhang,et al.  Comparison of Beta-value and M-value methods for quantifying methylation levels by microarray analysis , 2010, BMC Bioinformatics.

[12]  Markus Flierl,et al.  Probabilistic Multiview Depth Image Enhancement Using Variational Inference , 2015, IEEE Journal of Selected Topics in Signal Processing.

[13]  Zhanyu Ma.  Bayesian estimation of the Dirichlet distribution with expectation propagation , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[14]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine-mediated learning.

[15]  P. Green,et al.  Corrigendum: On Bayesian analysis of mixtures with an unknown number of components , 1997 .

[16]  Pasin Israsena,et al.  EEG-Based Emotion Recognition Using Deep Learning Network with Principal Component Based Covariate Shift Adaptation , 2014, TheScientificWorldJournal.

[17]  Guoli Wang,et al.  LS-NMF: A modified non-negative matrix factorization algorithm utilizing uncertainty estimates , 2006, BMC Bioinformatics.

[18]  Nizar Bouguila,et al.  Unsupervised selection of a finite Dirichlet mixture model: an MML-based approach , 2006, IEEE Transactions on Knowledge and Data Engineering.

[19]  A. Bird,et al.  CpG islands and the regulation of transcription. , 2011, Genes & development.

[20]  Qie Sun,et al.  Statistical analysis of energy consumption patterns on the heat demand of buildings in district heating systems , 2014 .

[21]  Weichung Joe Shih,et al.  A mixture model for estimating the local false discovery rate in DNA microarray analysis , 2004, Bioinform..

[22]  Alfred O. Hero,et al.  A Survey of Stochastic Simulation and Optimization Methods in Signal Processing , 2015, IEEE Journal of Selected Topics in Signal Processing.

[23]  Jaehoon Jung,et al.  Capacity and Error Probability Analysis of Diversity Reception Schemes Over Generalized- $K$ Fading Channels Using a Mixture Gamma Distribution , 2014, IEEE Transactions on Wireless Communications.

[24]  Arne Leijon,et al.  Human audio-visual consonant recognition analyzed with three bimodal integration models , 2009, INTERSPEECH.

[25]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[26]  Jun Guo,et al.  Dirichlet mixture modeling to estimate an empirical lower bound for LSF quantization , 2014, Signal Process..

[27]  Jun Guo,et al.  Line spectral frequencies modeling by a mixture of von Mises-Fisher distributions , 2015, Signal Process..

[28]  Lei Huang,et al.  Bayesian Information Criterion for Source Enumeration in Large-Scale Adaptive Antenna Array , 2016, IEEE Transactions on Vehicular Technology.

[29]  Arne Leijon,et al.  Vector quantization of LSF parameters with a mixture of dirichlet distributions , 2013, IEEE Transactions on Audio, Speech, and Language Processing.

[30]  Brendan J. Frey,et al.  Deep learning of the tissue-regulated splicing code , 2014, Bioinform..

[31]  Qi He,et al.  Keep It Simple with Time: A Reexamination of Probabilistic Topic Detection Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Maya R. Gupta,et al.  Introduction to the Dirichlet Distribution and Related Processes , 2010 .

[33]  Marina Meila,et al.  Bayesian Non-Parametric Clustering of Ranking Data , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Hagai Attias,et al.  A Variational Bayesian Framework for Graphical Models , 1999 .

[35]  Jun Guo,et al.  DNN Filter Bank Cepstral Coefficients for Spoofing Detection , 2017, IEEE Access.

[36]  Tyson A. Clark,et al.  Direct detection of DNA methylation during single-molecule, real-time sequencing , 2010, Nature Methods.

[37]  Peter W. Laird,et al.  A comparison of cluster analysis methods using DNA methylation data , 2004, Bioinform..

[38]  Lawrence Carin,et al.  Hidden Markov Models With Stick-Breaking Priors , 2009, IEEE Transactions on Signal Processing.

[39]  Arne Leijon,et al.  Human skin color detection in RGB space with Bayesian estimation of beta mixture models , 2010, 2010 18th European Signal Processing Conference.

[40]  Douglas Eck,et al.  Learning Features from Music Audio with Deep Belief Networks , 2010, ISMIR.

[41]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .

[42]  Sotirios Chatzis,et al.  The infinite Hidden Markov random field model , 2009, ICCV.

[43]  Jun Guo,et al.  Cross-modal subspace learning for fine-grained sketch-based image retrieval , 2017, Neurocomputing.

[44]  M. Esteller,et al.  Validation of a DNA methylation microarray for 450,000 CpG sites in the human genome , 2011, Epigenetics.

[45]  Nizar Bouguila,et al.  Unsupervised learning of a finite mixture model based on the Dirichlet distribution and its application , 2004, IEEE Transactions on Image Processing.

[46]  Nizar Bouguila,et al.  A Dirichlet Process Mixture of Generalized Dirichlet Distributions for Proportional Data Modeling , 2010, IEEE Transactions on Neural Networks.

[47]  Zhengrong Liang,et al.  An EM Approach to MAP Solution of Segmenting Tissue Mixtures: A Numerical Analysis , 2009, IEEE Transactions on Medical Imaging.

[48]  Nizar Bouguila,et al.  Hybrid Generative/Discriminative Approaches for Proportional Data Modeling and Classification , 2012, IEEE Transactions on Knowledge and Data Engineering.

[49]  Jun Guo,et al.  Activation force-based air pollution tracing , 2016, 2016 IEEE International Conference on Network Infrastructure and Digital Content (IC-NIDC).

[50]  C. Antoniak.  Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems , 1974 .

[51]  小森和樹.  Gene Expression Omnibus利用方法の検討 , 2016 .

[52]  Jun Guo,et al.  Feature selection for neutral vector in EEG signal classification , 2016, Neurocomputing.

[53]  Honggang Zhang,et al.  Cycled merging registration of point clouds for 3D human body modeling , 2016, 2016 First International Workshop on Sensing, Processing and Learning for Intelligent Machines (SPLINE).

[54]  Jun Guo,et al.  The Role of Data Analysis in the Development of Intelligent Energy Networks , 2017, IEEE Network.

[55]  Jun S. Liu,et al.  The Collapsed Gibbs Sampler in Bayesian Computations with Applications to a Gene Regulation Problem , 1994 .

[56]  Jun Guo,et al.  Effect of multi-condition training and speech enhancement methods on spoofing detection , 2016, 2016 First International Workshop on Sensing, Processing and Learning for Intelligent Machines (SPLINE).

[57]  Markus Flierl,et al.  Bayesian estimation of Dirichlet mixture model with variational inference , 2014, Pattern Recognit..

[58]  Christian P. Robert,et al.  The Bayesian choice : from decision-theoretic foundations to computational implementation , 2007 .

[59]  Jalil Taghia,et al.  Comparisons of Non-Gaussian Statistical Models in DNA Methylation Analysis , 2014, International journal of molecular sciences.

[60]  Richard Walker,et al.  PD Disease State Assessment in Naturalistic Environments Using Deep Learning , 2015, AAAI.

[61]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[62]  Fredrik Wallin,et al.  A Comprehensive Review of Smart Energy Meters in Intelligent Energy Networks , 2016, IEEE Internet of Things Journal.

[63]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition , 2012 .

[64]  Honggang Zhang,et al.  Nonlinear estimation of missing ΔLSF parameters by a mixture of Dirichlet distributions , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[65]  Arne Leijon,et al.  Predictive Distribution of the Dirichlet Mixture Model by Local Variational Inference , 2014, J. Signal Process. Syst..

[66]  Zhanyu Ma,et al.  A probabilistic principal component analysis based hidden Markov model for audio-visual speech recognition , 2008, 2008 42nd Asilomar Conference on Signals, Systems and Computers.

[67]  Douglas A. Reynolds,et al.  Robust text-independent speaker identification using Gaussian mixture speaker models , 1995, IEEE Trans. Speech Audio Process..

[68]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[69]  Izhar Wallach,et al.  The protein-small-molecule database, a non-redundant structural resource for the analysis of protein-ligand binding , 2009, Bioinform..

[70]  Christopher Holmes,et al.  Bayesian Nonparametrics: Frontmatter , 2010 .

[71]  Arne Leijon,et al.  Modelling speech line spectral frequencies with dirichlet mixture models , 2010, INTERSPEECH.

[72]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[73]  Chong Wang,et al.  Nested Hierarchical Dirichlet Processes , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[74]  Jianhua Zhang,et al.  Data scheme-based wireless channel modeling method: motivation, principle and performance , 2017, Journal of Communications and Information Networks.

[75]  Nizar Bouguila,et al.  High-Dimensional Unsupervised Selection and Estimation of a Finite Generalized Dirichlet Mixture Model Based on Minimum Message Length , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[76]  Jalil Taghia,et al.  Variational Inference for Watson Mixture Model , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[77]  Zhongwei Si,et al.  Learning Deep Features for DNA Methylation Data Analysis , 2016, IEEE Access.

[78]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[79]  Nicholas J. Foti,et al.  A Survey of Non-Exchangeable Priors for Bayesian Nonparametric Models , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[80]  W. Gilks.  Markov Chain Monte Carlo , 2005 .

[81]  Zhen Yang,et al.  The infinite Student's t-factor mixture analyzer for robust clustering and classification , 2012, Pattern Recognit..

[82]  Nizar Bouguila,et al.  Visual Scenes Categorization Using a Flexible Hierarchical Mixture Model Supporting Users Ontology , 2013, 2013 IEEE 25th International Conference on Tools with Artificial Intelligence.

[83]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[84]  Nizar Bouguila,et al.  Topic Novelty Detection Using Infinite Variational Inverted Dirichlet Mixture Models , 2015, 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA).

[85]  Jun Liu,et al.  User intention understanding from scratch , 2016, 2016 First International Workshop on Sensing, Processing and Learning for Intelligent Machines (SPLINE).

[86]  Zhanyu Ma,et al.  A variational Bayes beta Mixture Model for Feature Selection in DNA methylation Studies , 2013, J. Bioinform. Comput. Biol..

[87]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[88]  Nizar Bouguila,et al.  Online Learning of a Dirichlet Process Mixture of Beta-Liouville Distributions Via Variational Inference , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[89]  Carsten Wiuf,et al.  A Beta-mixture model for dimensionality reduction, sample classification and analysis , 2011, BMC Bioinformatics.

[90]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[91]  Masakazu Matsugu,et al.  Subject independent facial expression recognition with robust face detection using a convolutional neural network , 2003, Neural Networks.

[92]  Zhu Han,et al.  Unsupervised Profiling of Microglial Arbor Morphologies and Distribution Using a Nonparametric Bayesian Approach , 2016, IEEE Journal of Selected Topics in Signal Processing.

[93]  Margaret R. Karagas,et al.  Model-based clustering of DNA methylation array data: a recursive-partitioning algorithm for high-dimensional data arising as a mixture of beta distributions , 2008, BMC Bioinformatics.

[94]  Jalil Taghia,et al.  Bayesian Estimation of the von-Mises Fisher Mixture Model with Variational Inference , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[95]  Stefanos Zafeiriou,et al.  Variational Infinite Hidden Conditional Random Fields , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[96]  Arturas Petronis,et al.  Epigenetics as a unifying principle in the aetiology of complex traits and diseases , 2010, Nature.

[97]  Jürgen Schmidhuber,et al.  Multi-column deep neural networks for image classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[98]  Markus Flierl,et al.  Multiview depth map enhancement by variational bayes inference estimation of Dirichlet mixture models , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[99]  Xiaoming Chen.  Using Akaike Information Criterion for Selecting the Field Distribution in a Reverberation Chamber , 2013, IEEE Transactions on Electromagnetic Compatibility.

[100]  David J. Miller,et al.  Joint Parsimonious Modeling and Model Order Selection for Multivariate Gaussian Mixtures , 2010, IEEE Journal of Selected Topics in Signal Processing.

[101]  Arne Leijon,et al.  Expectation propagation for estimating the parameters of the beta distribution , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[102]  Jun Guo,et al.  Histogram transform model using MFCC features for text-independent speaker identification , 2014, 2014 48th Asilomar Conference on Signals, Systems and Computers.

[103]  Nizar Bouguila,et al.  Positive vectors clustering using inverted Dirichlet finite mixture models , 2012, Expert Syst. Appl..

[104]  Arne Leijon,et al.  Bayesian Estimation of Beta Mixture Models with Variational Inference , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[105]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[106]  Chunguang Li,et al.  The infinite Student's t-mixture for robust modeling , 2012, Signal Process..

[107]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[108]  Arne Leijon,et al.  PDF-optimized LSF vector quantization based on beta mixture models , 2010, INTERSPEECH.

[109]  H. Kitchener,et al.  The Dynamics and Prognostic Potential of DNA Methylation Changes at Stem Cell Gene Loci in Women's Cancer , 2012, PLoS genetics.

[110]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .