Infinite Plaid Models for Infinite Bi-Clustering

We propose a probabilistic model for non-exhaustive and overlapping (NEO) bi-clustering. Our goal is to extract a few sub-matrices from the given data matrix, where entries of a sub-matrix are characterized by a specific distribution or parameters. Existing NEO bi-clustering methods typically require the number of sub-matrices to be extracted, which is essentially difficult to fix a priori. In this paper, we extend the plaid model, known as one of the best NEO bi-clustering algorithms, to allow infinite bi-clustering; NEO bi-clustering without specifying the number of sub-matrices. Our model can represent infinite sub-matrices formally. We develop a MCMC inference without the finite truncation, which potentially addresses all possible numbers of sub-matrices. Experiments quantitatively and qualitatively verify the usefulness of the proposed model. The results reveal that our model can offer more precise and in-depth analysis of sub-matrices.

[1]  Inderjit S. Dhillon,et al.  Stochastic Blockmodel with Cluster Overlap, Relevance Selection, and Similarity-Based Smoothing , 2013, 2013 IEEE 13th International Conference on Data Mining.

[2]  L. Lazzeroni.  Plaid models for gene expression data , 2000 .

[3]  Thomas L. Griffiths,et al.  Learning Systems of Concepts with an Infinite Relational Model , 2006, AAAI.

[4]  Jun S Liu,et al.  Bayesian biclustering of gene expression data , 2008, BMC Genomics.

[5]  Richard M. Karp,et al.  Discovering local structure in gene expression data: the order-preserving submatrix problem. , 2003 .

[6]  Peter D. Hoff,et al.  Subset Clustering of Binary Sequences, with an Application to Genomic Abnormality Data , 2005, Biometrics.

[7]  Zoubin Ghahramani,et al.  An Infinite Latent Attribute Model for Network Data , 2012, ICML.

[8]  A. Nobel,et al.  Finding large average submatrices in high dimensional data , 2009, 0905.1682.

[9]  Jure Leskovec,et al.  Nonparametric Multi-group Membership Model for Dynamic Networks , 2013, NIPS.

[10]  J. Lafferty,et al.  Mixed-membership models of scientific publications , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[11]  Zoubin Ghahramani,et al.  Stochastic Inference for Scalable Probabilistic Modeling of Binary Matrices , 2014, ICML.

[12]  Mehmet Deveci,et al.  A comparative analysis of biclustering algorithms for gene expression data , 2013, Briefings Bioinform..

[13]  George M. Church,et al.  Biclustering of Expression Data , 2000, ISMB.

[14]  Naonori Ueda,et al.  Rectangular Tiling Process , 2014, ICML.

[15]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[16]  Yiming Yang,et al.  The Enron Corpus: A New Dataset for Email Classi(cid:12)cation Research , 2004 .

[17]  Naonori Ueda,et al.  Dynamic Infinite Relational Model for Time-varying Relational Data Analysis , 2010, NIPS.

[18]  Thomas L. Griffiths,et al.  The Indian Buffet Process: An Introduction and Review , 2011, J. Mach. Learn. Res..

[19]  Naonori Ueda,et al.  Subset Infinite Relational Models , 2012, AISTATS.

[20]  Sampsa Hautaniemi,et al.  Biclustering Methods: Biological Relevance and Application in Gene Expression Analysis , 2014, PloS one.

[21]  Zoubin Ghahramani,et al.  Modeling Dyadic Data with Binary Latent Factors , 2006, NIPS.

[22]  Qiang Fu,et al.  Bayesian Overlapping Subspace Clustering , 2009, 2009 Ninth IEEE International Conference on Data Mining.

[23]  Andrea Lancichinetti,et al.  Detecting the overlapping and hierarchical community structure in complex networks , 2008, 0802.1218.

[24]  Morten Mørup,et al.  Large scale inference in the Infinite Relational Model: Gibbs sampling is not enough , 2013, 2013 IEEE International Workshop on Machine Learning for Signal Processing (MLSP).

[25]  Hirokazu Kameoka,et al.  Complex NMF: A new sparse representation for acoustic signals , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[26]  Hirokazu Kameoka,et al.  Multichannel Extensions of Non-Negative Matrix Factorization With Complex-Valued Data , 2013, IEEE Transactions on Audio, Speech, and Language Processing.

[27]  Jennifer G. Dy,et al.  Dual beta process priors for latent cluster discovery in chronic obstructive pulmonary disease , 2014, KDD.

[28]  D. Blei,et al.  Truncation-free stochastic variational inference for Bayesian nonparametric models , 2012, NIPS 2012.

[29]  David B. Dunson,et al.  Beta-Negative Binomial Process and Poisson Factor Analysis , 2011, AISTATS.

[30]  Thomas L. Griffiths,et al.  Nonparametric Latent Feature Models for Link Prediction , 2009, NIPS.

[31]  S. Kaski,et al.  Bayesian biclustering with the plaid model , 2008, 2008 IEEE Workshop on Machine Learning for Signal Processing.