Abstract: We propose a frequency-based infinite relational model (FIRM), which takesinto account the frequency of relation whereas stochastic blockmodels ignore frequency.We also derive a variational inference method for the FIRM to apply to a large dataset.Experimental results show that the FIRM gives better clustering results than a stochasticblockmodel on a dataset which has the frequency of relation.key words : stochastic blockmodels, Dirichlet process, variational inference 1 Introduction Recently in machine learning, relationallearning hasreceived a great deal of attention, for example, to findsocial roles in social network. While the stochasticblockmodel has been popular for such relational learn-ing in sociology, it is now widely used for various appli-cations including clustering proteins, discovering con-cepts, etc [9, 13].Infinite relational models (IRMs) [9, 13] are stochas-tic blockmodels exploiting the Dirichlet process [6, 2]sothatwedonotneedtodeterminethenumberofclus-ters a priori. Stochastic blockmodels of mixed mem-bership (SBMM) are also stochastic blockmodels thatmodel multiple observation of tables [1].In this paper, we propose a frequency-based infi-nite relational model (FIRM). The FIRM takes intoaccount the frequency of relation, which is statisti-cally informative but ignored by stochastic blockmod-els. Our model generalizes the IRM and can also ob-servemultipletablesastheSBMM.ToapplytheFIRMto a large dataset, we also derived a variational in-ference algorithm for the FIRM. Experimental resultsshow the FIRM gives better clustering results than theIRM on a dataset which has the frequency of relation.
[1]
Yuji Matsumoto,et al.
Probabilistic Representation of Meanings
,
2002
.
[2]
Kathryn B. Laskey,et al.
Stochastic blockmodels: First steps
,
1983
.
[3]
J. Sethuraman.
A CONSTRUCTIVE DEFINITION OF DIRICHLET PRIORS
,
1991
.
[4]
C. Antoniak.
Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems
,
1974
.
[5]
G. Karypis,et al.
Criterion Functions for Document Clustering ∗ Experiments and Analysis
,
2001
.
[6]
Thomas L. Griffiths,et al.
Learning Systems of Concepts with an Infinite Relational Model
,
2006,
AAAI.
[7]
Michael I. Jordan,et al.
Variational inference for Dirichlet process mixtures
,
2006
.
[8]
Hans-Peter Kriegel,et al.
Infinite Hidden Relational Models
,
2006,
UAI.
[9]
E. Erosheva.
Bayesian Mixed Membership Models for Soft Classification 1
,
2004
.
[10]
T. Ferguson.
A Bayesian Analysis of Some Nonparametric Problems
,
1973
.
[11]
Zoubin Ghahramani,et al.
Variational Inference for Bayesian Mixtures of Factor Analysers
,
1999,
NIPS.
[12]
Edoardo M. Airoldi,et al.
Stochastic Block Models of Mixed Membership
,
2006
.
[13]
Hagai Attias,et al.
A Variational Bayesian Framework for Graphical Models
,
1999
.