Paired Restricted Boltzmann Machine for Linked Data

Restricted Boltzmann Machines (RBMs) are widely adopted unsupervised representation learning methods and have powered many data mining tasks such as collaborative filtering and document representation. Recently, linked data that contains both attribute and link information has become ubiquitous in various domains. For example, social media data is inherently linked via social relations and web data is networked via hyperlinks. It is evident from recent work that link information can enhance a number of real-world applications such as clustering and recommendations. Therefore, link information has the potential to advance RBMs for better representation learning. However, the majority of existing RBMs have been designed for independent and identically distributed data and are unequipped for linked data. In this paper, we aim to design a new type of Restricted Boltzmann Machines that takes advantage of linked data. In particular, we propose a paired Restricted Boltzmann Machine (pRBM), which is able to leverage the attribute and link information of linked data for representation learning. Experimental results on real-world datasets demonstrate the effectiveness of the proposed framework pRBM.

[1]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[2]  Tijmen Tieleman,et al.  Training restricted Boltzmann machines using approximations to the likelihood gradient , 2008, ICML '08.

[3]  Christian Igel,et al.  An Introduction to Restricted Boltzmann Machines , 2012, CIARP.

[4]  A. Vespignani,et al.  Modeling of Protein Interaction Networks , 2001, Complexus.

[5]  Huan Liu,et al.  Exploiting social relations for sentiment analysis in microblogging , 2013, WSDM.

[6]  Deli Zhao,et al.  Network Representation Learning with Rich Text Information , 2015, IJCAI.

[7]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[8]  Geoffrey E. Hinton,et al.  Deep Boltzmann Machines , 2009, AISTATS.

[9]  Ben Taskar,et al.  Discriminative Probabilistic Models for Relational Data , 2002, UAI.

[10]  Huan Liu,et al.  Unsupervised feature selection for linked social media data , 2012, KDD.

[11]  Jing Gao,et al.  LRBM: A Restricted Boltzmann Machine Based Approach for Representation Learning on Linked Data , 2014, 2014 IEEE International Conference on Data Mining.

[12]  Charu C. Aggarwal,et al.  Heterogeneous Network Embedding via Deep Architectures , 2015, KDD.

[13]  Hui Li,et al.  A Deep Learning Approach to Link Prediction in Dynamic Networks , 2014, SDM.

[14]  Huan Liu,et al.  Feature Selection with Linked Data in Social Media , 2012, SDM.

[15]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[16]  Charu C. Aggarwal,et al.  A Survey of Signed Network Mining in Social Media , 2015, ACM Comput. Surv..

[17]  Dejing Dou,et al.  Social restricted Boltzmann Machine: Human behavior prediction in health social networks , 2015, 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM).

[18]  Vlado Keselj,et al.  Representation Learning for Sparse, High Dimensional Multi-label Classification , 2012, RSCTC.

[19]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[20]  Ilya Sutskever,et al.  On the Convergence Properties of Contrastive Divergence , 2010, AISTATS.

[21]  Yuhao Wang,et al.  Predicting drug-target interactions using restricted Boltzmann machines , 2013, Bioinform..

[22]  Charu C. Aggarwal,et al.  Linked Document Embedding for Classification , 2016, CIKM.

[23]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[24]  David M. Blei,et al.  Supervised Topic Models , 2007, NIPS.

[25]  Geoffrey E. Hinton,et al.  Replicated Softmax: an Undirected Topic Model , 2009, NIPS.

[26]  Christopher Meek,et al.  Tied boltzmann machines for cold start recommendations , 2008, RecSys '08.

[27]  Geoffrey E. Hinton,et al.  Restricted Boltzmann machines for collaborative filtering , 2007, ICML '07.

[28]  Hector Garcia-Molina,et al.  Web graph similarity for anomaly detection , 2010, Journal of Internet Services and Applications.

[29]  David M. Blei,et al.  Relational Topic Models for Document Networks , 2009, AISTATS.

[30]  Baoxin Li,et al.  Unsupervised Sentiment Analysis for Social Media Images , 2015, IJCAI.

[31]  Geoffrey E. Hinton.  Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[32]  Pengtao Xie,et al.  Diversifying Restricted Boltzmann Machine for Document Modeling , 2015, KDD.