Inferring User Preferences by Probabilistic Logical Reasoning over Social Networks

We propose a framework for inferring the latent attitudes or preferences of users by performing probabilistic first-order logical reasoning over the social network graph. Our method answers questions about Twitter users like {\em Does this user like sushi?} or {\em Is this user a New York Knicks fan?} by building a probabilistic model that reasons over user attributes (the user's location or gender) and the social network (the user's friends and spouse), via inferences like homophily (I am more likely to like sushi if spouse or friends like sushi, I am more likely to like the Knicks if I live in New York). The algorithm uses distant supervision, semi-supervised data harvesting and vector space models to extract user attributes (e.g. spouse, education, location) and preferences (likes and dislikes) from text. The extracted propositions are then fed into a probabilistic reasoner (we investigate both Markov Logic and Probabilistic Soft Logic). Our experiments show that probabilistic logical reasoning significantly improves the performance on attribute and relation extraction, and also achieves an F-score of 0.791 at predicting a users likes or dislikes, significantly better than two strong baselines.

[1]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[2]  M. McPherson,et al.  Birds of a Feather: Homophily in Social Networks , 2001 .

[3]  Christopher Ré,et al.  Tuffy: Scaling up Statistical Inference in Markov Logic Networks using an RDBMS , 2011, Proc. VLDB Endow..

[4]  Kyumin Lee,et al.  You are where you tweet: a content-based approach to geo-locating twitter users , 2010, CIKM.

[5]  Zornitsa Kozareva,et al.  Learning Arguments and Supertypes of Semantic Relations Using Recursive Patterns , 2010, ACL.

[6]  Stephen H. Bach,et al.  Social Group Modeling with Probabilistic Soft Logic , 2012 .

[7]  Krishna P. Gummadi,et al.  You are who you know: inferring user profiles in online social networks , 2010, WSDM '10.

[8]  John Riedl,et al.  Item-based collaborative filtering recommendation algorithms , 2001, WWW '01.

[9]  Ee-Peng Lim,et al.  Finding Bursty Topics from Microblogs , 2012, ACL.

[10]  Patrick Paroubek,et al.  Twitter as a Corpus for Sentiment Analysis and Opinion Mining , 2010, LREC.

[11]  Eduard H. Hovy,et al.  Weakly Supervised User Profile Extraction from Twitter , 2014, ACL.

[12]  John D. Burger,et al.  Discriminating Gender on Twitter , 2011, EMNLP.

[13]  Chad Cumby Dan Roth,et al.  Feature Extraction Languages for Propositionalized Relational Learning , 2003 .

[14]  D. Rao Detecting Latent User Properties in Social Media , 2010 .

[15]  Johanna D. Moore,et al.  Twitter Sentiment Analysis: The Good the Bad and the OMG! , 2011, ICWSM.

[16]  Harith Alani,et al.  Semantic Sentiment Analysis of Twitter , 2012, SEMWEB.

[17]  Eunsol Choi,et al.  Scaling Semantic Parsers with On-the-Fly Ontology Matching , 2013, EMNLP.

[18]  Lise Getoor,et al.  Computing Marginal Distributions over Continuous Markov Networks for Statistical Relational Learning , 2010, NIPS.

[19]  Yoshua Bengio,et al.  Neural Probabilistic Language Models , 2006 .

[20]  Pedro M. Domingos,et al.  Entity Resolution with Markov Logic , 2006, Sixth International Conference on Data Mining (ICDM'06).

[21]  Koustuv Dasgupta,et al.  User interests in social media sites: an exploration with micro-blogs , 2009, CIKM.

[22]  Owen Rambow,et al.  Sentiment Analysis of Twitter Data , 2011 .

[23]  Oren Etzioni,et al.  Open domain event extraction from twitter , 2012, KDD.

[24]  Ioannis Konstas,et al.  On social networks and collaborative recommendation , 2009, SIGIR.

[25]  Claire Cardie,et al.  Joint Inference for Fine-grained Opinion Extraction , 2013, ACL.

[26]  Lise Getoor,et al.  Learning Probabilistic Relational Models , 1999, IJCAI.

[27]  Mark Steedman,et al.  Combined Distributional and Logical Semantics , 2013, TACL.

[28]  S. Lauritzen The EM algorithm for graphical association models with missing data , 1995 .

[29]  Jure Leskovec,et al.  Overlapping community detection at scale: a nonnegative matrix factorization approach , 2013, WSDM.

[30]  Jacob Ratkiewicz,et al.  Political Polarization on Twitter , 2011, ICWSM.

[31]  Lise Getoor,et al.  Probabilistic Similarity Logic , 2010, UAI.

[32]  Oren Etzioni,et al.  Learning First-Order Horn Clauses from Web Text , 2010, EMNLP.

[33]  S. Muggleton Stochastic Logic Programs , 1996 .

[34]  Claire Cardie,et al.  Annotating Expressions of Opinions and Emotions in Language , 2005, Lang. Resour. Evaluation.

[35]  Bo Zhao,et al.  PET: a statistical model for popular events tracking in social communities , 2010, KDD.

[36]  William Yang Wang,et al.  Programming with personalized pagerank: a locally groundable first-order probabilistic logic , 2013, CIKM.

[37]  Lise Getoor,et al.  A short introduction to probabilistic soft logic , 2012, NIPS 2012.

[38]  Luc De Raedt,et al.  Inductive Logic Programming: Theory and Methods , 1994, J. Log. Program..

[39]  William Yang Wang,et al.  Structure Learning via Parameter Learning , 2014, CIKM.

[40]  Lise Getoor,et al.  Probabilistic soft logic for trust analysis in social networks , 2012 .

[41]  Gisele L. Pappa,et al.  Inferring the Location of Twitter Messages Based on User Relationships , 2011, Trans. GIS.

[42]  Louiqa Raschid,et al.  Ieee/acm Transactions on Computational Biology and Bioinformatics 1 Network-based Drug-target Interaction Prediction with Probabilistic Soft Logic , 2022 .

[43]  Bart Selman,et al.  Referral Web: combining social networks and collaborative filtering , 1997, CACM.

[44]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[45]  Pedro M. Domingos Multi-Relational Record Linkage , 2003 .

[46]  Martin Ester,et al.  A matrix factorization technique with trust propagation for recommendation in social networks , 2010, RecSys '10.

[47]  Ben Taskar,et al.  Discriminative Probabilistic Models for Relational Data , 2002, UAI.

[48]  James Cussens,et al.  CLP(BN): Constraint Logic Programming for Probabilistic Knowledge , 2002, Probabilistic Inductive Logic Programming.

[49]  Brendan T. O'Connor,et al.  Improved Part-of-Speech Tagging for Online Conversational Text with Word Clusters , 2013, NAACL.

[50]  Oren Etzioni,et al.  Modeling Missing Data in Distant Supervision for Information Extraction , 2013, TACL.

[51]  David Yarowsky,et al.  Classifying latent user attributes in twitter , 2010, SMUC '10.

[52]  Ana-Maria Popescu,et al.  A Machine Learning Approach to Twitter User Classification , 2011, ICWSM.

[53]  Hong Joo Lee,et al.  Use of social network information to enhance collaborative filtering performance , 2010, Expert Syst. Appl..

[54]  Claire Cardie,et al.  Major Life Event Extraction from Twitter based on Congratulations/Condolences Speech Acts , 2014, EMNLP.

[55]  László Dezsö,et al.  Universal Grammar , 1981, Certainty in Action.

[56]  Andrew McCallum,et al.  Relation Extraction with Matrix Factorization and Universal Schemas , 2013, NAACL.

[57]  Eduard Hovy,et al.  Extracting Opinions, Opinion Holders, and Topics Expressed in Online News Media Text , 2006 .

[58]  Lukás Burget,et al.  Extensions of recurrent neural network language model , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[59]  Jennifer Neville,et al.  Relational Dependency Networks , 2007, J. Mach. Learn. Res..

[60]  Daniel Jurafsky,et al.  Distant supervision for relation extraction without labeled data , 2009, ACL.

[61]  Claire Cardie,et al.  Identifying Sources of Opinions with Conditional Random Fields and Extraction Patterns , 2005, HLT.

[62]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[63]  C. Lee Giles,et al.  Efficient identification of Web communities , 2000, KDD '00.

[64]  Mark Craven,et al.  Constructing Biological Knowledge Bases by Extracting Information from Text Sources , 1999, ISMB.

[65]  Andrew Chou,et al.  Semantic Parsing on Freebase from Question-Answer Pairs , 2013, EMNLP.

[66]  William Yang Wang,et al.  ProPPR: Efficient First-Order Probabilistic Logic Programming for Structure Discovery, Parameter Learning, and Scalable Inference , 2014, StarAI@AAAI.

[67]  Ben Goertzel,et al.  Probabilistic Logic Networks , 2009 .

[68]  Pedro M. Domingos,et al.  Discriminative Training of Markov Logic Networks , 2005, AAAI.

[69]  Ana-Maria Popescu,et al.  Extracting events and event descriptions from Twitter , 2011, WWW.

[70]  Keith W. Ross,et al.  What's in a Name: A Study of Names, Gender Inference, and Gender Behavior in Facebook , 2011, DASFAA Workshops.

[71]  J. A. Robinson,et al.  A Machine-Oriented Logic Based on the Resolution Principle , 1965, JACM.

[72]  Manfred Jaeger,et al.  Probabilistic Reasoning in Terminological Logics , 1994, KR.

[73]  Ellen Riloff,et al.  Learning Dictionaries for Information Extraction by Multi-Level Bootstrapping , 1999, AAAI/IAAI.

[74]  Nicole A. Lazar,et al.  Statistical Analysis With Missing Data , 2003, Technometrics.

[75]  Katrin Erk,et al.  Probabilistic Soft Logic for Semantic Textual Similarity , 2014, ACL.

[76]  Luke S. Zettlemoyer,et al.  Knowledge-Based Weak Supervision for Information Extraction of Overlapping Relations , 2011, ACL.

[77]  Pabitra Mitra,et al.  Feature weighting in content based recommendation system using social network analysis , 2008, WWW.

[78]  Ari Rappoport,et al.  Fully Unsupervised Discovery of Concept-Specific Relationships by Web Mining , 2007, ACL.

[79]  Ni Lao,et al.  Efficient inference and learning in a large knowledge base , 2015, Machine Learning.

[80]  Jimmy J. Lin,et al.  Smoothing techniques for adaptive online language models: topic tracking in tweet streams , 2011, KDD.

[81]  Heeyoung Lee,et al.  A Multi-Pass Sieve for Coreference Resolution , 2010, EMNLP.

[82]  Derek Ruths,et al.  Gender Inference of Twitter Users in Non-English Contexts , 2013, EMNLP.

[83]  Zornitsa Kozareva,et al.  Not All Seeds Are Equal: Measuring the Quality of Text Mining Seeds , 2010, NAACL.

[84]  Claire Cardie,et al.  Timeline generation: tracking individuals on twitter , 2013, WWW.

[85]  Ellen Riloff,et al.  Corpus-based Semantic Lexicon Induction with Web-based Corroboration , 2009 .

[86]  Wendy Liu,et al.  Homophily and Latent Attribute Inference: Inferring Latent Attributes of Twitter Users from Neighbors , 2012, ICWSM.

[87]  Henry A. Kautz,et al.  Finding your friends and following them to where you are , 2012, WSDM '12.

[88]  Lukás Burget,et al.  Recurrent neural network based language model , 2010, INTERSPEECH.

[89]  Pedro M. Domingos,et al.  Efficient Weight Learning for Markov Logic Networks , 2007, PKDD.

[90]  David Carmel,et al.  Social media recommendation based on people and tags , 2010, SIGIR.