Automatic identification of personal insults on social news sites

As online communities grow and the volume of user-generated content increases, the need for community management also rises. Community management has three main purposes: to create a positive experience for existing participants, to promote appropriate, socionormative behaviors, and to encourage potential participants to make contributions. Research indicates that the quality of content a potential participant sees on a site is highly influential; off-topic, negative comments with malicious intent are a particularly strong boundary to participation or set the tone for encouraging similar contributions. A problem for community managers, therefore, is the detection and elimination of such undesirable content. As a community grows, this undertaking becomes more daunting. Can an automated system aid community managers in this task? In this paper, we address this question through a machine learning approach to automatic detection of inappropriate negative user contributions. Our training corpus is a set of comments from a news commenting site that we tasked Amazon Mechanical Turk workers with labeling. Each comment is labeled for the presence of profanity, insults, and the object of the insults. Support vector machines trained on these data are combined with relevance and valence analysis systems in a multistep approach to the detection of inappropriate negative user contributions. The system shows great potential for semiautomated community management. © 2012 Wiley Periodicals, Inc.

[1]  Katharina Morik,et al.  Combining Statistical Learning with a Knowledge-Based Approach - A Case Study in Intensive Care Monitoring , 1999, ICML.

[2]  Bo Pang,et al.  Thumbs up? Sentiment Classification using Machine Learning Techniques , 2002, EMNLP.

[3]  Lawrence Birnbaum,et al.  Reasoning Through Search: A Novel Approach to Sentiment Classification , 2007 .

[4]  Panagiotis G. Ipeirotis,et al.  Get another label? improving data quality and data mining using multiple, noisy labelers , 2008, KDD.

[5]  Duncan J. Watts,et al.  Financial incentives and the "performance of crowds" , 2009, HCOMP '09.

[6]  Sara Owsley Sood,et al.  ESSE: Exploring mood on the web , 2009 .

[7]  Thorsten Joachims,et al.  Text Categorization with Support Vector Machines: Learning with Many Relevant Features , 1998, ECML.

[8]  Gerard Salton,et al.  Term-Weighting Approaches in Automatic Text Retrieval , 1988, Inf. Process. Manag..

[9]  Cliff Lampe,et al.  Follow the (slash) dot: effects of feedback on new members in an online community , 2005, GROUP.

[10]  Dell Zhang,et al.  Question classification using support vector machines , 2003, SIGIR.

[11]  Lawrence Birnbaum,et al.  Information access in context , 2001, Knowl. Based Syst..

[12]  Eugénio C. Oliveira,et al.  Tokenizing micro-blogging messages using a text classification approach , 2010, AND '10.

[13]  Michalis Faloutsos,et al.  On power-law relationships of the Internet topology , 1999, SIGCOMM '99.

[14]  Kristian J. Hammond,et al.  Domain Specific Affective Classification of Documents , 2006, AAAI Spring Symposium: Computational Approaches to Analyzing Weblogs.

[15]  Brian D. Davison,et al.  Detection of Harassment on Web 2.0 , 2009 .

[16]  Michael Gamon,et al.  Automatic Identification of Sentiment Vocabulary: Exploiting Low Association with Known Sentiment Terms , 2005, ACL 2005.

[17]  Henry Lieberman,et al.  Modeling the Detection of Textual Cyberbullying , 2011, The Social Mobile Web.

[18]  Paul Resnick,et al.  Slash(dot) and burn: distributed moderation in a large online conversation space , 2004, CHI.

[19]  Clifford Nass,et al.  Normative influences on thoughtful online participation , 2011, CHI.

[20]  M. Bradley,et al.  Affective Norms for English Words (ANEW): Instruction Manual and Affective Ratings , 1999 .

[21]  Martin Chodorow,et al.  Rethinking Grammatical Error Annotation and Evaluation with the Amazon Mechanical Turk , 2010 .

[22]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[23]  Rich Gazan.  When Online Communities Become Self-Aware , 2009 .

[24]  Sara Owsley Sood.  Anger Management : Using Sentiment Analysis to Manage Online Communities , 2010 .

[25]  Chris Callison-Burch,et al.  Fast, Cheap, and Creative: Evaluating Translation Quality Using Amazon’s Mechanical Turk , 2009, EMNLP.

[26]  Wolfgang Nejdl,et al.  How useful are your comments?: analyzing and predicting youtube comments and comment ratings , 2010, WWW '10.

[27]  Thomas Chesney,et al.  Griefing in virtual worlds: causes, casualties and coping strategies , 2009, Inf. Syst. J..

[28]  Peter D. Turney.  Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews , 2002, ACL.

[29]  Aniket Kittur,et al.  Crowdsourcing user studies with Mechanical Turk , 2008, CHI.

[30]  Vladimir I. Levenshtein,et al.  Binary codes capable of correcting deletions, insertions, and reversals , 1965 .

[31]  Bo Pang,et al.  Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales , 2005, ACL.

[32]  P. Ekman.  Emotions Revealed: Recognizing Faces and Feelings to Improve Communication and Emotional Life , 2003 .

[33]  Chin-Laung Lei,et al.  A Collusion-Resistant Automation Scheme for Social Moderation Systems , 2009, 2009 6th IEEE Consumer Communications and Networking Conference.