Issues and Best Practices in Content Analysis

This article discusses three issues concerning content analysis method and ends with a list of best practices in conducting and reporting content analysis projects. Issues addressed include the use of search and databases for sampling, the differences between content analysis and algorithmic text analysis, and which reliability coefficients should be calculated and reported. The “Best Practices” section provides steps to produce reliable and valid content analysis data and the appropriate reporting of those steps so the project can be properly evaluated and replicated.

[1]  Klaus Krippendorff,et al.  Answering the Call for a Standard Reliability Measure for Coding Data , 2007 .

[2]  Bruce Bimber,et al.  Finding News Stories: A Comparison of Searches Using Lexisnexis and Google News , 2008 .

[3]  Seth C. Lewis,et al.  Content Analysis and the Algorithmic Coder , 2015 .

[4]  Linda Shields,et al.  Content Analysis , 2015 .

[5]  Jiebo Luo,et al.  Towards Understanding the Effectiveness of Election Related Images in Social Media , 2013, 2013 IEEE 13th International Conference on Data Mining Workshops.

[6]  Roel Popping,et al.  Computer-assisted text analysis , 2000 .

[7]  Colin Seymour-Ure,et al.  Content Analysis in Communication Research. , 1972 .

[8]  Klaus Krippendorff,et al.  Agreement and Information in the Reliability of Coding , 2011 .

[9]  Dana Mastro,et al.  The Portrayal of Racial Minorities on Prime Time Television , 2000 .

[10]  Navneet Kaur,et al.  Opinion mining and sentiment analysis , 2016, 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom).

[11]  Tony Hak,et al.  Coder training: Theoretical training or practical socialization? , 1996 .

[12]  Kimberly A. Neuendorf,et al.  The Content Analysis Guidebook , 2001 .

[13]  O. Holsti.  Content Analysis for the Social Sciences and Humanities , 1969 .

[14]  Cory L. Armstrong,et al.  Views from the Margins: News Coverage of Women in Abortion Protests, 1960–2006 , 2011 .

[15]  W. James Potter,et al.  Rethinking validity and reliability in content analysis , 1999 .

[16]  Justin Grimmer,et al.  Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts , 2013, Political Analysis.

[17]  G. Feng.  Mistakes and How to Avoid Mistakes in Using Intercoder Reliability Indices , 2015 .

[18]  Kathleen A. Hansen.  Using Databases for Content Analysis , 2003 .

[19]  Stephen Lacy,et al.  Sampling Error and Selecting Intercoder Reliability Samples for Nominal Content Categories , 1996 .

[20]  Jie Qin.  Hero on Twitter, Traitor on News , 2015 .

[21]  K. Aihara,et al.  On the use of chance-adjusted agreement statistic to measure the assortative transmission of infectious diseases , 2013 .

[22]  Claudia C. Cogliser,et al.  Construct Validation Using Computer-Aided Text Analysis (CATA) , 2010 .

[23]  Klaus Krippendorff,et al.  Content Analysis: An Introduction to Its Methodology , 1980 .

[24]  Mike Thelwall,et al.  Hyperlink Analyses of the World Wide Web: A Review , 2006, J. Comput. Mediat. Commun..

[25]  K. Gwet.  Computing inter-rater reliability and its variance in the presence of high agreement. , 2008, The British journal of mathematical and statistical psychology.

[26]  Catherine A. Luther,et al.  Framing of the 2003 U.S.-Iraq War Demonstrations: An Analysis of News and Partisan Texts , 2005 .

[27]  K. Krippendorff.  Commentary: A Dissenting View on So-Called Paradoxes of Reliability Coefficients , 2013 .

[28]  Daniel Riffe,et al.  Analyzing media messages: Using quantitative content analysis in research, Third edition , 2014 .

[29]  Hong Kong Baptist.  Assumptions behind Intercoder Reliability Indices , 2017 .

[30]  Alfred Hermida,et al.  Content Analysis in an Era of Big Data: A Hybrid Approach to Computational and Manual Methods , 2013 .

[31]  Stephen Lacy,et al.  Assessing the Reporting of Reliability in Published Content Analyses: 1985–2010 , 2014 .

[32]  D. Karpf.  SOCIAL SCIENCE RESEARCH METHODS IN INTERNET TIME , 2012 .

[33]  Giuseppe Porro,et al.  Every tweet counts? How sentiment analysis of social media can improve our knowledge of citizens’ political preferences with an application to Italy and France , 2013, New Media Soc..

[34]  W. A. Scott,et al.  Reliability of Content Analysis ; The Case of Nominal Scale Cording , 1955 .

[35]  Aging on Television: The Picture in the Nineties , 2001 .

[36]  Mike Conway,et al.  The Subjective Precision of Computers: A Methodological Comparison with Human Coding in Content Analysis , 2006 .

[37]  Brendan R. Watson,et al.  Analyzing Media Messages , 2023 .

[38]  K. Gwet.  Handbook of Inter-Rater Reliability: The Definitive Guide to Measuring the Extent of Agreement Among Raters , 2014 .

[39]  Ricardo J. Wray,et al.  Validation of Database Search Terms for Content Analysis: The Case of Cancer News Coverage , 2006 .

[40]  K. Krippendorff.  Reliability in Content Analysis: Some Common Misconceptions and Recommendations , 2004 .