Toward the use of blog articles as a source of evidence for software engineering research

Background: Blog articles have potential value as a source of practitioner-generated evidence to complement already accepted sources of evidence in software engineering research e.g. interviews and surveys. To be valuable to research, a method for extracting the high quality articles from the vast quantity available needs to be developed. Objective: To better define the benefits and challenges, scope the problem, develop a set of criteria for evaluating blog articles to be used in the method, and propose research questions. Method: We conducted a two-phase pilot study, using a preliminary set of criteria, to explore the challenges of classifying blog articles. We analyse credibility criteria that have been used in previous research, and cross reference those criteria with previous research in evidence-based software engineering. Results: Based on our analysis, we decide that blog articles need to be rigorous, relevant, well written and experience based for them to be considered credible to researchers. Conclusion: Our work provides an overview of the problem domain, as well as presenting criteria and suggested measurements for these criteria. These can be used by others to find blog articles of potential value to their research.

[1]  Lionel Briand,et al.  Embracing the Engineering Side of Software Engineering , 2012, IEEE Softw..

[2]  Hikmat Ullah Khan,et al.  Modelling to identify influential bloggers in the blogosphere: A survey , 2017, Comput. Hum. Behav..

[3]  Sung-Un Yang,et al.  Measuring Social Media Credibility: A Study on a Measure of Blog Credibility , 2011 .

[4]  David Lo,et al.  What does software engineering community microblog about? , 2012, 2012 9th IEEE Working Conference on Mining Software Repositories (MSR).

[5]  Claire Grover,et al.  Re-using an Argument Corpus to Aid in the Curation of Social Media Collections , 2014, LREC.

[6]  Leif Singer,et al.  The (R) Evolution of social media in software engineering , 2014, FOSE.

[7]  Wu He,et al.  Mining Blogs And Forums To Understand the Use of Social Media in Customer Co-creation , 2015, Comput. J..

[8]  Jr. Frederick P. Brooks,et al.  The mythical man-month (anniversary ed.) , 1995 .

[9]  Ahmed E. Hassan,et al.  What are developers talking about? An analysis of topics and trends in Stack Overflow , 2014, Empirical Software Engineering.

[10]  Claes Wohlin.  An Evidence Profile for Software Engineering Research and Practice , 2013, Perspectives on the Future of Software Engineering.

[11]  Mark Dredze,et al.  You Are What You Tweet: Analyzing Twitter for Public Health , 2011, ICWSM.

[12]  Austen Rainer,et al.  Using argumentation theory to analyse software practitioners' defeasible evidence, inference and belief , 2017, Inf. Softw. Technol..

[13]  Richard F. Carter,et al.  Newspapers or Television: Which Do You Believe? , 1965 .

[14]  Matthew Hurst,et al.  BlogPulse: Automated Trend Discovery for Weblogs , 2003 .

[15]  Austen Rainer,et al.  Persuading developers to "buy into" software process improvement: a local opinion and empirical evidence , 2003, 2003 International Symposium on Empirical Software Engineering, 2003. ISESE 2003. Proceedings..

[16]  Robert L. Glass,et al.  Science and substance: a challenge to software engineers , 1994, IEEE Software.

[17]  Yang Li,et al.  Sentiment analysis of commit comments in GitHub: an empirical study , 2014, MSR 2014.

[18]  Miriam J. Metzger,et al.  Credibility for the 21st Century: Integrating Perspectives on Source, Message, and Media Credibility in the Contemporary Media Environment , 2003 .

[19]  Walid Maalej,et al.  How do developers blog?: an exploratory study , 2011, MSR '11.

[20]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[21]  Thomas J. Johnson,et al.  Wag the Blog: How Reliance on Traditional Media and the Internet Influence Credibility Perceptions of Weblogs Among Blog Users , 2004 .

[22]  Jennifer Jie Xu,et al.  Business Intelligence in Blogs: Understanding Consumer Interactions and Communities , 2012, MIS Q..

[23]  Paolo Torroni,et al.  Argumentation Mining , 2016, ACM Trans. Internet Techn..

[24]  Iryna Gurevych,et al.  Annotating Argument Components and Relations in Persuasive Essays , 2014, COLING.

[25]  Frederick P. Brooks,et al.  The Mythical Man-Month: Essays on Softw , 1978 .