A Software Requirements Ecosystem: Linking Forum, Issue Tracker, and FAQs for Requirements Management

User feedback is an important resource in modern software development, often containing requirements that help address user concerns and desires for a software product. The feedback in online channels is a recent focus for software engineering researchers, with multiple studies proposing automatic analysis tools. In this work, we investigate the product forums of two large open source software projects. Through a quantitative analysis, we show that forum feedback is often manually linked to related issue tracker entries and product documentation. By linking feedback to their existing documentation, development teams enhance their understanding of known issues, and direct their users to known solutions. We discuss how the links between forum, issue tracker, and product documentation form a requirements ecosystem that has not been identified in the previous literature. We apply state-of-the-art deep-learning to automatically match forum posts with related issue tracker entries. Our approach identifies requirement matches with a mean average precision of 58.9% and hit ratio of 82.2%. Additionally, we apply deep-learning using an innovative clustering technique, achieving promising performance when matching forum posts to related product documentation. We discuss the possible applications of these automated techniques to support the flow of requirements between forum, issue tracker, and product documentation.

[1]  Kelly Blincoe,et al.  Evaluating Unsupervised Text Embeddings on Software User Feedback , 2021, 2021 IEEE 29th International Requirements Engineering Conference Workshops (REW).

[2]  Eduard C. Groen,et al.  Classifying User Requirements from Online Feedback in Small Dataset Environments using Deep Learning , 2021, 2021 IEEE 29th International Requirements Engineering Conference (RE).

[3]  Emitza Guzman,et al.  On the Role of User Feedback in Software Evolution: a Practitioners’ Perspective , 2021, 2021 IEEE 29th International Requirements Engineering Conference (RE).

[4]  Kelly Blincoe,et al.  Voice of the Users: A study of software feedback differences between Germany and China , 2021, 2021 IEEE 29th International Requirements Engineering Conference Workshops (REW).

[5]  Walid Maalej,et al.  Unsupervised Topic Discovery in User Comments , 2021, 2021 IEEE 29th International Requirements Engineering Conference (RE).

[6]  Kelly Blincoe,et al.  Voice of the users: an extended study of software feedback engagement , 2021, Requirements Engineering.

[7]  Walid Maalej,et al.  Automatically Matching Bug Reports With Related App Reviews , 2021, 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE).

[8]  Qingkai Zeng,et al.  Traceability Transformed: Generating More Accurate Links with Pre-Trained BERT Models , 2021, 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE).

[9]  Emitza Guzman,et al.  Same Same but Different: Finding Similar User Feedback Across Multiple Platforms and Languages , 2020, 2020 IEEE 28th International Requirements Engineering Conference (RE).

[10]  Kelly Blincoe,et al.  Voice of the Users: A Demographic Study of Software Feedback Behaviour , 2020, 2020 IEEE 28th International Requirements Engineering Conference (RE).

[11]  Kelly Blincoe,et al.  Can a Conversation Paint a Picture? Mining Requirements In Software Forums , 2019, 2019 IEEE 27th International Requirements Engineering Conference (RE).

[12]  Lin Liu,et al.  Analysis of Requirements-Related Arguments in User Forums , 2019, 2019 IEEE 27th International Requirements Engineering Conference (RE).

[13]  Iryna Gurevych,et al.  Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks , 2019, EMNLP.

[14]  Walid Maalej,et al.  Extracting and Analyzing Context Information in User-Support Conversations on Twitter , 2019, 2019 IEEE 27th International Requirements Engineering Conference (RE).

[15]  Nan Hua,et al.  Universal Sentence Encoder , 2018, ArXiv.

[16]  Mohamed Ibrahim,et al.  A Little Bird Told Me: Mining Tweets for Requirements and Software Evolution , 2017, 2017 IEEE 25th International Requirements Engineering Conference (RE).

[17]  Emitza Guzman,et al.  An exploratory study of Twitter messages about software applications , 2017, Requirements Engineering.

[18]  Gerardo Canfora,et al.  SURF: Summarizer of User Reviews Feedback , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering Companion (ICSE-C).

[19]  Anna Perini,et al.  Crowdsourcing for Software Engineering The Crowd in Requirements Engineering The Landscape and Challenges , 2017 .

[20]  Harald C. Gall,et al.  ARdoc: app reviews development oriented classifier , 2016, SIGSOFT FSE.

[21]  Norbert Seyff,et al.  A Needle in a Haystack: What Do Twitter Users Say about Software? , 2016, 2016 IEEE 24th International Requirements Engineering Conference (RE).

[22]  Walid Maalej,et al.  Bug report, feature request, or simply praise? On automatically classifying app reviews , 2015, 2015 IEEE 23rd International Requirements Engineering Conference (RE).

[23]  Chris Callison-Burch,et al.  PPDB 2.0: Better paraphrase ranking, fine-grained entailment relations, word embeddings, and style classification , 2015, ACL.

[24]  Jeroen van den Hoven,et al.  Ethics for the Digital Age: Where Are the Moral Specs? , 2015, ECSS.

[25]  Steven L. Alter,et al.  Digital services for consumers , 2014, Electron. Mark..

[26]  Steven Bethard,et al.  Back to Basics for Monolingual Alignment: Exploiting Word Similarity and Contextual Evidence , 2014, TACL.

[27]  Ning Chen,et al.  AR-miner: mining informative reviews for developers from mobile app marketplace , 2014, ICSE.

[28]  Walid Maalej,et al.  User feedback in the appstore: An empirical study , 2013, 2013 21st IEEE International Requirements Engineering Conference (RE).

[29]  Bernd Brügge,et al.  User involvement in software evolution practice: A case study , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[30]  Yuanyuan Zhang,et al.  App store mining and analysis: MSR for app stores , 2012, 2012 9th IEEE Working Conference on Mining Software Repositories (MSR).

[31]  Thomas Zimmermann,et al.  What Makes a Good Bug Report? , 2008, IEEE Transactions on Software Engineering.

[32]  K. Petersen,et al.  Context in industrial software engineering research , 2009, 2009 3rd International Symposium on Empirical Software Engineering and Measurement.

[33]  Joydeep Ghosh,et al.  Similarity-Based Text Clustering: A Comparative Study , 2006, Grouping Multidimensional Data.

[34]  V. Braun,et al.  Using thematic analysis in psychology , 2006 .