What would users change in my app? summarizing app reviews for recommending software changes

Mobile app developers constantly monitor feedback in user reviews with the goal of improving their mobile apps and better meeting user expectations. Thus, automated approaches have been proposed in literature with the aim of reducing the effort required for analyzing feedback contained in user reviews via automatic classification/prioritization according to specific topics. In this paper, we introduce SURF (Summarizer of User Reviews Feedback), a novel approach to condense the enormous amount of information that developers of popular apps have to manage due to user feedback received on a daily basis. SURF relies on a conceptual model for capturing user needs useful for developers performing maintenance and evolution tasks. Then it uses sophisticated summarisation techniques for summarizing thousands of reviews and generating an interactive, structured and condensed agenda of recommended software changes. We performed an end-to-end evaluation of SURF on user reviews of 17 mobile apps (5 of them developed by Sony Mobile), involving 23 developers and researchers in total. Results demonstrate high accuracy of SURF in summarizing reviews and the usefulness of the recommended changes. In evaluating our approach we found that SURF helps developers in better understanding user needs, substantially reducing the time required by developers compared to manually analyzing user (change) requests and planning future software changes.

[1]  Alan Borning,et al.  Lightweight structural summarization as an aid to software evolution , 1996 .

[2]  Bo Pang,et al.  Thumbs up? Sentiment Classification using Machine Learning Techniques , 2002, EMNLP.

[3]  Harald C. Gall,et al.  How can i improve my app? Classifying user reviews for software maintenance and evolution , 2015, 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[4]  Gail C. Murphy,et al.  Automatic Summarization of Bug Reports , 2014, IEEE Transactions on Software Engineering.

[5]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[6]  Amjed Tahir,et al.  On Satisfying the Android OS Community: User Feedback Still Central to Developers' Portfolios , 2015, 2015 24th Australasian Software Engineering Conference.

[7]  Gabriele Bavota,et al.  User reviews matter! Tracking crowdsourced reviews to support evolution of successful apps , 2015, 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[8]  Collin McMillan,et al.  Automatic documentation generation via source code summarization of method context , 2014, ICPC 2014.

[9]  Walid Maalej,et al.  Bug report, feature request, or simply praise? On automatically classifying app reviews , 2015, 2015 IEEE 23rd International Requirements Engineering Conference (RE).

[10]  Norbert Seyff,et al.  AppEcho: a user-driven, in situ feedback approach for mobile platforms and applications , 2014, MOBILESoft 2014.

[11]  Yuanyuan Zhang,et al.  Feature lifecycles as they spread, migrate, remain, and die in App Stores , 2015, 2015 IEEE 23rd International Requirements Engineering Conference (RE).

[12]  Martha Palmer,et al.  Verb Semantics and Lexical Selection , 1994, ACL.

[13]  Yuanyuan Zhang,et al.  App store mining and analysis: MSR for app stores , 2012, 2012 9th IEEE Working Conference on Mining Software Repositories (MSR).

[14]  Tung Thanh Nguyen,et al.  Mining User Opinions in Mobile App Reviews: A Keyword-Based Approach (T) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[15]  Gail C. Murphy,et al.  Summarizing software artifacts: a case study of bug reports , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[16]  Emily Hill,et al.  Towards automatically generating summary comments for Java methods , 2010, ASE.

[17]  Lori Pollock,et al.  Automatic generation of descriptive summary comments for methods in object-oriented programs , 2012 .

[18]  Gabriele Bavota,et al.  The Impact of API Change- and Fault-Proneness on the User Ratings of Android Apps , 2015, IEEE Transactions on Software Engineering.

[19]  Harald C. Gall,et al.  The impact of test case summaries on bug fixing performance: an empirical investigation , 2016, PeerJ Prepr..

[20]  Jae-Gil Lee,et al.  Facilitating developer-user interactions with mobile app review digests , 2013, CHI Extended Abstracts.

[21]  Taeghyun Kang,et al.  Factors Influencing Quality of Mobile Apps: Role of Mobile App Development Life Cycle , 2014, ArXiv.

[22]  Gerardo Canfora,et al.  Mining source code descriptions from developer communications , 2012, 2012 20th IEEE International Conference on Program Comprehension (ICPC).

[23]  Harald C. Gall,et al.  DECA: Development Emails Content Analyzer , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering Companion (ICSE-C).

[24]  Rachel Harrison,et al.  Retrieving and analyzing mobile apps feature requests from online reviews , 2013, 2013 10th Working Conference on Mining Software Repositories (MSR).

[25]  Christos Faloutsos,et al.  Why people hate your app: making sense of user feedback in a mobile app store , 2013, KDD.

[26]  Harald C. Gall,et al.  Development Emails Content Analyzer: Intention Mining in Developer Discussions (T) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[27]  Owen Rambow,et al.  Summarizing Email Threads , 2004, NAACL.

[28]  Rishi Chandy,et al.  Identifying spam in the iOS app store , 2012, WebQuality '12.

[29]  David A. Wagner,et al.  Do Android users write about electric sheep? Examining consumer reviews in Google Play , 2013, 2013 IEEE 10th Consumer Communications and Networking Conference (CCNC).

[30]  Andrea De Lucia,et al.  Labeling source code with information retrieval methods: an empirical study , 2013, Empirical Software Engineering.

[31]  Walid Maalej,et al.  How Do Users Like This Feature? A Fine Grained Sentiment Analysis of App Reviews , 2014, 2014 IEEE 22nd International Requirements Engineering Conference (RE).

[32]  Andreas Zeller,et al.  It's not a bug, it's a feature: How misclassification impacts bug prediction , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[33]  Ning Chen,et al.  AR-miner: mining informative reviews for developers from mobile app marketplace , 2014, ICSE.

[34]  Ahmed E. Hassan,et al.  What Do Mobile App Users Complain About? , 2015, IEEE Software.

[35]  Martin F. Porter,et al.  An algorithm for suffix stripping , 1997, Program.

[36]  Kristina Winbladh,et al.  Analysis of user comments: An approach for software requirements evolution , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[37]  Gerardo Canfora,et al.  CODES: mining source code descriptions from developers discussions , 2014, ICPC 2014.

[38]  Anima Anandkumar,et al.  Modeling the Mobile Application Development Lifecycle , 2022 .

[39]  Bernd Brügge,et al.  User Feedback in Mobile Development , 2014, MobileDeLi '14.

[40]  Andrian Marcus,et al.  On the Use of Automated Text Summarization Techniques for Summarizing Source Code , 2010, 2010 17th Working Conference on Reverse Engineering.

[41]  Rachel Harrison,et al.  Online Reviews as First Class Artifacts in Mobile App Development , 2013, MobiCASE.

[42]  Lori L. Pollock,et al.  Automatic generation of natural language summaries for Java classes , 2013, 2013 21st International Conference on Program Comprehension (ICPC).

[43]  David Lo,et al.  NIRMAL: Automatic identification of software relevant tweets leveraging language model , 2015, 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER).

[44]  Barbara Paech,et al.  Does Involving Users in Software Development Really Influence System Success? , 2013, IEEE Software.

[45]  Xiaodong Gu,et al.  "What Parts of Your Apps are Loved by Users?" (T) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[46]  Walid Maalej,et al.  User feedback in the appstore: An empirical study , 2013, 2013 21st IEEE International Requirements Engineering Conference (RE).