Investigating the criticality of user‐reported issues through their relations with app rating

App quality impacts user experience and satisfaction. As a consequence, both app ratings and user feedback reported in app reviews are directly influenced by the user‐perceived app quality. Through an empirical study involving 210,517 reviews related to 317 Android apps, in this paper, we experiment with the combined usage of app rating and user reviews analysis (i) to investigate the most important factors influencing the perceived app quality, (ii) focusing on the topics discussed in user review that most relate with app rating. Besides, we investigate whether specific code quality metrics could be monitored to prevent the rising of negative user feedback (i.e., types of user review comments), connected with low ratings. Our study demonstrates that user comments reporting bugs are negatively correlated with the rating, while reviews reportingfeature requests do not. Interestingly, depending on the app category, we observed that different kinds of issues have rather different relationships with the rating and the user‐perceived quality of the app. In particular, we observe that for specific app categories (e.g., communication), some code quality factors have significant relationships with the raising of certain types of feedback, which, in turn, are negatively connected with app ratings.

[1]  Rachel Harrison,et al.  What are you complaining about?: a study of online reviews of mobile applications , 2013, BCS HCI.

[2]  Walid Maalej,et al.  How Do Users Like This Feature? A Fine Grained Sentiment Analysis of App Reviews , 2014, 2014 IEEE 22nd International Requirements Engineering Conference (RE).

[3]  Gernot Heiser,et al.  An Analysis of Power Consumption in a Smartphone , 2010, USENIX Annual Technical Conference.

[4]  Ying Zou,et al.  An Exploratory Study on the Relation between User Interface Complexity and the Perceived Quality , 2014, ICWE.

[5]  Diomidis Spinellis,et al.  Undocumented and unchecked: exceptions that spell trouble , 2014, MSR 2014.

[6]  Chris F. Kemerer,et al.  A Metrics Suite for Object Oriented Design , 2015, IEEE Trans. Software Eng..

[7]  Michael R. Lyu,et al.  Experience Report: Detecting Poor-Responsive UI in Android Applications , 2016, 2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE).

[8]  Marcela Ruiz,et al.  Requirements-Collector: Automating Requirements Specification from Elicitation Sessions and User Feedback , 2020, 2020 IEEE 28th International Requirements Engineering Conference (RE).

[9]  Ramesh Govindan,et al.  Estimating mobile application energy consumption using program analysis , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[10]  Gerardo Canfora,et al.  SURF: Summarizer of User Reviews Feedback , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering Companion (ICSE-C).

[11]  Sang-Hoon Kim,et al.  Controlling physical memory fragmentation in mobile systems , 2015, ISMM.

[12]  Ahmed E. Hassan,et al.  An Examination of the Current Rating System used in Mobile App Stores , 2017 .

[13]  Walid Maalej,et al.  User feedback in the appstore: An empirical study , 2013, 2013 21st IEEE International Requirements Engineering Conference (RE).

[14]  Gabriele Bavota,et al.  Crowdsourcing user reviews to support the evolution of mobile apps , 2018, J. Syst. Softw..

[15]  Ying Zou,et al.  Winning the app production rally , 2018, ESEC/SIGSOFT FSE.

[16]  Harald C. Gall,et al.  Analyzing reviews and code of mobile apps for better release planning , 2017, 2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER).

[17]  Harald C. Gall,et al.  What would users change in my app? summarizing app reviews for recommending software changes , 2016, SIGSOFT FSE.

[18]  Atanas Rountev,et al.  Testing for poor responsiveness in android applications , 2013, 2013 1st International Workshop on the Engineering of Mobile-Enabled Systems (MOBS).

[19]  Shaohua Wang,et al.  Towards prioritizing user-related issue reports of mobile applications , 2019, Empirical Software Engineering.

[20]  Harald C. Gall,et al.  Recommending and Localizing Change Requests for Mobile Apps Based on User Reviews , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE).

[21]  Alfonso Fuggetta,et al.  Software process , 2014, FOSE.

[22]  Shingo Takada,et al.  Responsiveness analysis tool for Android application , 2014, DeMobile@SIGSOFT FSE.

[23]  Gerardo Canfora,et al.  Exploring Mobile User Experience Through Code Quality Metrics , 2016, PROFES.

[24]  Mark Harman,et al.  Causal impact analysis for app releases in google play , 2016, SIGSOFT FSE.

[25]  Ahmed E. Hassan,et al.  Examining the Rating System Used in Mobile-App Stores , 2016, IEEE Software.

[26]  Gerardo Canfora,et al.  Android apps and user feedback: a dataset for software evolution and quality improvement , 2017, WAMA@ESEC/SIGSOFT FSE.

[27]  Maleknaz Nayebi,et al.  App store mining is not enough for app improvement , 2018, Empirical Software Engineering.

[28]  Cor-Paul Bezemer,et al.  Studying the consistency of star ratings and reviews of popular free hybrid Android and iOS apps , 2018, Empirical Software Engineering.

[29]  Marcos André Gonçalves,et al.  A Feature-Oriented Sentiment Rating for Mobile App Reviews , 2018, WWW.

[30]  Ilenia Fronza,et al.  Better Code for Better Apps: A Study on Source Code Quality and Market Success of Android Applications , 2015, 2015 2nd ACM International Conference on Mobile Software Engineering and Systems.

[31]  Gabriele Bavota,et al.  API change and fault proneness: a threat to the success of Android apps , 2013, ESEC/FSE 2013.

[32]  Yuanyuan Zhang,et al.  The App Sampling Problem for App Store Mining , 2015, 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.

[33]  Li Zhang,et al.  A user satisfaction analysis approach for software evolution , 2010, 2010 IEEE International Conference on Progress in Informatics and Computing.

[34]  Gabriele Bavota,et al.  Mining energy-greedy API usage patterns in Android apps: an empirical study , 2014, MSR 2014.

[35]  Ahmed E. Hassan,et al.  What Do Mobile App Users Complain About? , 2015, IEEE Software.

[36]  Jon G. Rokne,et al.  User Feedback from Tweets vs App Store Reviews: An Exploratory Study of Frequency, Timing and Content , 2018, 2018 5th International Workshop on Artificial Intelligence for Requirements Engineering (AIRE).

[37]  A. Zeller,et al.  Predicting Defects for Eclipse , 2007, Third International Workshop on Predictor Models in Software Engineering (PROMISE'07: ICSE Workshops 2007).

[38]  William G. J. Halfond,et al.  What Aspects of Mobile Ads Do Users Care About? An Empirical Study of Mobile In-app Ad Reviews , 2017, ArXiv.

[39]  Yuanyuan Zhang,et al.  A Survey of App Store Analysis for Software Engineering , 2017, IEEE Transactions on Software Engineering.

[40]  Yuanyuan Zhang,et al.  App store mining and analysis: MSR for app stores , 2012, 2012 9th IEEE Working Conference on Mining Software Repositories (MSR).

[41]  Harald C. Gall,et al.  How can i improve my app? Classifying user reviews for software maintenance and evolution , 2015, 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[42]  Alexander Felfernig,et al.  Counteracting Anchoring Effects in Group Decision Making , 2015, UMAP.

[43]  Michele Lanza,et al.  Evaluating defect prediction approaches: a benchmark and an extensive comparison , 2011, Empirical Software Engineering.

[44]  Jin-Soo Kim,et al.  Controlling physical memory fragmentation in mobile systems , 2015, ISMM.

[45]  Bernd Bruegge,et al.  Ensemble Methods for App Review Classification: An Approach for Software Evolution (N) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[46]  W. Pirie Spearman Rank Correlation Coefficient , 2006 .

[47]  Peter C. Rigby,et al.  The influence of App churn on App success and StackOverflow discussions , 2015, 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER).

[48]  Michael R. Lyu,et al.  Online App Review Analysis for Identifying Emerging Issues , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[49]  Harald C. Gall,et al.  ARdoc: app reviews development oriented classifier , 2016, SIGSOFT FSE.

[50]  Ying Zou,et al.  Too Many User-Reviews! What Should App Developers Look at First? , 2019, IEEE Transactions on Software Engineering.

[51]  Norbert Seyff,et al.  End-user Driven Feedback Prioritization , 2017, REFSQ Workshops.

[52]  Selim Ickin,et al.  Factors influencing quality of experience of commonly used mobile applications , 2012, IEEE Communications Magazine.

[53]  Aniello Cimitile,et al.  An exploratory study on the evolution of Android malware quality , 2018, J. Softw. Evol. Process..

[54]  Amjad Hudaib,et al.  Requirements Prioritization Techniques Review and Analysis , 2017, 2017 International Conference on New Trends in Computing Sciences (ICTCS).

[55]  Christos Faloutsos,et al.  Why people hate your app: making sense of user feedback in a mobile app store , 2013, KDD.

[56]  Gabriele Bavota,et al.  The Impact of API Change- and Fault-Proneness on the User Ratings of Android Apps , 2015, IEEE Transactions on Software Engineering.

[57]  David Lo,et al.  What are the characteristics of high-rated apps? A case study on free Android Applications , 2015, 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[58]  Rudolf Ferenc,et al.  Using the Conceptual Cohesion of Classes for Fault Prediction in Object-Oriented Systems , 2008, IEEE Transactions on Software Engineering.

[59]  Ning Chen,et al.  AR-miner: mining informative reviews for developers from mobile app marketplace , 2014, ICSE.