Understanding in-app advertising issues based on large scale app review analysis

Abstract Context: In-app advertising closely relates to app revenue. Reckless ad integration could adversely impact app quality and user experience, leading to loss of income. It is very challenging to balance the ad revenue and user experience for app developers. Objective: Towards tackling the challenge, we conduct a study on analyzing user concerns about in-app advertisement. Method: Specifically, we present a large-scale analysis on ad-related user feedback. The large user feedback data from App Store and Google Play allow us to summarize ad-related app issues comprehensively and thus provide practical ad integration strategies for developers. We first define common ad issues by manually labeling a statistically representative sample of ad-related feedback, and then build an automatic classifier to categorize ad-related feedback. We study the relations between different ad issues and user ratings to identify the ad issues poorly scored by users. We also explore the fix durations of ad issues across platforms for extracting insights into prioritizing ad issues for ad maintenance. Results: (1) We summarize 15 types of ad issues by manually annotating 903 out of 36,309 ad-related user reviews. From a statistical analysis of 36,309 ad-related reviews, we find that users care most about the number of unique ads and ad display frequency during usage. (2) Users tend to give relatively lower ratings when they report the security and notification related issues. (3) Regarding different platforms, we observe that the distributions of ad issues are significantly different between App Store and Google Play. (4) Some ad issue types are addressed more quickly by developers than other ad issues. Conclusion: We believe the findings we discovered can benefit app developers towards balancing ad revenue and user experience while ensuring app quality.

[1]  S. Ejaz Ahmed.  Effect Sizes for Research: A Broad Application Approach , 2006, Technometrics.

[2]  Walid Maalej,et al.  How Do Users Like This Feature? A Fine Grained Sentiment Analysis of App Reviews , 2014, 2014 IEEE 22nd International Requirements Engineering Conference (RE).

[3]  Xiaodong Gu,et al.  "What Parts of Your Apps are Loved by Users?" (T) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[4]  H. B. Mann,et al.  On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other , 1947 .

[5]  K. Chowdhury,et al.  CONSUMER ATTITUDE TOWARD MOBILE ADVERTISING IN AN EMERGING MARKET: AN EMPIRICAL STUDY , 2006 .

[6]  Ying Zou,et al.  Too Many User-Reviews! What Should App Developers Look at First? , 2019, IEEE Transactions on Software Engineering.

[7]  Yuanyuan Zhang,et al.  A Survey of App Store Analysis for Software Engineering , 2017, IEEE Transactions on Software Engineering.

[8]  Harald C. Gall,et al.  How can i improve my app? Classifying user reviews for software maintenance and evolution , 2015, 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[9]  William G. J. Halfond,et al.  What Aspects of Mobile Ads Do Users Care About? An Empirical Study of Mobile In-app Ad Reviews , 2017, ArXiv.

[10]  M. McHugh,et al.  The Chi-square test of independence , 2013, Biochemia medica.

[11]  Rachel Harrison,et al.  What are you complaining about?: a study of online reviews of mobile applications , 2013, BCS HCI.

[12]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[13]  Michalis Faloutsos,et al.  ProfileDroid: multi-layer profiling of android applications , 2012, Mobicom '12.

[14]  Zan Wang,et al.  Large-Scale Empirical Studies on Effort-Aware Security Vulnerability Prediction Methods , 2020, IEEE Transactions on Reliability.

[15]  Ahmed E. Hassan,et al.  Impact of Ad Libraries on Ratings of Android Mobile Apps , 2014, IEEE Software.

[16]  Peng Liang,et al.  Automatic Classification of Non-Functional Requirements from Augmented App User Reviews , 2017, EASE.

[17]  Andrew Begel,et al.  Analyze this! 145 questions for data scientists in software engineering , 2013, ICSE.

[18]  Gabriele Bavota,et al.  Crowdsourcing user reviews to support the evolution of mobile apps , 2018, J. Syst. Softw..

[19]  Jan Vitek,et al.  FSE/CACM Rebuttal2: Correcting A Large-Scale Study of Programming Languages and Code Quality in GitHub , 2019, ArXiv.

[20]  David Lo,et al.  A Large Scale Study of Multiple Programming Languages and Code Quality , 2016, 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER).

[21]  Tung Thanh Nguyen,et al.  Phrase-based extraction of user opinions in mobile app reviews , 2016, 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE).

[22]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[23]  Miryung Kim,et al.  The Emerging Role of Data Scientists on Software Development Teams , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[24]  J. David Morgenthaler,et al.  Evaluating static analysis defect warnings on production software , 2007, PASTE '07.

[25]  Ahmed E. Hassan,et al.  Analyzing and automatically labelling the types of user issues that are raised in mobile app reviews , 2015, Empirical Software Engineering.

[26]  Rachel Harrison,et al.  Retrieving and analyzing mobile apps feature requests from online reviews , 2013, 2013 10th Working Conference on Mining Software Repositories (MSR).

[27]  Michael R. Lyu,et al.  Exploring the effects of ad schemes on the performance cost of mobile phones , 2018, A-Mobile@ASE.

[28]  YOU’VE GOT MOBILE ADS! YOUNG CONSUMERS’ RESPONSES TO MOBILE ADS WITH DIFFERENT TYPES OF INTERACTIVITY , 2013 .

[29]  Zibin Zheng,et al.  MalPat: Mining Patterns of Malicious and Benign Android Apps via Permission-Related APIs , 2018, IEEE Transactions on Reliability.

[30]  Yuanyuan Zhang,et al.  App store mining and analysis: MSR for app stores , 2012, 2012 9th IEEE Working Conference on Mining Software Repositories (MSR).

[31]  Geoff Holmes,et al.  Classifier chains for multi-label classification , 2009, Machine Learning.

[32]  Michael R. Lyu,et al.  Online App Review Analysis for Identifying Emerging Issues , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[33]  Marcos André Gonçalves,et al.  A Feature-Oriented Sentiment Rating for Mobile App Reviews , 2018, WWW.

[34]  Michael R. Lyu,et al.  Experience Report: Understanding Cross-Platform App Issues from User Reviews , 2016, 2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE).

[35]  Vitaly Shmatikov,et al.  What Mobile Ads Know About Mobile Users , 2016, NDSS.

[36]  Ding Li,et al.  Lightweight Measurement and Estimation of Mobile Ad Energy Consumption , 2016, 2016 IEEE/ACM 5th International Workshop on Green and Sustainable Software (GREENS).

[37]  Suman Nath,et al.  Prefetching mobile ads: can advertising systems afford it? , 2013, EuroSys '13.

[38]  Erik Derr,et al.  Reliable Third-Party Library Detection in Android and its Security Applications , 2016, CCS.

[39]  Karl Pearson F.R.S..  X. On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling , 2009 .

[40]  Harald C. Gall,et al.  What would users change in my app? summarizing app reviews for recommending software changes , 2016, SIGSOFT FSE.

[41]  Gabriele Bavota,et al.  Release Planning of Mobile Apps Based on User Reviews , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[42]  Yuanyuan Zhang,et al.  The App Sampling Problem for App Store Mining , 2015, 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.

[43]  Walid Maalej,et al.  Bug report, feature request, or simply praise? On automatically classifying app reviews , 2015, 2015 IEEE 23rd International Requirements Engineering Conference (RE).

[44]  David W. Hosmer,et al.  Applied Logistic Regression , 1991 .

[45]  Kenneth C. C. Yang,et al.  Factors affecting consumers' responses to mobile advertising from a social norm theoretical perspective , 2010, Telematics Informatics.

[46]  Xuxian Jiang,et al.  Unsafe exposure analysis of mobile in-app advertisements , 2012, WISEC '12.

[47]  Narseo Vallina-Rodriguez,et al.  Breaking for commercials: characterizing mobile advertising , 2012, Internet Measurement Conference.

[48]  Roksana Boreli,et al.  Characterising user targeting for in-App Mobile Ads , 2014, 2014 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[49]  Jieming Zhu,et al.  PAID: Prioritizing app issues for developers by tracking user reviews over versions , 2015, 2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE).

[50]  William G. J. Halfond,et al.  Truth in Advertising: The Hidden Cost of Mobile Ads for Software Developers , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[51]  Suman Nath,et al.  MAdScope: Characterizing Mobile In-App Targeted Ads , 2015, MobiSys.

[52]  Ahmed E. Hassan,et al.  What Do Mobile App Users Complain About? , 2015, IEEE Software.

[53]  Tung Thanh Nguyen,et al.  Mining User Opinions in Mobile App Reviews: A Keyword-Based Approach (T) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[54]  Xuanzhe Liu,et al.  PRADA: Prioritizing Android Devices for Apps by Mining Large-Scale Usage Data , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[55]  Siti Mariyam Shamsuddin,et al.  Classification with class imbalance problem: A review , 2015, SOCO 2015.

[56]  Zhenchang Xing,et al.  Inference of development activities from interaction with uninstrumented applications , 2017, Empirical Software Engineering.

[57]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[58]  Yuanyuan Zhang,et al.  Investigating the relationship between price, rating, and popularity in the Blackberry World App Store , 2017, Inf. Softw. Technol..