An Empirical Study of In-App Advertising Issues Based on Large Scale App Review Analysis

In-app advertising closely relates to app revenue. Reckless ad integration could adversely impact app reliability and user experience, leading to loss of income. It is very challenging to balance the ad revenue and user experience for app developers. In this paper, we present a large-scale analysis on ad-related user feedback. The large user feedback data from App Store and Google Play allow us to summarize ad-related app issues comprehensively and thus provide practical ad integration strategies for developers. We first define common ad issues by manually labeling a statistically representative sample of ad-related feedback, and then build an automatic classifier to categorize ad-related feedback. We study the relations between different ad issues and user ratings to identify the ad issues poorly scored by users. We also explore the fix durations of ad issues across platforms for extracting insights into prioritizing ad issues for ad maintenance. We summarize 15 types of ad issues by manually annotating 903/36,309 ad-related user reviews. From a statistical analysis of 36,309 ad-related reviews, we find that users care most about the number of unique ads and ad display frequency during usage. Besides, users tend to give relatively lower ratings when they report the security and notification related issues. Regarding different platforms, we observe that the distributions of ad issues are significantly different between App Store and Google Play. Moreover, some ad issue types are addressed more quickly by developers than other ad issues. We believe the findings we discovered can benefit app developers towards balancing ad revenue and user experience while ensuring app reliability.

[1]  Stanley Lemeshow,et al.  Applied Logistic Regression, Second Edition , 1989 .

[2]  Andrew Begel,et al.  Analyze this! 145 questions for data scientists in software engineering , 2013, ICSE.

[3]  S. Ejaz Ahmed.  Effect Sizes for Research: A Broad Application Approach , 2006, Technometrics.

[4]  Tung Thanh Nguyen,et al.  Phrase-based extraction of user opinions in mobile app reviews , 2016, 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE).

[5]  YOU’VE GOT MOBILE ADS! YOUNG CONSUMERS’ RESPONSES TO MOBILE ADS WITH DIFFERENT TYPES OF INTERACTIVITY , 2013 .

[6]  Gabriele Bavota,et al.  Crowdsourcing user reviews to support the evolution of mobile apps , 2018, J. Syst. Softw..

[7]  H. B. Mann,et al.  On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other , 1947 .

[8]  K. Pearson.  On the Criterion that a Given System of Deviations from the Probable in the Case of a Correlated System of Variables is Such that it Can be Reasonably Supposed to have Arisen from Random Sampling , 1900 .

[9]  Ian Stewart.  Ad infinitum , 1989, Nature.

[10]  Kenneth C. C. Yang,et al.  Factors affecting consumers' responses to mobile advertising from a social norm theoretical perspective , 2010, Telematics Informatics.

[11]  Jieming Zhu,et al.  PAID: Prioritizing app issues for developers by tracking user reviews over versions , 2015, 2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE).

[12]  Vitaly Shmatikov,et al.  What Mobile Ads Know About Mobile Users , 2016, NDSS.

[13]  Ahmed E. Hassan,et al.  What Do Mobile App Users Complain About? , 2015, IEEE Software.

[14]  Premkumar T. Devanbu,et al.  A large scale study of programming languages and code quality in github , 2014, SIGSOFT FSE.

[15]  Yuanyuan Zhang,et al.  A Survey of App Store Analysis for Software Engineering , 2017, IEEE Transactions on Software Engineering.

[16]  Xiaodong Gu,et al.  "What Parts of Your Apps are Loved by Users?" (T) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[17]  Zibin Zheng,et al.  MalPat: Mining Patterns of Malicious and Benign Android Apps via Permission-Related APIs , 2018, IEEE Transactions on Reliability.

[18]  William G. J. Halfond,et al.  What Aspects of Mobile Ads Do Users Care About? An Empirical Study of Mobile In-app Ad Reviews , 2017, ArXiv.

[19]  Suman Nath,et al.  Prefetching mobile ads: can advertising systems afford it? , 2013, EuroSys '13.

[20]  Rachel Harrison,et al.  What are you complaining about?: a study of online reviews of mobile applications , 2013, BCS HCI.

[21]  Walid Maalej,et al.  Bug report, feature request, or simply praise? On automatically classifying app reviews , 2015, 2015 IEEE 23rd International Requirements Engineering Conference (RE).

[22]  Roksana Boreli,et al.  Characterising user targeting for in-App Mobile Ads , 2014, 2014 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[23]  Siti Mariyam Shamsuddin,et al.  Classification with class imbalance problem: A review , 2015, SOCO 2015.

[24]  Erik Derr,et al.  Reliable Third-Party Library Detection in Android and its Security Applications , 2016, CCS.

[25]  Ying Zou,et al.  Too Many User-Reviews! What Should App Developers Look at First? , 2019, IEEE Transactions on Software Engineering.

[26]  Narseo Vallina-Rodriguez,et al.  Breaking for commercials: characterizing mobile advertising , 2012, Internet Measurement Conference.

[27]  Michael R. Lyu,et al.  Experience Report: Understanding Cross-Platform App Issues from User Reviews , 2016, 2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE).

[28]  Yuanyuan Zhang,et al.  App store mining and analysis: MSR for app stores , 2012, 2012 9th IEEE Working Conference on Mining Software Repositories (MSR).

[29]  Xuxian Jiang,et al.  Unsafe exposure analysis of mobile in-app advertisements , 2012, WISEC '12.

[30]  William G. J. Halfond,et al.  Truth in Advertising: The Hidden Cost of Mobile Ads for Software Developers , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[31]  Gries,et al.  The chi-square test for independence , 2022 .

[32]  Suman Nath,et al.  MAdScope: Characterizing Mobile In-App Targeted Ads , 2015, MobiSys.

[33]  J. David Morgenthaler,et al.  Evaluating static analysis defect warnings on production software , 2007, PASTE '07.

[34]  Ahmed E. Hassan,et al.  Impact of Ad Libraries on Ratings of Android Mobile Apps , 2014, IEEE Software.

[35]  Harald C. Gall,et al.  What would users change in my app? summarizing app reviews for recommending software changes , 2016, SIGSOFT FSE.

[36]  Geoff Holmes,et al.  Classifier chains for multi-label classification , 2009, Machine Learning.

[37]  Tung Thanh Nguyen,et al.  Mining User Opinions in Mobile App Reviews: A Keyword-Based Approach (T) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[38]  Yuanyuan Zhang,et al.  The App Sampling Problem for App Store Mining , 2015, 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.

[39]  Karl Pearson F.R.S..  X. On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling , 2009 .

[40]  Miryung Kim,et al.  The Emerging Role of Data Scientists on Software Development Teams , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[41]  Zan Wang,et al.  Large-Scale Empirical Studies on Effort-Aware Security Vulnerability Prediction Methods , 2020, IEEE Transactions on Reliability.

[42]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[43]  Xuanzhe Liu,et al.  PRADA: Prioritizing Android Devices for Apps by Mining Large-Scale Usage Data , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[44]  Michael R. Lyu,et al.  Online App Review Analysis for Identifying Emerging Issues , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[45]  Walid Maalej,et al.  How Do Users Like This Feature? A Fine Grained Sentiment Analysis of App Reviews , 2014, 2014 IEEE 22nd International Requirements Engineering Conference (RE).

[46]  Michalis Faloutsos,et al.  ProfileDroid: multi-layer profiling of android applications , 2012, Mobicom '12.

[47]  Ahmed E. Hassan,et al.  Analyzing and automatically labelling the types of user issues that are raised in mobile app reviews , 2015, Empirical Software Engineering.

[48]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[49]  Yuanyuan Zhang,et al.  Investigating the relationship between price, rating, and popularity in the Blackberry World App Store , 2017, Inf. Softw. Technol..

[50]  David Lo,et al.  A Large Scale Study of Multiple Programming Languages and Code Quality , 2016, 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER).

[51]  K. Chowdhury,et al.  CONSUMER ATTITUDE TOWARD MOBILE ADVERTISING IN AN EMERGING MARKET: AN EMPIRICAL STUDY , 2006 .

[52]  Gabriele Bavota,et al.  Release Planning of Mobile Apps Based on User Reviews , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[53]  Michael R. Lyu,et al.  Exploring the effects of ad schemes on the performance cost of mobile phones , 2018, A-Mobile@ASE.

[54]  Marcos André Gonçalves,et al.  A Feature-Oriented Sentiment Rating for Mobile App Reviews , 2018, WWW.

[55]  Ding Li,et al.  Lightweight Measurement and Estimation of Mobile Ad Energy Consumption , 2016, 2016 IEEE/ACM 5th International Workshop on Green and Sustainable Software (GREENS).

[56]  Rachel Harrison,et al.  Retrieving and analyzing mobile apps feature requests from online reviews , 2013, 2013 10th Working Conference on Mining Software Repositories (MSR).

[57]  Zhenchang Xing,et al.  Inference of development activities from interaction with uninstrumented applications , 2017, Empirical Software Engineering.