Learn More, Pay Less! Lessons Learned from Applying the Wizard-of-Oz Technique for Exploring Mobile App Requirements

Mobile apps have exploded in popularity, encouraging developers to provide content to the massive user base of the main app stores. Although there exist automated techniques that can classify user comments into various topics with high levels of precision, recent studies have shown that the top apps in the app stores do not have customer ratings that directly correlate with the app's success. This implies that no single requirements elicitation technique can cover the full depth required to produce a successful product and that applying alternative requirements gathering techniques can lead to success when these two are combined. Since user involvement has been found to be the most impactful contribution to project success, in this paper we will explore how the Wizard-of-Oz (WOz) technique and user reviews available in Google Play, can be integrated to produce a product that meets the demand of more stakeholders than either method alone. To compare the role of early interactive requirements specification and app reviews, we conducted two studies: (i) a case study analysis on 13 mobile app development teams who used very early stages Requirements Engineering (RE) by applying WOz, and (ii) a study analyzing 40 (70, 592 reviews) similar mobile apps on Google Play. The results of both studies show that while each of WOz and app review analysis techniques can be applied to capture specific types of requirements, an integrated process including both methods would eliminate the communication gap between users and developers at early stages of the development process and mitigates the risk of requirements change in later stages.

[1]  Zahra Shakeri Hossein Abad,et al.  Towards Tool Support for Situational Engineering of Agile Methodologies , 2010, 2010 Asia Pacific Software Engineering Conference.

[2]  Ning Chen,et al.  AR-miner: mining informative reviews for developers from mobile app marketplace , 2014, ICSE.

[3]  K. R. Priya.  Grounded Theory Methodology , 2013 .

[4]  Kristina Winbladh,et al.  Analysis of user comments: An approach for software requirements evolution , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[5]  Lennart Molin,et al.  Wizard-of-Oz prototyping for co-operative interaction design of graphical user interfaces , 2004, NordiCHI '04.

[6]  Rozilawati Razali,et al.  SELECTING THE RIGHT STAKEHOLDERS FOR REQUIREMENTS ELICITATION: A SYSTEMATIC APPROACH , 2011 .

[7]  Blair MacIntyre,et al.  Wizard of Oz support throughout an iterative design process , 2005, IEEE Pervasive Computing.

[8]  Zahra Shakeri Hossein Abad,et al.  Enhancing Tool Support for Situational Engineering of Agile Methodologies in Eclipse , 2012, SERA.

[9]  Anthony I. Wasserman,et al.  Software engineering issues for mobile application development , 2010, FoSER '10.

[10]  Behrouz Homayoun Far,et al.  Let’s Hear it from RETTA: A Requirements Elicitation Tool for TrAffic Management Systems , 2017, 2017 IEEE 25th International Requirements Engineering Conference (RE).

[11]  Malin Wik.  Using the Wizard-of-Oz technique in requirements engineering processes : A trial in a tourism context , 2015 .

[12]  Zahra Shakeri Hossein Abad,et al.  What are Practitioners Asking about Requirements Engineering? An Exploratory Analysis of Social Q&A Sites , 2016, 2016 IEEE 24th International Requirements Engineering Conference Workshops (REW).

[13]  Dimitris Karagiannis,et al.  Modelling mobile app requirements for semantic traceability , 2015, Requirements Engineering.

[14]  M. Patton,et al.  Qualitative evaluation and research methods , 1992 .

[15]  D. Rennie.  Grounded Theory Methodology , 1998 .

[16]  Walid Maalej,et al.  User feedback in the appstore: An empirical study , 2013, 2013 21st IEEE International Requirements Engineering Conference (RE).

[17]  Rozilawati Razali,et al.  A practical guide to requirements elicitation techniques selection - An empirical study , 2012 .

[18]  Kurt Schneider,et al.  What Works Better? A Study of Classifying Requirements , 2017, 2017 IEEE 25th International Requirements Engineering Conference (RE).

[19]  Zahra Shakeri Hossein Abad,et al.  Requirements Engineering Visualization: A Systematic Literature Review , 2016, 2016 IEEE 24th International Requirements Engineering Conference (RE).

[20]  A. Strauss,et al.  Grounded Theory in Practice , 1997 .

[21]  Max Welling,et al.  Fast collapsed gibbs sampling for latent dirichlet allocation , 2008, KDD.

[22]  Neil A. Ernst,et al.  Case studies in just-in-time requirements analysis , 2012, 2012 Second IEEE International Workshop on Empirical Requirements Engineering (EmpiRE).

[23]  Rozilawati Razali,et al.  Usability Dimensions for Mobile Applications-A Review , 2013 .

[24]  Taeghyun Kang,et al.  Factors Influencing Quality of Mobile Apps: Role of Mobile App Development Life Cycle , 2014, ArXiv.

[25]  Alain Abran,et al.  The state of the art of mobile application usability evaluation , 2012, 2012 25th IEEE Canadian Conference on Electrical and Computer Engineering (CCECE).

[26]  Yang Li,et al.  Design Challenges and Principles for Wizard of Oz Testing of Location-Enhanced Applications , 2007, IEEE Pervasive Computing.

[27]  Philippe Kruchten,et al.  Real Challenges in Mobile App Development , 2013, 2013 ACM / IEEE International Symposium on Empirical Software Engineering and Measurement.

[28]  David M. Blei,et al.  Probabilistic topic models , 2012, Commun. ACM.