Toward Empirically Investigating Non-Functional Requirements of iOS Developers on Stack Overflow

Context: Mobile application developers are getting more concerned due to the importance of quality requirements or non-functional requirements (NFR) in software quality. Developers around the globe are actively asking a question(s) and sharing solutions to the problems related to software development on Stack Overflow (SO). The knowledge shared by developers on SO contains useful information related to software development such as feature requests (functional/non-functional), code snippets, reporting bugs or sentiments. Extracting the NFRs shared by iOS developers on programming Q&A website SO has become a challenge and a less researched area. Objective: To identify and understand the real problems, needs, trends, and the critical NFRs or quality requirements discussed on Stack Overflow related to iOS mobile application development. Method: We extracted and used only the iOS posts data of SO. We applied the well-known statistical topical model Latent Dirichlet Allocation (LDA) to identify the main topics in iOS posts on SO. Then, we labeled the extracted topics with quality requirements or NFRs by using the wordlists to assess the trend, evolution, hot and unresolved NFRs in all iOS discussions. Results: Our findings revealed that the highly frequent topics the iOS developers discussed are related to usability, reliability, and functionality followed by efficiency. Interestingly, the most problematic areas unresolved are also usability, reliability, and functionality though followed by portability. Besides, the evolution trend of each of the six different quality requirements or NFRs over time is depicted through comprehensive visualization. Conclusion: Our first empirical investigation on approximately 1.5 million iOS posts and comments of SO gives insight on comprehending the NFRs in iOS application development through the lens of real-world practitioners.

[1]  Didar Zowghi,et al.  An investigation into the notion of non-functional requirements , 2010, SAC '10.

[2]  Charles A. Sutton,et al.  Why, when, and what: Analyzing Stack Overflow questions by topic, type, and code , 2013, 2013 10th Working Conference on Mining Software Repositories (MSR).

[3]  Dan Yang,et al.  A component recommender for bug reports using Discriminative Probability Latent Semantic Analysis , 2016, Inf. Softw. Technol..

[4]  Nicole Novielli,et al.  The challenges of sentiment detection in the social programmer ecosystem , 2015, SSE@SIGSOFT FSE.

[5]  P. Choudhary.  Analyzing Rater Agreement: Manifest Variable Methods , 2006 .

[6]  Frank Maurer,et al.  What makes a good code example?: A study of programming Q&A in StackOverflow , 2012, 2012 28th IEEE International Conference on Software Maintenance (ICSM).

[7]  Michael W. Godfrey,et al.  Automated topic naming to support cross-project analysis of software maintenance activities , 2011, MSR '11.

[8]  Jure Leskovec,et al.  Discovering value from community activity on focused question answering sites: a case study of stack overflow , 2012, KDD.

[9]  Xavier Franch,et al.  How do software architects consider non-functional requirements: An exploratory study , 2012, 2012 20th IEEE International Requirements Engineering Conference (RE).

[10]  Ahmed E. Hassan,et al.  What Do Mobile App Users Complain About? , 2015, IEEE Software.

[11]  Jessica Koehler,et al.  Open Innovation The New Imperative For Creating And Profiting From Technology , 2016 .

[12]  Kristina Winbladh,et al.  Analysis of user comments: An approach for software requirements evolution , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[13]  Walid Maalej,et al.  How Do Users Like This Feature? A Fine Grained Sentiment Analysis of App Reviews , 2014, 2014 IEEE 22nd International Requirements Engineering Conference (RE).

[14]  Lena Mamykina,et al.  Design lessons from the fastest q&a site in the west , 2011, CHI.

[15]  John Mylopoulos,et al.  Representing and Using Nonfunctional Requirements: A Process-Oriented Approach , 1992, IEEE Trans. Software Eng..

[16]  Dietmar Pfahl,et al.  How can open source software development help requirements management gain the potential of open innovation: An exploratory study , 2012, Proceedings of the 2012 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement.

[17]  Rodrigo Pereira dos Santos,et al.  Supporting governance of mobile application developers from mining and analyzing technical questions in stack overflow , 2018, J. Softw. Eng. Res. Dev..

[18]  Fernando Kamei,et al.  What programmers say about refactoring tools?: an empirical investigation of stack overflow , 2013, WRT '13.

[19]  Miryung Kim,et al.  Validity concerns in software engineering research , 2010, FoSER '10.

[20]  Yuanyuan Zhang,et al.  A Survey of App Store Analysis for Software Engineering , 2017, IEEE Transactions on Software Engineering.

[21]  Gustavo Pinto,et al.  A study on the most popular questions about concurrent programming , 2015, PLATEAU@SPLASH.

[22]  Gabriele Bavota,et al.  On the Impact of API Change-and Fault-Proneness on the User Ratings of Android Apps — Additional Analyses — , 2014 .

[23]  Walid Maalej,et al.  User feedback in the appstore: An empirical study , 2013, 2013 21st IEEE International Requirements Engineering Conference (RE).

[24]  Christoph Treude,et al.  Crowd Documentation : Exploring the Coverage and the Dynamics of API Discussions on Stack Overflow , 2012 .

[25]  Martin Glinz,et al.  On Non-Functional Requirements , 2007, 15th IEEE International Requirements Engineering Conference (RE 2007).

[26]  Chong Feng,et al.  Challenges of mobile applications development: Initial results , 2017, 2017 8th IEEE International Conference on Software Engineering and Service Science (ICSESS).

[27]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[28]  Xavier Franch,et al.  Non-functional Requirements in Architectural Decision Making , 2013, IEEE Software.

[29]  Christoph Treude,et al.  How do programmers ask and answer questions on the web?: NIER track , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[30]  Xinli Yang,et al.  What Security Questions Do Developers Ask? A Large-Scale Study of Stack Overflow Posts , 2016, Journal of Computer Science and Technology.

[31]  Yuanyuan Zhang,et al.  App store mining and analysis: MSR for app stores , 2012, 2012 9th IEEE Working Conference on Mining Software Repositories (MSR).

[32]  Jure Leskovec,et al.  Steering user behavior with badges , 2013, WWW.

[33]  Arshad Ahmad,et al.  The Importance of Knowledge Management Practices in Overcoming the Global Software Engineering Challenges in Requirements Understanding , 2008 .

[34]  Walid Maalej,et al.  On the automatic classification of app reviews , 2016, Requirements Engineering.

[35]  Ling Xu,et al.  Which Non-functional Requirements Do Developers Focus On? An Empirical Study on Stack Overflow Using Topic Analysis , 2015, 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.

[36]  W. Damm,et al.  Boosting Re-use of Embedded Automotive Applications Through Rich Components , 2005 .

[37]  M. Mukaka,et al.  Statistics corner: A guide to appropriate use of correlation coefficient in medical research. , 2012, Malawi medical journal : the journal of Medical Association of Malawi.

[38]  Mahrukh Umar,et al.  Analyzing Non-Functional Requirements (NFRs) for software development , 2011, 2011 IEEE 2nd International Conference on Software Engineering and Service Science.

[39]  Ahmed E. Hassan,et al.  What are developers talking about? An analysis of topics and trends in Stack Overflow , 2014, Empirical Software Engineering.

[40]  Andreas Borg,et al.  The Bad Conscience of Requirements Engineering : An Investigation in Real-World Treatment of Non-Functional Requirements , 2003 .

[41]  Andrew Marder.  Stack Overflow Badges and User Behavior: An Econometric Approach , 2015, 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.

[42]  Ying Zou,et al.  A study of the relation of mobile device attributes with the user-perceived quality of Android apps , 2017, Empirical Software Engineering.

[43]  Xavier Franch,et al.  Dealing with Non-Functional Requirements in Model-Driven Development , 2010, 2010 18th IEEE International Requirements Engineering Conference.

[44]  Nicole Novielli,et al.  Mining Successful Answers in Stack Overflow , 2015, 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.

[45]  Ali Mesbah,et al.  Mining questions asked by web developers , 2014, MSR 2014.

[46]  Walid Maalej,et al.  Bug report, feature request, or simply praise? On automatically classifying app reviews , 2015, 2015 IEEE 23rd International Requirements Engineering Conference (RE).

[47]  Sylvain Lecomte,et al.  Discovering discussion topics about development of cross-platform mobile applications using a cross-compiler development framework , 2017, ArXiv.

[48]  Abram Hindle,et al.  Mining StackOverflow to Filter Out Off-Topic IRC Discussion , 2015, 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.

[49]  Arshad Ahmad,et al.  Requirements Understanding in Global Software Engineering: Industrial Surveys , 2011 .

[50]  Matthias Galster,et al.  A Taxonomy for Identifying and Specifying Non-Functional Requirements in Service-Oriented Development , 2008, 2008 IEEE Congress on Services - Part I.

[51]  Emad Shihab,et al.  What are mobile developers asking about? A large scale study using stack overflow , 2016, Empirical Software Engineering.

[52]  Eleni Stroulia,et al.  On the Personality Traits of StackOverflow Users , 2013, 2013 IEEE International Conference on Software Maintenance.

[53]  Lawrence Chung,et al.  Dealing with Non-Functional Requirements: Three Experimental Studies of a Process-Oriented Approach , 1995, 1995 17th International Conference on Software Engineering.

[54]  Li Zhang,et al.  A user satisfaction analysis approach for software evolution , 2010, 2010 IEEE International Conference on Progress in Informatics and Computing.

[55]  Julio Cesar Sampaio do Prado Leite,et al.  On Non-Functional Requirements in Software Engineering , 2009, Conceptual Modeling: Foundations and Applications.

[56]  Alberto Bacchelli,et al.  ETA: Estimated Time of Answer Predicting Response Time in Stack Overflow , 2015, 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.

[57]  Scott Grant,et al.  Estimating the Optimal Number of Latent Concepts in Source Code Analysis , 2010, 2010 10th IEEE Working Conference on Source Code Analysis and Manipulation.

[58]  Andreas Vogelsang,et al.  How to Specify Non-Functional Requirements to Support Seamless Modeling? A Study Design and Preliminary Results , 2015, 2015 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM).

[59]  Zhenchang Xing,et al.  What help do developers seek, when and how? , 2013, 2013 20th Working Conference on Reverse Engineering (WCRE).

[60]  Jens Grabowski,et al.  Intuition vs. Truth: Evaluation of Common Myths about StackOverflow Posts , 2015, 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.

[61]  Jane Cleland-Huang,et al.  The Detection and Classification of Non-Functional Requirements with Application to Early Aspects , 2006, 14th IEEE International Requirements Engineering Conference (RE'06).

[62]  Julita Vassileva,et al.  Toward Social Learning Environments , 2008, IEEE Transactions on Learning Technologies.

[63]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[64]  Christos Faloutsos,et al.  Why people hate your app: making sense of user feedback in a mobile app store , 2013, KDD.

[65]  Stephen W. Thomas.  Mining Unstructured Software Repositories Using IR Models , 2012 .

[66]  Arshad Ahmad,et al.  Knowledge Management: A Solution to Requirements Understanding in Global Software Engineering , 2012 .

[67]  Ling Xu,et al.  Automated Duplicate Bug Report Detection Using Multi-Factor Analysis , 2016, IEICE Trans. Inf. Syst..

[68]  Chanchal Kumar Roy,et al.  Answering questions about unanswered questions of Stack Overflow , 2013, 2013 10th Working Conference on Mining Software Repositories (MSR).

[69]  Arshad Ahmad,et al.  Requirements Understanding: A Challenge in Global Software Development, Industrial Surveys in Kingdom of Saudi Arabia , 2012, 2012 IEEE 36th Annual Computer Software and Applications Conference.

[70]  Eduard C. Groen,et al.  Users — The Hidden Software Product Quality Experts?: A Study on How App Users Report Quality Aspects in Online Reviews , 2017, 2017 IEEE 25th International Requirements Engineering Conference (RE).

[71]  Tu Minh Phuong,et al.  Topic-based defect prediction: NIER track , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[72]  Bogdan Dit,et al.  An exploratory analysis of mobile development issues using stack overflow , 2013, 2013 10th Working Conference on Mining Software Repositories (MSR).

[73]  Letha H. Etzkorn,et al.  Bug localization using latent Dirichlet allocation , 2010, Inf. Softw. Technol..

[74]  Reidar Conradi,et al.  Adoption of open source software in software-intensive organizations - A systematic literature review , 2010, Inf. Softw. Technol..

[75]  Michael W. Godfrey,et al.  What's hot and what's not: Windowed developer topic analysis , 2009, 2009 IEEE International Conference on Software Maintenance.

[76]  Ruslan Salakhutdinov,et al.  Evaluation methods for topic models , 2009, ICML '09.

[77]  Per Runeson,et al.  A theory of openness for software engineering tools in software organizations , 2017, Inf. Softw. Technol..

[78]  Robert Feldt,et al.  Validity Threats in Empirical Software Engineering Research - An Initial Survey , 2010, SEKE.

[79]  Andreas Vogelsang,et al.  Are "Non-functional" Requirements really Non-functional? An Investigation of Non-functional Requirements in Practice , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[80]  Chong Feng,et al.  A survey on mining stack overflow: question and answering (Q&A) community , 2018, Data Technol. Appl..

[81]  Thomas Olsson,et al.  Non-functional requirements in industry - three case studies adopting an experience-based NFR method , 2005, 13th IEEE International Conference on Requirements Engineering (RE'05).

[82]  Ying Fu,et al.  Automated classification of software change messages by semi-supervised Latent Dirichlet Allocation , 2015, Inf. Softw. Technol..

[83]  Barry W. Boehm,et al.  Quantitative evaluation of software quality , 1976, ICSE '76.

[84]  Scott Grant,et al.  Encouraging user behaviour with achievements: An empirical study , 2013, 2013 10th Working Conference on Mining Software Repositories (MSR).

[85]  Per Runeson,et al.  Open innovation in software engineering: a systematic mapping study , 2015, Empirical Software Engineering.

[86]  Katsuro Inoue,et al.  Quick Trigger on Stack Overflow: A Study of Gamification-Influenced Member Tendencies , 2015, 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.

[87]  Jean-Michel Bruel,et al.  Modeling and verification of Functional and Non-Functional Requirements of ambient Self-Adaptive Systems , 2015, J. Syst. Softw..

[88]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[89]  Björn Regnell,et al.  Requirements engineering in open innovation: a research agenda , 2015, ICSSP.

[90]  Jeffrey C. Carver,et al.  Building reputation in StackOverflow: An empirical investigation , 2013, 2013 10th Working Conference on Mining Software Repositories (MSR).

[91]  Stefan Kugele,et al.  Optimizing Automatic Deployment Using Non-functional Requirement Annotations , 2008, ISoLA.

[92]  Felipe Ebert,et al.  An Empirical Study on the Usage of the Swift Programming Language , 2016, 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER).

[93]  Chong Feng,et al.  An Empirical Study of Investigating Mobile Applications Development Challenges , 2018, IEEE Access.

[94]  Alberto Bacchelli,et al.  One-Day Flies on StackOverflow - Why the Vast Majority of StackOverflow Users Only Posts Once , 2015, 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.

[95]  Tomi Männistö,et al.  Requirements Engineering as a Driver for Innovations , 2007, 15th IEEE International Requirements Engineering Conference (RE 2007).

[96]  Rachel Harrison,et al.  Retrieving and analyzing mobile apps feature requests from online reviews , 2013, 2013 10th Working Conference on Mining Software Repositories (MSR).

[97]  Chong Feng,et al.  An Empirical Study on How iOS Developers Report Quality Aspects on Stack Overflow , .

[98]  Ahmed E. Hassan,et al.  Studying software evolution using topic models , 2014, Sci. Comput. Program..

[99]  D. Hinkle,et al.  Applied statistics for the behavioral sciences , 1979 .