Which Feature is Unusable? Detecting Usability and User Experience Issues from User Reviews

Usability and user experience (UUX) strongly affect software quality and success. User reviews allow software users to report UUX issues. However, this information can be difficult to access due to the varying quality of the reviews, its large numbers and unstructured nature. In this work we propose an approach to automatically detect the UUX strengths and issues of software features according to user reviews. We use a collocation algorithm for extracting the features, lexical sentiment analysis for uncovering users' satisfaction about a particular feature and machine learning for detecting the specific UUX issues affecting the software application. Additionally, we present two visualizations of the results. An initial evaluation of the approach against human judgement obtained mixed results.

[1]  Kurt Schneider,et al.  Feedback in Context: Supporting the Evolution of IT-Ecosystems , 2010, PROFES.

[2]  Nigel Bevan,et al.  Classifying and selecting UX and usability measures , 2008 .

[3]  Kasper Hornbæk,et al.  Old wine in new bottles or novel challenges: a critical analysis of empirical studies of user experience , 2011, CHI.

[4]  Ben Shneiderman,et al.  The eyes have it: a task by data type taxonomy for information visualizations , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[5]  Fabrizio Sebastiani,et al.  Machine learning in automated text categorization , 2001, CSUR.

[6]  Paul A. Pavlou,et al.  Can online reviews reveal a product's true quality?: empirical findings and analytical modeling of Online word-of-mouth communication , 2006, EC '06.

[7]  Kyo Chul Kang,et al.  Feature-Oriented Domain Analysis (FODA) Feasibility Study , 1990 .

[8]  Tung Thanh Nguyen,et al.  Mining User Opinions in Mobile App Reviews: A Keyword-Based Approach (T) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[9]  Xiaodong Gu,et al.  "What Parts of Your Apps are Loved by Users?" (T) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[10]  Bernd Brügge,et al.  Retrieving Diverse Opinions from App Reviews , 2015, 2015 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM).

[11]  Walid Maalej,et al.  User feedback in the appstore: An empirical study , 2013, 2013 21st IEEE International Requirements Engineering Conference (RE).

[12]  Harald C. Gall,et al.  What would users change in my app? summarizing app reviews for recommending software changes , 2016, SIGSOFT FSE.

[13]  Rajesh Vasa,et al.  An Analysis of the Mobile App Review Landscape: Trends and Implications , 2013 .

[14]  Harald C. Gall,et al.  How can i improve my app? Classifying user reviews for software maintenance and evolution , 2015, 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[15]  Anders Bruun,et al.  Is usability evaluation important: the perspective of novice software developers , 2013, BCS HCI.

[16]  Walid Maalej,et al.  Bug report, feature request, or simply praise? On automatically classifying app reviews , 2015, 2015 IEEE 23rd International Requirements Engineering Conference (RE).

[17]  Mike Thelwall,et al.  Sentiment in short strength detection informal text , 2010 .

[18]  Rachel Harrison,et al.  Retrieving and analyzing mobile apps feature requests from online reviews , 2013, 2013 10th Working Conference on Mining Software Repositories (MSR).

[19]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[20]  Bernd Bruegge,et al.  Ensemble Methods for App Review Classification: An Approach for Software Evolution (N) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[21]  Walid Maalej,et al.  How Do Users Like This Feature? A Fine Grained Sentiment Analysis of App Reviews , 2014, 2014 IEEE 22nd International Requirements Engineering Conference (RE).

[22]  Walid Maalej,et al.  Democratic mass participation of users in Requirements Engineering? , 2015, 2015 IEEE 23rd International Requirements Engineering Conference (RE).

[23]  Norbert Seyff,et al.  AppEcho: a user-driven, in situ feedback approach for mobile platforms and applications , 2014, MOBILESoft 2014.

[24]  Eduard C. Groen,et al.  Towards Crowd-Based Requirements Engineering A Research Preview , 2015, REFSQ.

[25]  Berkant Barla Cambazoglu,et al.  A large-scale sentiment analysis for Yahoo! answers , 2012, WSDM '12.

[26]  Jurek Kirakowski,et al.  What is Usability , 1991 .

[27]  Leif Singer,et al.  Using popular social network sites to support requirements elicitation, prioritization and negotiation , 2015, Journal of Internet Services and Applications.

[28]  Bernd Brügge,et al.  FAVe: Visualizing User Feedback for Software Evolution , 2014, 2014 Second IEEE Working Conference on Software Visualization.

[29]  Gabriele Bavota,et al.  Release Planning of Mobile Apps Based on User Reviews , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[30]  Walid Maalej,et al.  On the Socialness of Software , 2011, 2011 IEEE Ninth International Conference on Dependable, Autonomic and Secure Computing.

[31]  Ning Chen,et al.  AR-miner: mining informative reviews for developers from mobile app marketplace , 2014, ICSE.

[32]  Jakob Grue Simonsen,et al.  Extracting usability and user experience information from online user reviews , 2013, CHI.

[33]  Kristina Winbladh,et al.  Analysis of user comments: An approach for software requirements evolution , 2013, 2013 35th International Conference on Software Engineering (ICSE).