The past, present, and future of UX empirical research

Rethinking UX requires mapping trends in empirical research to find out how the field has developed. This study addresses that need by analyzing over 400 academic empirical studies published between 2000--2016. Our research questions are, "How have the artifacts, analysis, and methods of UX research changed since the year 2000?" and "Do scholars use research questions and hypotheses to ground their research in UX?" Our research found that services, websites, and imagined objects/prototypes were among the most frequently studied artifacts, while usability studies, surveys, and interviews were the most commonly used methods. We found a significant increase in quantitative and mixed methods studies since 2010. This study showed that only 1 out of every 5 publications employed research questions to guide inquiry. We hope that these findings help UX as a field more accurately and broadly conceive of its identity with clear standards for evaluating existing research and rethinking future research opportunities as a discipline.

[1]  Jeff Sauro,et al.  The challenges and opportunities of measuring the user experience , 2016 .

[2]  Nuno Jardim Nunes,et al.  The state of user experience evaluation practice , 2014, NordiCHI.

[3]  A. Twycross Research design: qualitative, quantitative and mixed methods approaches Research design: qualitative, quantitative and mixed methods approaches Creswell John W Sage 320 £29 0761924426 0761924426 [Formula: see text]. , 2004, Nurse researcher.

[4]  Carolyn D. Rude,et al.  Mapping the Research Questions in Technical Communication , 2009 .

[5]  Virpi Roto,et al.  User Experience Evaluation - Which Method to Choose? , 2011, INTERACT.

[6]  Ann Light,et al.  Designing Connected Products: UX for the Consumer Internet of Things , 2015 .

[7]  Valeria Herskovic,et al.  Measuring User Experience in Latin America: An Exploratory Survey , 2015, CLIHC.

[8]  Kelli Cargile Cook,et al.  Failing Forward , 2017 .

[9]  Thed N. van Leeuwen,et al.  Using Google Scholar in research evaluation of humanities and social science programs: A comparison with Web of Science data , 2016 .

[10]  Hiroyuki Miki Reconsidering the Notion of User Experience for Human-Centered Design , 2013, HCI.

[11]  M. Bhandari,et al.  Research questions, hypotheses and objectives , 2010 .

[12]  Murni Mahmud,et al.  A survey of user experience practice: A point of meet between academic and industry , 2014, 2014 3rd International Conference on User Science and Engineering (i-USEr).

[14]  Emilio Delgado López-Cózar,et al.  Can we use Google Scholar to identify highly-cited documents? , 2017, J. Informetrics.

[15]  Vincent Koenig,et al.  User experience: A concept without consensus? Exploring practitioners' perspectives through an international survey , 2015, Comput. Hum. Behav..

[16]  Bong Gwan Jun,et al.  My Smart TV Agent: Designing Smart TV Persona for Linguistic UX , 2014, HCI.

[17]  Nigel Bevan,et al.  What is the difference between the purpose of usability and user experience evaluation methods , 2009 .

[18]  Kasper Hornbæk,et al.  Foci and blind spots in user experience research , 2012, INTR.

[19]  Gwan-Su Yi,et al.  Finding type 2 diabetes causal single nucleotide polymorphism combinations and functional modules from genome-wide association data , 2013, BMC Medical Informatics and Decision Making.

[20]  Francisco Rebelo,et al.  Using Virtual Reality to Assess User Experience , 2012, Hum. Factors.

[21]  Lim Tek Yong User experience evaluation methods for mobile devices , 2013, Third International Conference on Innovative Computing Technology (INTECH 2013).

[22]  Ann M. Blakeslee,et al.  The State of Research in Technical Communication , 2004 .

[23]  Paul van Schaik,et al.  Attitudes towards user experience (UX) measurement , 2014, Int. J. Hum. Comput. Stud..

[24]  Marc Hassenzahl,et al.  User experience - a research agenda , 2006, Behav. Inf. Technol..

[25]  Marc Hassenzahl,et al.  User experience (UX): towards an experiential perspective on product quality , 2008, IHM '08.

[26]  Alice Gross,et al.  Why do I like it?: investigating the product-specificity of user experience , 2012, NordiCHI.

[27]  Effie Lai-Chong Law,et al.  The measurability and predictability of user experience , 2011, EICS '11.

[28]  Stéfan Jacques Darmoni,et al.  Is the coverage of google scholar enough to be used alone for systematic reviews , 2013, BMC Medical Informatics and Decision Making.

[29]  Martin Tomitsch,et al.  Back to the Future: Identifying Interface Trends from the Past, Present and Future in Immersive Applications , 2015, OZCHI.

[30]  K. Perreault,et al.  Research Design: Qualitative, Quantitative, and Mixed Methods Approaches , 2011 .

[31]  Larry L. Constantine,et al.  Back to the future , 2001, CACM.

[32]  Tatiana Batova,et al.  A Systematic Literature Review of Changes in Roles/Skills in Component Content Management Environments and Implications for Education , 2017 .

[33]  Virpi Roto,et al.  User experience evaluation methods: current state and development needs , 2010, NordiCHI.

[34]  Helen L. Partridge,et al.  Information experience: new perspectives and research directions , 2014 .

[35]  Karel Vredenburg,et al.  A survey of user-centered design practice , 2002, CHI.

[36]  Anne-Wil Harzing,et al.  A longitudinal study of Google Scholar coverage between 2012 and 2013 , 2013, Scientometrics.

[37]  Gaby Haddow Improving research communication and its application in practice , 2006 .

[38]  Madian Khabsa,et al.  Digital commons , 2020, Internet Policy Rev..

[39]  Marc Pallot,et al.  User Experience: Buzzword or New Paradigm? , 2012 .

[40]  Virpi Roto User Experience Evaluation in Nokia , 2008 .

[41]  Kasper Hornbæk,et al.  Old wine in new bottles or novel challenges: a critical analysis of empirical studies of user experience , 2011, CHI.

[42]  M. Ishtiaq Book Review Creswell, J. W. (2014). Research Design: Qualitative, Quantitative and Mixed Methods Approaches (4th ed.). Thousand Oaks, CA: Sage , 2019, English Language Teaching.

[43]  Kristi Wiley,et al.  Review of "The user experience team of one: A research and design survival guide by L. Buley" Rosenfeld Media 2013 978-1-933820-18-7. , 2015, CDQR.

[44]  Virpi Roto,et al.  Understanding, scoping and defining user experience: a survey approach , 2009, CHI.

[45]  Rex Hartson,et al.  The UX book, process and guidelines for ensuring a quality user experience by Rex Hartson and Pardha S. Pyla , 2012, SOEN.