Using Qualitative Data Analysis to Measure User Experience in a Serious Game for Premed Students

The University of Texas Transformation in Medical Education Portal (UT TIME Portal) is a game-based learning platform for select premed students, with a particular emphasis on communication and professionalism. In addition to quantitative data on system usage and user performance, the UT TIME Portal generates rich sets of qualitative data collected through discussion board posts and pre- and post- surveys. Using NVivo 10’s built-in tools, our team used this qualitative data to measure game experience outcomes in many ways by building and testing out hypotheses about our user experience design. The ability to tag, code and organize themes to then be analyzed in the context of quantitative data generated by the UT TIME Portal adds an important dimension to understanding the user experience and generates insights not possible to glean from quantitative data alone.

[1]  Henny van der Meijden,et al.  Effects of constructing versus playing an educational game on student motivation and deep learning strategy use , 2011, Comput. Educ..

[2]  D. Sui,et al.  Crossing the qualitative- quantitative divide II , 2013 .

[3]  Amit P. Sheth,et al.  Harnessing Twitter "Big Data" for Automatic Emotion Identification , 2012, 2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Confernece on Social Computing.

[4]  Molly Cooke,et al.  American medical education 100 years after the Flexner report. , 2006, The New England journal of medicine.

[5]  Traci Sitzmann.  A META-ANALYTIC EXAMINATION OF THE INSTRUCTIONAL EFFECTIVENESS OF COMPUTER-BASED SIMULATION GAMES , 2011 .

[6]  T. B. Rogers,et al.  Self-reference and the encoding of personal information. , 1977, Journal of personality and social psychology.

[7]  P. Shaver,et al.  Emotion knowledge: further exploration of a prototype approach. , 1987, Journal of personality and social psychology.

[8]  Megan Woods,et al.  Advancing Qualitative Research Using Qualitative Data Analysis Software (QDAS)? Reviewing Potential Versus Practice in Published Studies using ATLAS.ti and NVivo, 1994–2013 , 2016 .

[9]  Daniel Jacob,et al.  Beyond fun and games: Toward an adaptive and emergent learning platform for pre-med students with the UT TIME Portal , 2016, 2016 IEEE International Conference on Serious Games and Applications for Health (SeGAH).

[10]  Fengfeng Ke,et al.  A Qualitative Meta-Analysis of Computer Games as Learning Tools , 2009 .

[11]  Charu C. Aggarwal,et al.  Social Network Data Analytics , 2011 .

[12]  Nancy L. Leech,et al.  Beyond Constant Comparison Qualitative Data Analysis: Using NVivo. , 2011 .

[13]  Aaron Smith,et al.  U.S. Smartphone Use in 2015 , 2015 .

[14]  Gary Bente,et al.  Why so serious? On the relation of serious games and learning , 2010, Eludamos: Journal for Computer Game Culture.

[15]  James M. Boyle,et al.  A systematic literature review of empirical evidence on computer games and serious games , 2012, Comput. Educ..

[16]  Alan E. Kazdin,et al.  Research Design in Clinical Psychology , 2021 .

[17]  Marc Berg,et al.  Meeting patients' needs with patient information systems: potential benefits of qualitative research methods , 2001, Int. J. Medical Informatics.

[18]  Murtaza Haider,et al.  Beyond the hype: Big data concepts, methods, and analytics , 2015, Int. J. Inf. Manag..

[19]  Marija Dalbello,et al.  A genealogy of digital humanities , 2011, J. Documentation.

[20]  L Leape,et al.  Transforming healthcare: a safety imperative , 2009, Quality and Safety in Health Care.