Engaging Participants during Selection Studies in Virtual Reality

Selection studies are prevalent and indispensable for VR research. However, due to the tedious and repetitive nature of many such experiments, participants can become disengaged during the study, which is likely to impact the results and conclusions. In this work, we investigate participant disengagement in VR selection experiments and how this issue affects the outcomes. Moreover, we evaluate the usefulness of four engagement strategies to keep participants engaged during VR selection studies and investigate how they impact user performance when compared to a baseline condition with no engagement strategy. Based on our findings, we distill several design recommendations that can be useful for future VR selection studies or user tests in other domains that employ similar repetitive features.

[1]  Jorge Gonçalves,et al.  Citizen Motivation on the Go: The Role of Psychological Empowerment , 2014, Interact. Comput..

[2]  Karin Coninx,et al.  Exploring the Effects of Environment Density and Target Visibility on Object Selection in 3D Virtual Environments , 2007, 2007 IEEE Symposium on 3D User Interfaces.

[3]  Alexandre N. Tuch,et al.  Do points, levels and leaderboards harm intrinsic motivation?: an empirical analysis of common gamification elements , 2013, Gamification.

[4]  Géry Casiez,et al.  RayCursor: A 3D Pointing Facilitation Technique based on Raycasting , 2019, CHI.

[5]  Marc Erich Latoschik,et al.  Usability of Gamified Knowledge Learning in VR and Desktop-3D , 2019, CHI.

[6]  Mayra Donaji Barrera Machuca,et al.  The Effect of Stereo Display Deficiencies on Virtual Hand Pointing , 2019, CHI.

[7]  Ken Hinckley,et al.  A survey of design issues in spatial input , 1994, UIST '94.

[8]  Niels Henze,et al.  It Does Not Fitts My Data! Analysing Large Amounts of Mobile Touch Data , 2011, INTERACT.

[9]  Sandra G. Hart,et al.  Nasa-Task Load Index (NASA-TLX); 20 Years Later , 2006 .

[10]  Robert J. Teather,et al.  Pointing at 3D targets in a stereo head-tracked virtual environment , 2011, 2011 IEEE Symposium on 3D User Interfaces (3DUI).

[11]  Regan L. Mandryk,et al.  The Motivational Push of Games: The Interplay of Intrinsic Motivation and External Rewards in Games for Training , 2016, CHI PLAY.

[12]  Jari Takatalo,et al.  Review of Intrinsic Motivation in Simulation-based Game Testing , 2018, CHI.

[13]  Mark Billinghurst,et al.  Pinpointing: Precise Head- and Eye-Based Target Selection for Augmented Reality , 2018, CHI.

[14]  D. Hoang.  FLOW: The Psychology of Optimal Experience , 2018 .

[15]  E. Deci,et al.  Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. , 2000, The American psychologist.

[16]  Jesper Juul,et al.  Good Feedback for bad Players? A preliminary Study of ‘juicy’ Interface feedback , 2016 .

[17]  Paul A. Cairns,et al.  A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form , 2018, Int. J. Hum. Comput. Stud..

[18]  Christof Lutteroth,et al.  Interactive Feedforward for Improving Performance and Maintaining Intrinsic Motivation in VR Exergaming , 2018, CHI.

[19]  Judy Edworthy,et al.  The effects of music tempo and loudness level on treadmill exercise , 2006, Ergonomics.

[20]  Katharina Reinecke,et al.  Pay Attention, Please: Formal Language Improves Attention in Volunteer and Paid Online Experiments , 2019, CHI.

[21]  Arthur C. Graesser,et al.  Posture as a Predictor of Learner's Affective Engagement , 2007 .

[22]  J. J. Higgins,et al.  The aligned rank transform for nonparametric factorial analyses using only anova procedures , 2011, CHI.

[23]  Maria V. Sanchez-Vives,et al.  Influence of Music on Anxiety Induced by Fear of Heights in Virtual Reality , 2016, Front. Psychol..

[24]  Karthik Ramani,et al.  Modeling Cumulative Arm Fatigue in Mid-Air Interaction based on Perceived Exertion and Kinetics of Arm Motion , 2017, CHI.

[25]  David R. Flatla,et al.  Calibration games: making calibration tasks enjoyable by adding motivating game elements , 2011, UIST.

[26]  Barrett Ens,et al.  Modeling endpoint distribution of pointing selection tasks in virtual reality environments , 2019, ACM Trans. Graph..

[27]  Robert J. Teather,et al.  Effects of tracking technology, latency, and spatial jitter on object movement , 2009, 2009 IEEE Symposium on 3D User Interfaces.

[28]  Rita Orji,et al.  Personalizing Persuasive Strategies in Gameful Systems to Gamification User Types , 2018, CHI.

[29]  Lindsay Alexander Shaw,et al.  HIIT With Hits: Using Music and Gameplay to Induce HIIT in Exergames , 2019, ACSW.

[30]  Heinz Mandl,et al.  How gamification motivates: An experimental study of the effects of specific game design elements on psychological need satisfaction , 2017, Comput. Hum. Behav..

[31]  Lu Wang,et al.  Effect of a Virtual Coach on Athletes' Motivation , 2006, PERSUASIVE.

[32]  Greg Welch,et al.  A Systematic Review of Social Presence: Definition, Antecedents, and Implications , 2018, Front. Robot. AI.

[33]  Carol T. Kulik,et al.  Old Friends, New Faces: Motivation Research in the 1990s , 1999 .

[34]  Jacob O. Wobbrock,et al.  The effects of task dimensionality, endpoint deviation, throughput calculation, and experiment design on pointing measures and models , 2011, CHI.

[35]  Tibert Verhagen,et al.  Understanding users' motivations to engage in virtual worlds: A multipurpose model and empirical testing , 2012, Comput. Hum. Behav..

[36]  Andrew Raij,et al.  Exploring micro-incentive strategies for participant compensation in high-burden studies , 2011, UbiComp '11.

[37]  I. Scott MacKenzie,et al.  Towards a standard for pointing device evaluation, perspectives on 27 years of Fitts' law research in HCI , 2004, Int. J. Hum. Comput. Stud..

[38]  Heiko Hecht,et al.  Pleasant music as a countermeasure against visually induced motion sickness. , 2014, Applied ergonomics.

[39]  Kathleen A Martin Ginis,et al.  Music enhances performance and perceived enjoyment of sprint interval exercise. , 2015, Medicine and science in sports and exercise.

[40]  B. J. Fogg,et al.  Computers as persuasive social actors , 2003 .

[41]  Hendrik Knoche,et al.  A Comparison of Gamified HCI Studies with Lab and Crowd Participants , 2017, EAI Endorsed Trans. Creative Technol..

[42]  M. Csíkszentmihályi,et al.  The Concept of Flow , 2014 .

[43]  Jiabin Yuan,et al.  Crossing-Based Selection with Virtual Reality Head-Mounted Displays , 2019, CHI.

[44]  Richard Rouse,et al.  Game design : theory and practice , 2001 .

[45]  Morten Fjeld,et al.  Flow is Not Enough: Understanding the Needs of Advanced Amateur Runners to Design Motivation Technology , 2015, CHI.

[46]  Carl Gutwin,et al.  Anchoring Effects and Troublesome Asymmetric Transfer in Subjective Ratings , 2019, CHI.

[47]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[48]  Lennart E. Nacke,et al.  Gamification : Toward a Definition , 2022 .

[49]  Janet C. Read,et al.  FittsFarm: Comparing Children's Drag-and-Drop Performance Using Finger and Stylus Input on Tablets , 2019, INTERACT.

[50]  Pawel W. Wozniak,et al.  Gamification in Science: A Study of Requirements in the Context of Reproducible Research , 2019, CHI.

[51]  Michael Christian,et al.  WORK ENGAGEMENT: A QUANTITATIVE REVIEW AND TEST OF ITS RELATIONS WITH TASK AND CONTEXTUAL PERFORMANCE , 2011 .

[52]  Juho Hamari,et al.  Player Types: A Meta-synthesis , 2014, Trans. Digit. Games Res. Assoc..

[53]  Niels Henze,et al.  Using Presence Questionnaires in Virtual Reality , 2019, CHI.

[54]  E. Vesterinen,et al.  Affective Computing , 2009, Encyclopedia of Biometrics.

[55]  E. Deci,et al.  The "What" and "Why" of Goal Pursuits: Human Needs and the Self-Determination of Behavior , 2000 .

[56]  Rita Orji,et al.  Persuasive technology for health and wellness: State-of-the-art and emerging trends , 2018, Health Informatics J..

[57]  Tianyu Zhang,et al.  DepthMove: Leveraging Head Motions in the Depth Dimension to Interact with Virtual Reality Head-Worn Displays , 2019, 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[58]  Tovi Grossman,et al.  Multimodal selection techniques for dense and occluded 3D virtual environments , 2009, Int. J. Hum. Comput. Stud..

[59]  Harri Oinas-Kukkonen,et al.  Persuasive Systems Design: Key Issues, Process Model, and System Features , 2009, Commun. Assoc. Inf. Syst..

[60]  Graham Stew,et al.  The phenomenon of boredom , 2006 .

[61]  Ivan Poupyrev,et al.  3D User Interfaces: Theory and Practice , 2004 .

[62]  Jorge Gonçalves,et al.  Gamification of Mobile Experience Sampling Improves Data Quality and Quantity , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[63]  J. Bailenson,et al.  Social interaction in augmented reality , 2019, PloS one.

[64]  Kathrin Gerling,et al.  Understanding the Effects of Gamification and Juiciness on Players , 2019, 2019 IEEE Conference on Games (CoG).

[65]  M. Csíkszentmihályi.  Flow: The Psychology of Optimal Experience , 1990 .

[66]  Bin Xu,et al.  Personality-targeted Gamification: A Survey Study on Personality Traits and Motivational Affordances , 2016, CHI.

[67]  D. Cox,et al.  An Analysis of Transformations , 1964 .

[68]  Jussi Holopainen,et al.  Good Game Feel: An Empirically Grounded Framework for Juicy Design , 2018, DiGRA Conference.

[69]  Deborah I. Fels,et al.  Gamification in theory and action: A survey , 2015, Int. J. Hum. Comput. Stud..

[70]  Thomas W. Malone,et al.  Toward a Theory of Intrinsically Motivating Instruction , 1981, Cogn. Sci..

[71]  Jorge Gonçalves,et al.  Effect of Distinct Ambient Noise Types on Mobile Interaction , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[72]  Robert J. Teather,et al.  EZCursorVR: 2D Selection with Virtual Reality Head-Mounted Displays , 2018, Graphics Interface.

[73]  P. Fitts.  The information capacity of the human motor system in controlling the amplitude of movement. , 1954, Journal of experimental psychology.

[74]  E. Deci,et al.  Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions. , 2000, Contemporary educational psychology.

[75]  Robert J. Teather,et al.  Visual aids in 3D point selection experiments , 2014, SUI.

[76]  L. Svendsen,et al.  Philosophy of Boredom , 2021, Philosophy.

[77]  Ferran Argelaguet,et al.  A survey of 3D object selection techniques for virtual environments , 2013, Comput. Graph..

[78]  I. Scott MacKenzie,et al.  Pointing by gaze, head, and foot in a head-mounted display , 2019, ETRA.

[79]  Robert J. Teather,et al.  An arm-mounted inertial controller for 6DOF input: Design and evaluation , 2017, 2017 IEEE Symposium on 3D User Interfaces (3DUI).

[80]  Juho Hamari,et al.  Does Gamification Work? -- A Literature Review of Empirical Studies on Gamification , 2014, 2014 47th Hawaii International Conference on System Sciences.

[81]  I. Scott MacKenzie,et al.  Fitts' throughput and the speed-accuracy tradeoff , 2008, CHI.

[82]  Evangelos Karapanos,et al.  23 Ways to Nudge: A Review of Technology-Mediated Nudging in Human-Computer Interaction , 2019, CHI.

[83]  Jorge Gonçalves,et al.  Game of words: tagging places through crowdsourcing on public displays , 2014, Conference on Designing Interactive Systems.