Applying the Cumulative Fatigue Model to Interaction on Large, Multi-Touch Displays

Large touch displays have long been studied in the lab, and are beginning to see widespread deployment in public spaces. However, a common limitation is fatigue -- often called 'gorilla arm' -- that prevents users from working with large displays for extended periods of time. A first step towards addressing fatigue is quantifying it, and while methods have been developed to quantify users' fatigue in mid-air interactions, there remains little understanding of fatigue on touch-based interfaces. To address this gap, we evaluated the accuracy of Jang et al.'s mid-air Cumulative Fatigue model for touch interaction tasks on a large display. We found that their model underestimates subjective fatigue for multi-touch interaction, but can provide accurate estimates through fine-tuning of model parameters. We discuss the implications of this finding, and the need to further develop tools to evaluate fatigue on large, multi-touch displays.

[1]  Marko Jurmu,et al.  Multipurpose Interactive Public Displays in the Wild: Three Years Later , 2012, Computer.

[2]  R. Enoka,et al.  Muscle Fatigue and the Mechanisms of Task Failure , 2004, Exercise and sport sciences reviews.

[3]  M. Sheelagh T. Carpendale,et al.  Sticky tools: full 6DOF force-based interaction for multi-touch tables , 2009, ITS '09.

[4]  W. Rohmert Ermittlung von Erholungspausen für statische Arbeit des Menschen , 1960, Internationale Zeitschrift für angewandte Physiologie einschließlich Arbeitsphysiologie.

[5]  Fouad Bennis,et al.  A new simple dynamic muscle fatigue model and its validation , 2022, ArXiv.

[6]  Edward Lank,et al.  Watchpoint: Freehand Pointing with a Smartwatch in a Ubiquitous Display Environment , 2016, AVI.

[7]  Olivier Chapuis,et al.  Mid-air pan-and-zoom on wall-sized displays , 2011, CHI.

[8]  Wendy Yee Potential Limitations of Multi-touch Gesture Vocabulary: Differentiation, Adoption, Fatigue , 2009, HCI.

[9]  Carmelo Ardito,et al.  Interaction with Large Displays , 2015, ACM Comput. Surv..

[10]  Gisela Sjøgaard,et al.  Muscle blood flow during isometric activity and its relation to muscle fatigue , 2004, European Journal of Applied Physiology and Occupational Physiology.

[11]  Alessio Malizia,et al.  Don't touch me: multi-user annotations on a map in large display environments , 2010, AVI.

[12]  Antti Oulasvirta,et al.  Performance and Ergonomics of Touch Surfaces: A Comparative Study using Biomechanical Simulation , 2015, CHI.

[13]  M. Sheelagh T. Carpendale,et al.  Collaborative coupling over tabletop displays , 2006, CHI.

[14]  Jing Z. Liu,et al.  A dynamical model of muscle activation, fatigue, and recovery. , 2001, Biophysical journal.

[15]  Svend Erik Mathiassen,et al.  Prediction of shoulder flexion endurance from personal factors , 1999 .

[16]  E. Heath Borg's Perceived Exertion and Pain Scales , 1998 .

[17]  Marko Jurmu,et al.  UBI-Hotspot 1.0: Large-Scale Long-Term Deployment of Interactive Public Displays in a City Center , 2010, 2010 Fifth International Conference on Internet and Web Applications and Services.

[18]  Ken Hinckley,et al.  A survey of design issues in spatial input , 1994, UIST '94.

[19]  Antti Oulasvirta,et al.  It's Mine, Don't Touch!: interactions at a large multi-touch display in a city centre , 2008, CHI.

[20]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[21]  Fouad Bennis,et al.  Human Muscle Fatigue Model in Dynamic Motions , 2012, ARK.

[22]  M. Sheelagh T. Carpendale,et al.  Territoriality in collaborative tabletop workspaces , 2004, CSCW.

[23]  R Kadefors,et al.  Subjective and objective evaluation of shoulder muscle fatigue. , 1994, Ergonomics.

[24]  S. Segerstrom,et al.  Heart Rate Variability Reflects Self-Regulatory Strength, Effort, and Fatigue , 2007, Psychological science.

[25]  Feng Wang,et al.  Empirical evaluation for finger input properties in multi-touch interaction , 2009, CHI.

[26]  M. Finlayson,et al.  An evaluation of subjective and objective measures of fatigue in patients with brain injury and healthy controls. , 1998, Brain injury.

[27]  Marko Jurmu,et al.  Scroll, tilt or move it: using mobile phones to continuously control pointers on large public displays , 2009, OZCHI.

[28]  Bengt Ahlström,et al.  Overcoming touchscreen user fatigue by workplace design , 1992, CHI '92.

[29]  K. Avin,et al.  Endurance time is joint-specific: A modelling and meta-analysis investigation , 2010, Ergonomics.

[30]  Karthik Ramani,et al.  Modeling Cumulative Arm Fatigue in Mid-Air Interaction based on Perceived Exertion and Kinetics of Arm Motion , 2017, CHI.

[31]  Mario Cifrek,et al.  Surface EMG based muscle fatigue evaluation in biomechanics. , 2009, Clinical biomechanics.

[32]  John Seely Brown,et al.  The Origins of Ubiquitous Computing Research at PARC in the Late 1980s , 1999, IBM Syst. J..

[33]  Chris North,et al.  Move to improve: promoting physical navigation to increase user performance with large displays , 2007, CHI.

[34]  Chris Harrison,et al.  On-body interaction: armed and dangerous , 2012, TEI.

[35]  A. Kendon Gesture: Visible Action as Utterance , 2004 .

[36]  L. F. Frey Law,et al.  A theoretical approach for modeling peripheral muscle fatigue and recovery. , 2008, Journal of biomechanics.

[37]  Jefferson Y. Han Low-cost multi-touch sensing through frustrated total internal reflection , 2005, UIST.

[38]  Daniel Vogel,et al.  Soft-Constraints to Reduce Legacy and Performance Bias to Elicit Whole-body Gestures with Low Arm Fatigue , 2015, CHI.

[39]  Roberto Merletti,et al.  Assessment of force and fatigue in isometric contractions of the upper trapezius muscle by surface EMG signal and perceived exertion scale. , 2008, Gait & posture.

[40]  Kori Inkpen Quinn,et al.  Investigating teamwork and taskwork in single- and multi-display groupware systems , 2009, Personal and Ubiquitous Computing.

[41]  Pedro Lopes,et al.  Let's kick it: how to stop wasting the bottom third of your large screen display , 2014, CHI.

[42]  Steven K. Feiner,et al.  Rubbing and tapping for precise and rapid selection on touch-screen displays , 2008, CHI.

[43]  G Németh,et al.  Correlation between electromyographic spectral changes and subjective assessment of lumbar muscle fatigue in subjects without pain from the lower back. , 1999, Clinical biomechanics.

[44]  Ken Perlin,et al.  Mechanical force redistribution: enabling seamless, large-format, high-accuracy surface interaction , 2014, CHI.

[45]  John C. Tang,et al.  Liveboard: a large interactive display supporting group meetings, presentations, and remote collaboration , 1992, CHI.

[46]  Jonathan Hook,et al.  Comparing Fatigue When Using Large Horizontal and Vertical Multi-touch Interaction Displays , 2015, INTERACT.

[47]  Edward Lank,et al.  Tiltcasting: 3D Interaction on Large Displays using a Mobile Device , 2015, UIST.

[48]  Dennis Wixon,et al.  The Natural User Interface , 2011 .

[49]  Daniel Vogel,et al.  Distant freehand pointing and clicking on very large, high resolution displays , 2005, UIST.

[50]  Jun Rekimoto,et al.  HoloWall: designing a finger, hand, body, and object sensitive wall , 1997, UIST '97.

[51]  Pourang Irani,et al.  Consumed endurance: a metric to quantify arm fatigue of mid-air interactions , 2014, CHI.