An empirical comparison of use-in-motion evaluation scenarios for mobile computing devices

There is a clear need for evaluation methods that are specifically suited to mobile device evaluation, largely due to the vast differences between traditional desktop computing and mobile computing. One difference of particular interest that needs to be accounted for is that mobile computing devices are frequently used while the user is in motion, in contrast to desktop computing. This study aims to validate the appropriateness of two evaluation methods that vary in representativeness of mobility, one that uses a treadmill to simulate motion and another that uses a controlled walking scenario.The results lead to preliminary guidelines based on study objectives for researchers wishing to use more appropriate evaluation methodologies for empirical, data-driven mobile computing studies. The guidelines indicate that using a treadmill for mobile evaluation can yield representative performance measures,whereas a controlled walking scenario is more likely to adequately simulate the atual user experience.

[1]  Johanna D. Moore,et al.  Proceedings of the Conference on Human Factors in Computing Systems , 1989 .

[2]  Connor Graham,et al.  A Review of Mobile HCI Research Methods , 2003, Mobile HCI.

[3]  Aftab E. Patla,et al.  Review article Understanding the roles of vision in the control of human locomotion , 1997 .

[4]  John Annett,et al.  Conceptual Foundations of Human Factors Measurement , 2007 .

[5]  Stephen A. Brewster,et al.  Overcoming the Lack of Screen Space on Mobile Computers , 2002, Personal and Ubiquitous Computing.

[6]  G. L. Pellecchia,et al.  Postural sway increases with attentional demands of concurrent cognitive task. , 2003, Gait & posture.

[7]  L Vogt,et al.  Comparison of angular lumbar spine and pelvis kinematics during treadmill and overground locomotion. , 2002, Clinical biomechanics.

[8]  Allen Newell,et al.  The psychology of human-computer interaction , 1983 .

[9]  Jan Stage,et al.  New techniques for usability evaluation of mobile systems , 2004, Int. J. Hum. Comput. Stud..

[10]  B. Nigg,et al.  A kinematic comparison of overground and treadmill running. , 1995, Medicine and science in sports and exercise.

[11]  Stephen A. Brewster,et al.  Gestural and audio metaphors as a means of control for mobile devices , 2002, CHI.

[12]  P. Hancock,et al.  Human Mental Workload , 1988 .

[13]  Parag C. Pendharkar,et al.  Human-computer interaction issues for mobile computing in a variable work context , 2004, Int. J. Hum. Comput. Stud..

[14]  Jukka Häkkinen,et al.  Examining mobile phone text legibility while walking , 2004, CHI EA '04.

[15]  F. Alton,et al.  A kinematic comparison of overground and treadmill walking. , 1998, Clinical biomechanics.

[16]  Chris Baber,et al.  Defining and evaluating context for wearable computing , 2004, Int. J. Hum. Comput. Stud..

[17]  Andrew Sears,et al.  Capturing the effects of context on human performance in mobile computing systems , 2007, Personal and Ubiquitous Computing.

[18]  B H Kantowitz,et al.  Selecting Measures for Human Factors Research , 1992, Human factors.

[19]  A. Schache,et al.  A comparison of overground and treadmill running for measuring the three-dimensional kinematics of the lumbo-pelvic-hip complex. , 2001, Clinical biomechanics.

[20]  Eric Horvitz,et al.  Sensing techniques for mobile interaction , 2000, UIST '00.

[21]  Murray Mp,et al.  Treadmill vs. floor walking: kinematics, electromyogram, and heart rate. , 1985, Journal of applied physiology.

[22]  Albrecht Schmidt,et al.  Advanced Interaction in Context , 1999, HUC.

[23]  Steinar Kristoffersen,et al.  Designing Interaction Styles for a Mobile Use Context , 1999, HUC.

[24]  Tapio Seppänen,et al.  Adapting applications in handheld devices using fuzzy context information , 2003, Interact. Comput..

[25]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[26]  Antti Oulasvirta,et al.  Finding meaningful uses for context-aware technologies: the humanistic research strategy , 2004, CHI.

[27]  Constantine Stephanidis,et al.  Human-computer interaction : theory and practice , 2003 .

[28]  David R. Morse,et al.  Using while moving: HCI issues in fieldwork environments , 2000, TCHI.

[29]  Albrecht Schmidt,et al.  There is more to context than location , 1999, Comput. Graph..

[30]  Andy H. Lee,et al.  Assessing the driving performance of older adult drivers: on-road versus simulated driving. , 2003, Accident; analysis and prevention.

[31]  W Poewe,et al.  Influence of Concurrent Tasks on Gait: A Dual-Task Approach , 1995, Perceptual and motor skills.

[32]  Jesper Kjeldskov,et al.  Is It Worth the Hassle? Exploring the Added Value of Evaluating the Usability of Context-Aware Mobile Systems in the Field , 2004, Mobile HCI.

[33]  C. Bard,et al.  Attentional demands for static and dynamic equilibrium , 2004, Experimental Brain Research.

[34]  Mark D. Dunlop,et al.  The Challenge of Mobile Devices for Human Computer Interaction , 2002, Personal and Ubiquitous Computing.