Communicative modalities for mobile device interaction

An investigation of communicative modalities in relation to mobile device interaction while walking is presented. A user evaluation compared three communicative modality conditions: Auditory, Visual, and Mixed (a redundant audio-visual modality). Findings determined that redundant audio-visual modalities are as good as (but no better than) the visual modality, and both are superior to the auditory modality. Reported findings also determined that walking speeds are unaffected by communicative modality. Shape drawing tasks were performed on a touch screen using each modality, and a robust, novel error calculation algorithm was developed to assess the drawing error between the user input and the desired shapes. Drawing error was determined to be significantly higher with the Auditory condition, but drawing speed was unaffected by the communicative modality. The evaluation finds that the visual modality should be leveraged as the primary communicative modality for mobile, map-based interfaces. The drawing error algorithm can be applied to any domain that requires determining precise matchings to known information when drawing.

[1]  Juan Pablo Hourcade,et al.  Evaluating one handed thumb tapping on mobile touchscreen devices , 2008, Graphics Interface.

[2]  Pietro Perona,et al.  Continuous dynamic time warping for translation-invariant curve alignment with applications to signature verification , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[3]  Christopher D. Wickens,et al.  Attentional Models of Multitask Pilot Performance Using Advanced Display Technology , 2003, Hum. Factors.

[4]  Mark E. Campbell,et al.  A robust sketch interface for natural robot control , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  S. Chiba,et al.  Dynamic programming algorithm optimization for spoken word recognition , 1978 .

[6]  Susanne Boll,et al.  Supporting Mobile Collaboration in Spatially Distributed Workgroups with Digital Interactive Maps , 2012, Mob. Networks Appl..

[7]  Yang Li,et al.  Experimental analysis of touch-screen gesture designs in mobile environments , 2011, CHI.

[8]  Christopher D. Wickens,et al.  Auditory-Visual Redundancy in Vehicle Control Interruptions , 2011 .

[9]  Sean Hayes,et al.  A Comparison of Communicative Modes for Map-Based Tasking , 2011 .

[10]  B. S. Manoj,et al.  Communication challenges in emergency response , 2007, Commun. ACM.

[11]  Jan Stage,et al.  New techniques for usability evaluation of mobile systems , 2004, Int. J. Hum. Comput. Stud..

[12]  J. G. Hollands,et al.  Engineering Psychology and Human Performance , 1984 .

[13]  M. Haselkorn,et al.  Challenges to effective information and communication systems in humanitarian relief organizations , 2005, IPCC 2005. Proceedings. International Professional Communication Conference, 2005..

[14]  Sidney L. Smith,et al.  Guidelines for Designing User Interface Software , 1986 .

[15]  Jessie Y. C. Chen,et al.  Human Performance Issues and User Interface Design for Teleoperated Robots , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[16]  Stephen A. Brewster,et al.  The effects of walking speed on target acquisition on a touchscreen interface , 2011, Mobile HCI.

[17]  Alexander Zipf,et al.  Towards Task Oriented Map-based Mobile Guides , 2003 .

[18]  Andrew Sears,et al.  An empirical comparison of use-in-motion evaluation scenarios for mobile computing devices , 2005, Int. J. Hum. Comput. Stud..

[19]  Daniel J. Wigdor,et al.  Direct-touch vs. mouse input for tabletop displays , 2007, CHI.

[20]  R. Mayer,et al.  Learning Science in Virtual Reality Multimedia Environments: Role of Methods and Media , 2002 .

[21]  Enrico Rukzio,et al.  Investigating selection and reading performance on a mobile phone while walking , 2010, Mobile HCI.

[22]  Jaehyun Park,et al.  Touch key design for target selection on a mobile phone , 2008, Mobile HCI.

[23]  Michael A. Goodrich,et al.  Towards using Unmanned Aerial Vehicles (UAVs) in Wilderness Search and Rescue: Lessons from field trials , 2009 .

[24]  Andrew Sears,et al.  Capturing the effects of context on human performance in mobile computing systems , 2007, Personal and Ubiquitous Computing.

[25]  Christopher D. Wickens,et al.  Redundancy, Modality, and Priority in Dual Task Interference , 2003 .

[26]  Jesse M. Heines,et al.  Selecting While Walking: An Investigation of Aiming Performance in a Mobile Work Context , 2007, AMCIS.

[27]  Donghun Lee,et al.  Effect of key size and activation area on the performance of a regional error correction method in a touch-screen QWERTY keyboard , 2009 .

[28]  Marcos Faúndez-Zanuy,et al.  On-line signature recognition based on VQ-DTW , 2007, Pattern Recognit..

[29]  Niels Henze,et al.  100,000,000 taps: analysis and improvement of touch performance in the large , 2011, Mobile HCI.

[30]  Andrew Sears,et al.  How do people tap when walking? An empirical investigation of nomadic data entry , 2007, Int. J. Hum. Comput. Stud..

[31]  Keith Cheverst,et al.  Developing a context-aware electronic tourist guide: some issues and experiences , 2000, CHI.

[32]  Christopher D. Wickens,et al.  When is Less More? Attention and Workload in Auditory, Visual, and Redundant Patient-Monitoring Conditions , 2001 .

[33]  Jacob O. Wobbrock,et al.  Getting off the treadmill: evaluating walking user interfaces for mobile devices in public spaces , 2008, Mobile HCI.

[34]  J. J. Higgins,et al.  The aligned rank transform for nonparametric factorial analyses using only anova procedures , 2011, CHI.

[35]  Patrick Baudisch,et al.  Precise selection techniques for multi-touch screens , 2006, CHI.

[36]  Patrick Baudisch,et al.  The generalized perceived input point model and how to double touch accuracy by extracting fingerprints , 2010, CHI.

[37]  Mark H. Chignell,et al.  Mobile text entry: relationship between walking speed and text input task difficulty , 2005, Mobile HCI.

[38]  Rhoda Margesson,et al.  Haiti Earthquake: Crisis and Response , 2010 .

[39]  Larry S. Davis,et al.  3-D model-based tracking of humans in action: a multi-view approach , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[40]  Marjorie Skubic,et al.  A sketch interface for mobile robots , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[41]  Abdelsalam Helal,et al.  Decentralized ad-hoc groupware API and framework for mobile collaboration , 2001, GROUP.

[42]  Mohammed Yeasin,et al.  Speech-gesture driven multimodal interfaces for crisis management , 2003, Proc. IEEE.

[43]  Suresh Venkatasubramanian,et al.  Curve Matching, Time Warping, and Light Fields: New Algorithms for Computing Similarity between Curves , 2007, Journal of Mathematical Imaging and Vision.

[44]  Eamonn J. Keogh,et al.  Scaling up dynamic time warping for datamining applications , 2000, KDD '00.

[45]  Pietro Carlo Cacciabue,et al.  A user-centred approach for designing driving support systems: the case of collision avoidance , 2006, Cognition, Technology & Work.

[46]  James J. Jenkins,et al.  Recall of passages of synthetic speech , 1982 .

[47]  Yael Shmueli,et al.  Blending Speech Output and Visual Text in the Multimodal Interface , 2008, Hum. Factors.

[48]  Steve Howard,et al.  Evaluating the usability of a mobile guide: The influence of location, participants and resources , 2005, Behav. Inf. Technol..

[49]  Kori Inkpen Quinn,et al.  Walk 'n scroll: a comparison of software-based navigation techniques for different levels of mobility , 2005, Mobile HCI.

[50]  Sean Hayes,et al.  Visually-Cued Touch Gestures for Accurate Mobile Interaction , 2011 .

[51]  Muzaffar Bashir,et al.  Reduced Dynamic Time Warping for Handwriting Recognition Based on Multidimensional Time Series of a Novel Pen Device , 2008 .

[52]  Kori Inkpen Quinn,et al.  Target selection on mobile devices using display segmentation , 2007, Mobile HCI.

[53]  Magnus Morin,et al.  Visual exploration of communication in command and control , 2002, Proceedings Sixth International Conference on Information Visualisation.

[54]  Lili Yang,et al.  On-Site Information Systems Design for Emergency First Responders , 2009 .

[55]  Robin R. Murphy,et al.  Human-robot interactions during the robot-assisted urban search and rescue response at the World Trade Center , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[56]  Rajinder Koul,et al.  Synthetic Speech Perception in Individuals With and Without Disabilities , 2003, Augmentative and alternative communication.

[57]  Andrew S. Glassner,et al.  Graphics Gems , 1990 .

[58]  Eli R. Hooten,et al.  Comparing input error for mouse and touch input , 2011, 2011 IEEE International Conference on Systems, Man, and Cybernetics.