Twisting touch: combining deformation and touch as input within the same interaction cycle on handheld devices

We present a study that investigates the potential of combining, within the same interaction cycle, deformation and touch input in a handheld device. Using a flexible, input-only device connected to an external display, we compared a multitouch input technique and two hybrid deformation-plus-touch input techniques (bending and twisting the device, plus either front- or back-touch), in an image-docking task. We compared and analyzed the performance (completion time) and user experience (UX) obtained in each case, using multiple assessment metrics. We found that combining device deformation with front-touch produced the best UX. All the interaction techniques showed the same efficiency in task completion. This was a surprising finding, since multitouch (an integral input technique) was expected to be the most efficient technique in an image docking task (an interaction in an integral perceptual space). We discuss these findings in relation to self-reported qualitative data and observed interaction-procedure metrics. We found that the interaction procedures with the hybrid techniques were more sequential but also more paced. These findings suggest that the benefits of deformation input can still be observed when deformation and touch are combined in an input device.

[1]  Wonkyum Lee,et al.  FlexRemote: Exploring the Effectiveness of Deformable User Interface as an Input Device for TV , 2011, HCI.

[2]  Jun-ichiro Watanabe,et al.  Bookisheet: bendable device for browsing content using the metaphor of leafing through the pages , 2008, UbiComp.

[3]  Marc Hassenzahl,et al.  The Interplay of Beauty, Goodness, and Usability in Interactive Products , 2004, Hum. Comput. Interact..

[4]  Roel Vertegaal,et al.  Towards more paper-like input: flexible input devices for foldable interaction styles , 2008, UIST '08.

[5]  Roel Vertegaal,et al.  PaperPhone: understanding the use of bend gestures in mobile devices with flexible electronic paper displays , 2011, CHI.

[6]  Ken Hinckley,et al.  Sensor synaesthesia: touch in motion, and motion in touch , 2011, CHI.

[7]  Patrick Baudisch,et al.  Touch input on curved surfaces , 2011, CHI.

[8]  Xu Jia,et al.  How users manipulate deformable displays as input devices , 2010, CHI.

[9]  Patrick Baudisch,et al.  Separability of spatial manipulations in multi-touch interfaces , 2009, Graphics Interface.

[10]  Max Mühlhäuser,et al.  Xpaaand: interaction techniques for rollable displays , 2011, CHI.

[11]  Zi Ye,et al.  Cobra: flexible displays for mobilegaming scenarios , 2010, CHI EA '10.

[12]  Susanna Paasovaara,et al.  Kinetic device: designing interactions with a deformable mobile interface , 2012, CHI Extended Abstracts.

[13]  Hiroshi Ishii,et al.  Tangible bits: towards seamless interfaces between people, bits and atoms , 1997, CHI.

[14]  Roel Vertegaal,et al.  FlexView: an evaluation of depth navigation on deformable mobile devices , 2013, TEI '13.

[15]  Roel Vertegaal,et al.  BendFlip: Examining Input Techniques for Electronic Book Readers with Flexible Form Factors , 2011, INTERACT.

[16]  Sandra G. Hart,et al.  Nasa-Task Load Index (NASA-TLX); 20 Years Later , 2006 .

[17]  Katrin Wolf,et al.  PinchPad: performance of touch-based gestures while grasping devices , 2012, TEI.

[18]  Jürgen Steimle,et al.  FoldMe: interacting with double-sided foldable displays , 2012, Tangible and Embedded Interaction.

[19]  Patrick Baudisch,et al.  Back-of-device interaction allows creating very small touch devices , 2009, CHI.

[20]  Yanqing Wang,et al.  The structure of object transportation and orientation in human-computer interaction , 1998, CHI.

[21]  Ivan Poupyrev,et al.  Gummi: a bendable computer , 2004, CHI '04.

[22]  Dominik Schmidt,et al.  A Comparison of Direct and Indirect Multi-touch Input for Large Surfaces , 2009, INTERACT.

[23]  George W. Fitzmaurice,et al.  Exploring interactive curve and surface manipulation using a bend and twist sensitive input strip , 1999, SI3D.

[24]  HolmanDavid,et al.  Organic user interfaces , 2008 .

[25]  Jan O. Borchers,et al.  Twend: twisting and bending as new interaction gesture in mobile devices , 2008, CHI Extended Abstracts.

[26]  Tomoko Yonezawa,et al.  Intuitive page-turning interface of e-books on flexible e-paper based on user studies , 2008, ACM Multimedia.

[27]  Ravin Balakrishnan,et al.  Zliding: fluid zooming and sliding for high precision parameter manipulation , 2005, UIST.

[28]  Katrin Wolf,et al.  Does proprioception guide back-of-device pointing as well as vision? , 2012, CHI Extended Abstracts.

[29]  Jun Rekimoto,et al.  GraspZoom: zooming and scrolling control model for single-handed mobile interaction , 2009, Mobile HCI.

[30]  Johan Kildal,et al.  Interacting with Deformable User Interfaces: Effect of Material Stiffness and Type of Deformation Gesture , 2012, HAID.

[31]  Michael Eisenberg,et al.  Easigami: virtual creation by physical folding , 2012, TEI.

[32]  Jaakko Keränen,et al.  Hands on music: physical approach to interaction with digital music , 2009, Mobile HCI.

[33]  Graham A. Wilson,et al.  Feeling it: the roles of stiffness, deformation range and feedback in the control of deformable ui , 2012, ICMI '12.

[34]  Nitesh Goyal.  COMET : Collaboration in Applications for Mobile Environments by Twisting , 2009 .

[35]  Robert J. K. Jacob,et al.  Integrality and separability of input devices , 1994, TCHI.

[36]  Karen Holtzblatt,et al.  Rapid Contextual Design: A How-To Guide to Key Techniques for User-Centered Design , 2004, UBIQ.