A Comparison of Techniques for Cross-Device Interaction from Mobile Devices to Large Displays

In recent years there has been an increasing interest in cross-device interaction research involving mobile computing. We contribute to this research with a comparative study of four interaction techniques for moving information from a mobile device to a large display. The four techniques (Pinch, Swipe, Throw, and Tilt) were compared through a laboratory experiment with 53 participants, measuring their effectiveness, efficiency and error size. Findings from the experiment revealed that the Swipe technique performed best on all measures. In terms of effectiveness, the Tilt technique performed the worst, and especially so with small targets. In terms of efficiency and error size, the Pinch technique was the slowest and also the most imprecise. We also found that target size mattered considerably for all techniques, confirming previous research. Based on our findings we discuss why the individual techniques performed as observed, and discuss implications for using mobile devices in cross-device interaction design.

[1]  Timo Ojala,et al.  MobiToss: a novel gesture based interface for creating and sharing mobile multimedia art on large public displays , 2008, ACM Multimedia.

[2]  Dominik Schmidt,et al.  A cross-device interaction style for mobiles and surfaces , 2012, DIS '12.

[3]  Xiang 'Anthony' Chen,et al.  Duet: exploring joint interactions on a smart phone and a smart watch , 2014, CHI.

[4]  Andrés Lucero,et al.  MobiComics: collaborative use of mobile phones and large displays for public expression , 2012, Mobile HCI.

[5]  Marko Jurmu,et al.  Scroll, tilt or move it: using mobile phones to continuously control pointers on large public displays , 2009, OZCHI.

[6]  Nikolaos K. Tselios,et al.  Does size matter?: investigating the impact of mobile phone screen size on users' perceived usability, effectiveness and efficiency. , 2013, MobileHCI '13.

[7]  Larry L. Peterson,et al.  Reasoning about naming systems , 1993, TOPL.

[8]  Andreas Butz,et al.  Shoot & copy: phonecam-based information transfer from public displays onto mobile phones , 2007, Mobility '07.

[9]  Jesper Kjeldskov,et al.  Investigating Cross-Device Interaction Techniques: A Case of Card Playing on Handhelds and Tablets , 2015, OZCHI.

[10]  Jesper Kjeldskov,et al.  The 4C framework: principles of interaction in digital ecosystems , 2014, UbiComp.

[11]  Jörg Müller,et al.  Cuenesics: using mid-air gestures to select items on interactive public displays , 2014, MobileHCI '14.

[12]  Brad A. Myers,et al.  Using handhelds and PCs together , 2001, CACM.

[13]  Hrvoje Benko,et al.  Multi-point interactions with immersive omnidirectional visualizations in a dome , 2010, ITS '10.

[14]  Itiro Siio,et al.  Memory Stones: An Intuitive Information Transfer Technique between Multi-touch Computers , 2015, HotMobile.

[15]  Jesper Kjeldskov,et al.  JuxtaPinch: exploring multi-device interaction in collocated photo sharing , 2014, MobileHCI '14.

[16]  KjeldskovJesper,et al.  A comparison of techniques for cross-device interaction from mobile devices to large displays , 2017 .

[17]  Pourang Irani,et al.  Consumed endurance: a metric to quantify arm fatigue of mid-air interactions , 2014, CHI.

[18]  Daniel J. Wigdor,et al.  Conductor: enabling and understanding cross-device interaction , 2014, CHI.

[19]  Jun Rekimoto,et al.  Multiple-Computer User Interfaces: A Cooperative Environment Consisting of Multiple Digital Devices , 1998, CoBuild.

[20]  Yvonne Rogers,et al.  Spatially-aware or Spatially-agnostic?: Elicitation and Evaluation of User-Defined Cross-Device Interactions , 2015, CHI.

[21]  J. Tukey,et al.  Performance of Some Resistant Rules for Outlier Labeling , 1986 .

[22]  Dominik Schmidt.  PhoneTouch: a technique for direct phone interaction on surfaces , 2010, ITS '10.

[23]  Jun Rekimoto,et al.  Pick-and-drop: a direct manipulation technique for multiple computer environments , 1997, UIST '97.

[24]  Raimund Dachselt,et al.  Natural throw and tilt interaction between mobile phones and distant displays , 2009, CHI Extended Abstracts.

[25]  Saul Greenberg,et al.  Cross-device interaction via micro-mobility and f-formations , 2012, UIST.

[26]  Meredith Ringel Morris,et al.  Code space: touch + air gesture hybrid interactions for supporting developer meetings , 2011, ITS '11.

[27]  Tero Jokela,et al.  A Diary Study on Combining Multiple Information Devices in Everyday Activities and Tasks , 2015, CHI.