Understanding Usability Challenges in Shifting between Multiple Devices during a Task

The use of multiple interactive devices in conducting one task is becoming more and more commonplace today. However, as the user interfaces (UIs) and interaction strategies have mostly been designed for individual devices or for a small number of paired devices, spontaneously combining different devices may result in usability issues that are worse than it was before the change takes place. In this paper, we present and demonstrate the concept of device composite – a combination of devices together with a set of suitable user interfaces and interaction strategies which will be activated when these devices are used together. Focusing on the usability side of the multi-device usage scenarios, we categorize the device composites into six basic interaction settings or configurations, and discuss corresponding usability issues associated with them. Two of the composite situations were developed into a fully-working prototype device ecosystem, and a user study was conducted with 18 participants. Findings and insights are reported suggesting further design directions for multi-device usability.

[1]  Alan Snyder,et al.  Encapsulation and inheritance in object-oriented programming languages , 1986, OOPLSA '86.

[2]  Takashi Ohta,et al.  Pinch: An Interface That Relates Applications on Multiple Touch-Screen by 'Pinching' Gesture , 2012, Advances in Computer Entertainment.

[3]  Kris Luyten,et al.  Distributed user interface elements to support smart interaction spaces , 2005, Seventh IEEE International Symposium on Multimedia (ISM'05).

[4]  Michael Rohs,et al.  Sweep and point and shoot: phonecam-based interactions for large public displays , 2005, CHI Extended Abstracts.

[5]  Laurent Karsenty,et al.  Inter‐Usability of Multi‐Device Systems – A Conceptual Framework , 2005 .

[6]  Gerd Kortuem,et al.  Supporting device discovery and spontaneous interaction with spatial references , 2009, Personal and Ubiquitous Computing.

[7]  Ken Hinckley,et al.  Synchronous gestures for multiple persons and computers , 2003, UIST '03.

[8]  Hyowon Lee,et al.  SocioCon: A Social Circle for Your Interactive Devices , 2018, HCI.

[9]  Hyowon Lee,et al.  'Hop-to-select' traverse with gestural input in an eye-off interaction , 2017, OZCHI.

[10]  Daniel J. Wigdor,et al.  Panelrama: enabling easy specification of cross-device web applications , 2014, CHI.

[11]  Gerd Kortuem,et al.  Concepts and issues in interfaces for multiple users and multiple devices , 2004 .

[12]  Ellen W. Zegura,et al.  Serendipity: enabling remote computing among intermittently connected mobile devices , 2012, MobiHoc '12.

[13]  Clifton Forlines,et al.  MultiSpace: enabling electronic document micro-mobility in table-centric, multi-device environments , 2006, First IEEE International Workshop on Horizontal Interactive Human-Computer Systems (TABLETOP '06).

[14]  Suranga Nanayakkara,et al.  Visual Field Visualizer: Easier & Scalable way to be Aware of the Visual Field , 2018, AH.

[15]  Pan Hui,et al.  Towards collaborative multi-device computing , 2018, 2018 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops).

[16]  Patrick Baudisch,et al.  Stitching: pen gestures that span multiple displays , 2004, AVI.

[17]  Jörg Ott,et al.  Pervasive Communities in the Internet of People , 2018, 2018 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops).

[18]  Bernt Schiele,et al.  Smart-Its Friends: A Technique for Users to Easily Establish Connections between Smart Artefacts , 2001, UbiComp.

[19]  Erik Stolterman,et al.  Toward a framework for ecologies of artifacts: how are digital artifacts interconnected within a personal life? , 2008, NordiCHI.

[20]  Jun Rekimoto.  SyncTap: synchronous user operation for spontaneous network connection , 2004, Personal and Ubiquitous Computing.

[21]  Timo Ojala,et al.  Mobile group interaction with interactive video on large public display , 2005, SIGGRAPH '05.

[22]  Jesper Kjeldskov,et al.  The 4C framework: principles of interaction in digital ecosystems , 2014, UbiComp.

[23]  Jesper Kjeldskov,et al.  JuxtaPinch: exploring multi-device interaction in collocated photo sharing , 2014, MobileHCI '14.

[24]  Andrés Lucero,et al.  Pass-them-around: collaborative use of mobile phones for photo sharing , 2011, CHI.

[25]  Mark W. Newman,et al.  Designing for serendipity: supporting end-user configuration of ubiquitous computing environments , 2002, DIS '02.