Cross-device interaction with large displays in public: insights from both users' and observers' perspectives

1Using a mixture of physical gestures and one's smartphone is a convenient way for people to engage and interact with large displays in public. Yet, one of the challenges of cross-device interactions is to design techniques that encourage participation. This paper presents a study of people using four different cross-device interaction techniques in a public setting, to identify how both users and observers feel about the device actions and bodily gestures required to interact with a large display using smartphones. We collected both direct feedback and observational data of users' and observers' attitudes and reactions to using these techniques in public. We identified five key factors influencing people's experience of interacting while being observed by others: Familiarity, Social Acceptability, Purpose, Easiness and Playfulness. We argue that it is important to consider observer attitudes when designing cross-device interactions for large displays in public, to encourage the participation of passers-by.

[1]  Raimund Dachselt,et al.  Natural throw and tilt interaction between mobile phones and distant displays , 2009, CHI Extended Abstracts.

[2]  Karen Holtzblatt,et al.  Contextual design: using customer work models to drive systems design , 1998, CHI Conference Summary.

[3]  Li-Chieh Chen,et al.  Exploring the Ergonomic Issues of User-Defined Mid-Air Gestures for Interactive Product Exhibition , 2016, HCI.

[4]  Anselm L. Strauss,et al.  Basics of qualitative research : techniques and procedures for developing grounded theory , 1998 .

[5]  S. Agamanolis Designing Displays for Human Connectedness , 2003 .

[6]  Carl Gutwin,et al.  There and Back Again: Cross-Display Object Movement in Multi-Display Environments , 2009, Hum. Comput. Interact..

[7]  Markku Turunen,et al.  Investigating mid-air gestures and handhelds in motion tracked environments , 2016, PerDis.

[8]  Ann Morrison,et al.  Worlds of information: designing for engagement at a public multi-touch display , 2010, CHI.

[9]  Yvonne Rogers,et al.  Enticing People to Interact with Large Public Displays in Public Spaces , 2003, INTERACT.

[10]  Randolph G. Bias,et al.  Research Methods for Human-Computer Interaction , 2010, J. Assoc. Inf. Sci. Technol..

[11]  Dominik Schmidt,et al.  A cross-device interaction style for mobiles and surfaces , 2012, DIS '12.

[12]  Daniel Vogel,et al.  Territoriality and behaviour on and around large vertical publicly-shared displays , 2012, DIS '12.

[13]  Eva Hornecker,et al.  Urban HCI: spatial aspects in the design of shared encounters for media facades , 2012, CHI.

[14]  Marc Hassenzahl,et al.  User experience (UX): towards an experiential perspective on product quality , 2008, IHM '08.

[15]  Kenton O'Hara,et al.  Public and Situated Displays: Social and Interactional Aspects of Shared Display Technologies , 2004 .

[16]  Yvonne Rogers,et al.  The introduction of a shared interactive surface into a communal space , 2004, CSCW.

[17]  Elaine Toms,et al.  What is user engagement? A conceptual framework for defining user engagement with technology , 2008, J. Assoc. Inf. Sci. Technol..

[18]  Jörg Müller,et al.  MirrorTouch: combining touch and mid-air gestures for public displays , 2014, MobileHCI '14.

[19]  Stacey D. Scott,et al.  Overcoming Interaction Barriers in Large Public Displays Using Personal Devices , 2014, ITS '14.

[20]  Judy Kay,et al.  An in-the-wild study of learning mid-air gestures to browse hierarchical information at a large interactive public display , 2015, UbiComp.

[21]  Michael Rohs,et al.  Sweep and point and shoot: phonecam-based interactions for large public displays , 2005, CHI Extended Abstracts.

[22]  Kim Halskov,et al.  Designing urban media façades: cases and challenges , 2010, CHI.

[23]  Anthony Tang,et al.  Designing for bystanders: reflections on building a public digital forum , 2008, CHI.

[24]  Alireza Sahami Shirazi,et al.  Interaction techniques for creating and exchanging content with public displays , 2013, CHI.

[25]  Michael Rohs,et al.  User-defined gestures for connecting mobile phones, public displays, and tabletops , 2010, Mobile HCI.

[26]  Florian Alt,et al.  Looking glass: a field study on noticing interactivity of a shop window , 2012, CHI.

[27]  Judy Kay,et al.  Skeletons and Silhouettes: Comparing User Representations at a Gesture-based Large Display , 2016, CHI.

[28]  Nigel Davies,et al.  ENGAGE: Early Insights in Measuring Multi-Device Engagements , 2015, PerDis.

[29]  Judy Kay,et al.  Who cares about the Content? An Analysis of Playful Behaviour at a Public Display , 2014, PerDis.

[30]  Karen Holtzblatt,et al.  Contextual design , 1997, INTR.

[31]  M. Sheelagh T. Carpendale,et al.  Interactive Public Displays , 2013, IEEE Computer Graphics and Applications.

[32]  Andrés Lucero,et al.  MobiComics: collaborative use of mobile phones and large displays for public expression , 2012, Mobile HCI.

[33]  Saul Greenberg,et al.  The notification collage: posting information to public and personal displays , 2001, CHI.

[34]  Anthony Collins,et al.  Investigating intuitiveness and effectiveness of gestures for free spatial interaction with large displays , 2012, PerDis '12.

[35]  Jesper Kjeldskov,et al.  Investigating Cross-Device Interaction between a Handheld Device and a Large Display , 2017, CHI.

[36]  Sebastian Boring,et al.  Gradual engagement: facilitating information exchange between digital devices as a function of proximity , 2012, ITS.

[37]  Keith Cheverst,et al.  Exploring bluetooth based mobile phone interaction with the hermes photo display , 2005, Mobile HCI.

[38]  Kris Luyten,et al.  Gestu-Wan - An Intelligible Mid-Air Gesture Guidance System for Walk-up-and-Use Displays , 2015, INTERACT.