Cuenesics: using mid-air gestures to select items on interactive public displays

Most of today's public displays only show predefined contents that passers-by are not able to change. We argue that interactive public displays would benefit from immediately usable mid-air techniques for choosing options, expressing opinions or more generally selecting one among several items. We propose a design space for hand-gesture based mid-air selection techniques on interactive public displays, along with four specific techniques that we evaluated at three different locations in the the field. Our findings include: 1) if no hint is provided, people successfully use Point+Dwell for selecting items, 2) the user representation could be switched from Mirror to Cursor after registration without causing confusion, 3) people tend to explore items before confirming one, 4) in a public context, people frequently interact inadvertently (without looking at the screen). We conclude by providing recommendations for designers of interactive public displays to support immediate usability for mid-air selection.

[1]  Myron W. Krueger,et al.  Artificial reality II , 1991 .

[2]  Erik Nilsen,et al.  PERCEPTUAL-MOTOR CONTROL IN HUMAN-COMPUTER INTERACTION , 1991, SGCH.

[3]  User learning and performance with marking menus , 1994, CHI '94.

[4]  Doug A. Bowman,et al.  Design and evaluation of menu systems for immersive virtual environments , 2001, Proceedings IEEE Virtual Reality 2001.

[5]  Yvonne Rogers,et al.  Enticing People to Interact with Large Public Displays in Public Spaces , 2003, INTERACT.

[6]  Daniel Vogel,et al.  Interactive public ambient displays: transitioning from implicit to explicit, public to personal, interaction with multiple users , 2004, UIST '04.

[7]  Ben Shneiderman,et al.  Immediate usability: a case study of public access design for a community photo library , 2004, Interact. Comput..

[8]  Mike Wu,et al.  Gesture registration, relaxation, and reuse for multi-point direct-touch surfaces , 2006, First IEEE International Workshop on Horizontal Interactive Human-Computer Systems (TABLETOP '06).

[9]  Andy Cockburn,et al.  Faster cascading menu selections with enlarged activation areas , 2006, Graphics Interface.

[10]  Pierre Dragicevic,et al.  Earpod: eyes-free menu selection using touch input and reactive audio feedback , 2007, CHI.

[11]  Antti Oulasvirta,et al.  It's Mine, Don't Touch!: interactions at a large multi-touch display in a city centre , 2008, CHI.

[12]  Jan O. Borchers,et al.  Overcoming Assumptions and Uncovering Practices: When Does the Public Really Look at Public Displays? , 2008, Pervasive.

[13]  Albrecht Schmidt,et al.  Display Blindness: The Effect of Expectations on Attention towards Digital Signage , 2009, Pervasive.

[14]  Tovi Grossman,et al.  PenLight: combining a mobile projector and a digital pen for dynamic visual overlay , 2009, CHI.

[15]  Yoshifumi Kitamura,et al.  Body-centric interaction techniques for very large wall displays , 2010, NordiCHI.

[16]  Tovi Grossman,et al.  The design and evaluation of multitouch marking menus , 2010, CHI.

[17]  Gilles Bailly,et al.  Finger-count & radial-stroke shortcuts: 2 techniques for augmenting linear menus on multi-touch surfaces , 2010, CHI.

[18]  Albrecht Schmidt,et al.  Requirements and design space for interactive public displays , 2010, ACM Multimedia.

[19]  Wendy Ju,et al.  Animate Objects: How Physical Motion Encourages Public Interaction , 2010, PERSUASIVE.

[20]  Nadine Mandran,et al.  Wavelet menus on handheld devices: stacking metaphor for novice mode and eyes-free selection for expert mode , 2010, AVI.

[21]  Ann Morrison,et al.  Worlds of information: designing for engagement at a public multi-touch display , 2010, CHI.

[22]  Gilles Bailly,et al.  Gesture-aware remote controls: guidelines and interaction technique , 2011, ICMI '11.

[23]  Jörg Müller,et al.  Comparing Free Hand Menu Techniques for Distant Displays Using Linear, Marking and Finger-Count Menus , 2011, INTERACT.

[24]  Yvonne Rogers,et al.  Rethinking 'multi-user': an in-the-wild study of how groups approach a walk-up-and-use tabletop interface , 2011, CHI.

[25]  Florian Alt,et al.  Looking glass: a field study on noticing interactivity of a shop window , 2012, CHI.

[26]  Anthony Collins,et al.  Investigating intuitiveness and effectiveness of gestures for free spatial interaction with large displays , 2012, PerDis '12.

[27]  Jörg Müller,et al.  Chained displays: configurations of public displays can be used to influence actor-, audience-, and passer-by behavior , 2012, CHI.

[28]  Krystian Samp,et al.  Designing graphical menus for novices and experts: connecting design characteristics with design goals , 2013, CHI.

[29]  Nina Valkanova,et al.  MyPosition: A Public Polling Projection Display Designed to Spark Civic Participation , 2013 .

[30]  Jorge Gonçalves,et al.  Multipurpose Public Displays: How Shortcut Menus Affect Usage , 2013, IEEE Computer Graphics and Applications.

[31]  Jörg Müller,et al.  StrikeAPose: revealing mid-air gestures on public displays , 2013, CHI.

[32]  Jörg Müller,et al.  MyPosition: sparking civic discourse by a public interactive poll visualization , 2014, CSCW.