How to evaluate public displays

After years in the lab, interactive public displays are finding their way into public spaces, shop windows, and public institutions. They are equipped with a multitude of sensors as well as (multi-) touch surfaces allowing not only the audience to be sensed, but also their effectiveness to be measured. The lack of generally accepted design guidelines for public displays and the fact that there are many different objectives (e.g., increasing attention, optimizing interaction times, finding the best interaction technique) make it a challenging task to pick the most suitable evaluation method. Based on a literature survey and our own experiences, this paper provides an overview of study types, paradigms, and methods for evaluation both in the lab and in the real world. Following a discussion of design challenges, we provide a set of guidelines for researchers and practitioners alike to be applied when evaluating public displays.

[1]  Nemanja Memarovic,et al.  FunSquare: first experiences with autopoiesic content , 2011, MUM.

[2]  Anind K. Dey,et al.  Heuristic evaluation of ambient displays , 2003, CHI '03.

[3]  Allison Druin,et al.  Technology probes: inspiring design for and with families , 2003, CHI '03.

[4]  Antti Oulasvirta,et al.  It's Mine, Don't Touch!: interactions at a large multi-touch display in a city centre , 2008, CHI.

[5]  Marko Jurmu,et al.  UBI-Hotspot 1.0: Large-Scale Long-Term Deployment of Interactive Public Displays in a City Center , 2010, 2010 Fifth International Conference on Internet and Web Applications and Services.

[6]  Florian Alt,et al.  Looking glass: a field study on noticing interactivity of a shop window , 2012, CHI.

[7]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[8]  Jan O. Borchers,et al.  Overcoming Assumptions and Uncovering Practices: When Does the Public Really Look at Public Displays? , 2008, Pervasive.

[9]  Harry Hochheiser,et al.  Research Methods for Human-Computer Interaction , 2008 .

[10]  Michael Rohs,et al.  Sweep and point and shoot: phonecam-based interactions for large public displays , 2005, CHI Extended Abstracts.

[11]  Alireza Sahami Shirazi,et al.  Digifieds: insights into deploying digital public notice areas in the wild , 2011, MUM.

[12]  Patrick Baudisch,et al.  Improving drag-and-drop on wall-size displays , 2005, Graphics Interface.

[13]  C Mottram,et al.  The urban screen as a socialising platform: exploring the role of place within the urban space , 2008 .

[14]  Jennifer G. Sheridan,et al.  Public Ubiquitous Computing Systems: Lessons from the e-Campus Display Deployments , 2006, IEEE Pervasive Computing.

[15]  Manfred Tscheligi,et al.  AmIQuin - An Ambient Mannequin for the Shopping Environment , 2009, AmI.

[16]  Tara Matthews,et al.  Defining, Designing, and Evaluating Peripheral Displays: An Analysis Using Activity Theory , 2007, Hum. Comput. Interact..

[17]  Elke E. Mattheiss,et al.  Attentional Behavior of Users on the Move Towards Pervasive Advertising Media , 2011, Pervasive Advertising.

[18]  Margot Brereton,et al.  Designing Interaction for Local Communications: An Urban Screen Study , 2009, INTERACT.

[19]  Kori Inkpen Quinn,et al.  Single display privacyware: augmenting public displays with private information , 2001, CHI.

[20]  Hrvoje Benko,et al.  Combining multiple depth cameras and projectors for interactions on, above and between surfaces , 2010, UIST.

[21]  Albrecht Schmidt,et al.  Designing Shared Public Display Networks - Implications from Today's Paper-Based Notice Areas , 2011, Pervasive.

[22]  Daniel Vogel,et al.  Interactive public ambient displays: transitioning from implicit to explicit, public to personal, interaction with multiple users , 2004, UIST '04.

[23]  Antonio Krüger,et al.  ReflectiveSigns: Digital Signs That Adapt to Audience Attention , 2009, Pervasive.

[24]  Timo Ojala,et al.  It's a Jungle Out There: Fantasy and Reality of Evaluating Public Displays "in the Wild" , 2011 .

[25]  Keith Cheverst,et al.  The Challenge of Evaluating Situated Display based Technology Interventions Designed to Foster a Sense of Community , 2008 .

[26]  Ann Morrison,et al.  Worlds of information: designing for engagement at a public multi-touch display , 2010, CHI.

[27]  Keith Cheverst,et al.  Exploring the Utility of Remote Messaging and Situated Office Door Displays , 2003, Mobile HCI.

[28]  James T. Miller,et al.  An Empirical Evaluation of the System Usability Scale , 2008, Int. J. Hum. Comput. Interact..

[29]  Yvonne Rogers,et al.  Ambient influence: can twinkly lights lure and abstract representations trigger behavioral change? , 2010, UbiComp.

[30]  George W. Fitzmaurice,et al.  Spotlight: directing users' attention on large displays , 2005, CHI.

[31]  Albrecht Schmidt,et al.  Mobile Contextual Display Systems , 2009 .

[32]  Yvonne Rogers,et al.  The introduction of a shared interactive surface into a communal space , 2004, CSCW.

[33]  Andreas Butz,et al.  Touch projector: mobile interaction through video , 2010, CHI.

[34]  Daniel Vogel,et al.  Distant freehand pointing and clicking on very large, high resolution displays , 2005, UIST.

[35]  Dominik Schmidt.  PhoneTouch: a technique for direct phone interaction on surfaces , 2010, ITS '10.

[36]  Albrecht Schmidt,et al.  Display Blindness: The Effect of Expectations on Attention towards Digital Signage , 2009, Pervasive.

[37]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[38]  Yvonne Rogers,et al.  Enticing People to Interact with Large Public Displays in Public Spaces , 2003, INTERACT.

[39]  Daniel Fitton,et al.  Exploring the Evolution of Office Door Displays , 2003 .

[40]  Albrecht Schmidt,et al.  Audience behavior around large interactive cylindrical screens , 2011, CHI.

[41]  Kenton P. O'Hara,et al.  Public and Situated Displays , 2003, The Kluwer International series on Computer Supported Cooperative Work.