Exploring spatially-aware cross-device interaction techniques for mobile collaborative sensemaking

Abstract The collaborative decision-making process is traditionally supported by multi-user interfaces, such as large multi-touch screens or interactive tabletops for accessing, relating and comparing different data sources. Since such multi-user interfaces are typically expensive and unavailable outside dedicated environments (e.g. labs, smart rooms), recent works have proposed “bring your own device” approaches that allow users to join their mobile devices (e.g. smartphones, tablets) in an ad-hoc manner to temporarily create multi-user cross-device systems. Such approaches can be enabled by spatially-aware cross-device interactions that have only been explored for simple operations. We conducted a three-step research study involving a total number of 65 users in 18 groups, in order to propose a composition paradigm that offers three interaction techniques for performing more complex operations, such as forwarding to multiple devices queries or query results or aggregating and visualizing search results across device boundaries.

[1]  Margaret M. Burnett,et al.  A practical guide to controlled experiments of software engineering tools with human participants , 2013, Empirical Software Engineering.

[2]  Nicolai Marquardt,et al.  Proxemic interactions: the new ubicomp? , 2011, INTR.

[3]  Florian Daniel,et al.  Interactive, Live Mashup Development Through UI-Oriented Computing , 2015, RMC@ICWE.

[4]  Cesare Pautasso,et al.  Rapid Mashup Development Tools , 2016, Communications in Computer and Information Science.

[5]  Christian Holz,et al.  SurfaceConstellations: A Modular Hardware Platform for Ad-Hoc Reconfigurable Cross-Device Workspaces , 2018, CHI.

[6]  Claudio S. Pinhanez,et al.  A study on the manipulation of 2D objects in a projector/camera-based augmented reality environment , 2005, CHI.

[7]  Antonella De Angeli,et al.  Service Composition for Non-programmers: Prospects, Problems, and Design Recommendations , 2010, 2010 Eighth IEEE European Conference on Web Services.

[8]  Mary Czerwinski,et al.  Users' Theories of the Desktop Metaphor, or Why We Should Seek Metaphor-Free Interfaces , 2007 .

[9]  Philip T. Kortum,et al.  Determining what individual SUS scores mean: adding an adjective rating scale , 2009 .

[10]  Andrés Lucero,et al.  Pass-them-around: collaborative use of mobile phones for photo sharing , 2011, CHI.

[11]  Ralf Klamma,et al.  DireWolf - Distributing and Migrating User Interfaces for Widget-Based Web Applications , 2013, ICWE.

[12]  Carl Gutwin,et al.  Superflick: a natural and efficient technique for long-distance object placement on digital tables , 2006, Graphics Interface.

[13]  Carla Mathison,et al.  Designing Augmented Reality Experiences in Authentic Learning Environments , 2012 .

[14]  Arne Jönsson,et al.  Wizard of Oz studies: why and how , 1993, IUI '93.

[15]  Mary Beth Rosson,et al.  Scenario-based design , 2002 .

[16]  Carl Gutwin,et al.  There and Back Again: Cross-Display Object Movement in Multi-Display Environments , 2009, Hum. Comput. Interact..

[17]  Robert Hardy,et al.  Touch & interact: touch-based interaction of mobile phones with displays , 2008, Mobile HCI.

[18]  Moira C. Norrie,et al.  MultiMasher: Providing Architectural Support and Visual Tools for Multi-device Mashups , 2014, WISE.

[19]  Saul Greenberg,et al.  Cross-device interaction via micro-mobility and f-formations , 2012, UIST.

[20]  James T. Miller,et al.  An Empirical Evaluation of the System Usability Scale , 2008, Int. J. Hum. Comput. Interact..

[21]  Giuseppe Desolda,et al.  Spatial Awareness in Mobile Devices to Compose Data Source: A Utilization Study , 2015, IS-EUD.

[22]  Campos,et al.  Proceedings of the 4th ACM SIGCHI Symposium on Engineering Interactive Computing Systems, EICS'12 , 2012 .

[23]  Patrick Baudisch,et al.  Stitching: pen gestures that span multiple displays , 2004, AVI.

[24]  Stacey D. Scott,et al.  Group vs Individual: Impact of TOUCH and TILT Cross-Device Interactions on Mixed-Focus Collaboration , 2018, CHI.

[25]  Natasa Milic-Frayling,et al.  Materializing the query with facet-streams: a hybrid surface for collaborative search on tabletops , 2011, CHI.

[26]  Florian Alt,et al.  Pocket Transfers: Interaction Techniques for Transferring Content from Situated Displays to Mobile Devices , 2018, CHI.

[27]  Teddy Seyed,et al.  Eliciting usable gestures for multi-display environments , 2012, ITS.

[28]  Alan J. Dix,et al.  TAPAS: A tangible End-User Development tool supporting the repurposing of Pervasive Displays , 2017, J. Vis. Lang. Comput..

[29]  Raimund Dachselt,et al.  Natural throw and tilt interaction between mobile phones and distant displays , 2009, CHI Extended Abstracts.

[30]  Jacob Buur,et al.  Video card game: an augmented environment for user centred design discussions , 2000, DARE '00.

[31]  Andrea Furlan,et al.  The Interactive API (iAPI) , 2013, ICWE Workshops.

[32]  Christopher Andrews,et al.  Space to think: large high-resolution displays for sensemaking , 2010, CHI.

[33]  Alireza Sahami Shirazi,et al.  Designing Consistent Gestures Across Device Types: Eliciting RSVP Controls for Phone, Watch, and Glasses , 2018, CHI.

[34]  J. B. Brooke,et al.  SUS: a retrospective , 2013 .

[35]  Cesare Pautasso,et al.  End-User Development of Mashups with NaturalMash , 2014, J. Vis. Lang. Comput..

[36]  Yvonne Rogers,et al.  HuddleLamp: Spatially-Aware Mobile Displays for Ad-hoc Around-the-Table Collaboration , 2014, ITS '14.

[37]  Alireza Sahami Shirazi,et al.  Interaction techniques for creating and exchanging content with public displays , 2013, CHI.

[38]  Carmelo Ardito,et al.  Creation and use of service-based Distributed Interactive Workspaces , 2014, J. Vis. Lang. Comput..

[39]  Florian Daniel,et al.  Mashups - Concepts, Models and Architectures , 2014, Data-Centric Systems and Applications.

[40]  Daniel J. Wigdor,et al.  Conductor: enabling and understanding cross-device interaction , 2014, CHI.

[41]  Roman Rädle,et al.  When Tablets meet Tabletops: The Effect of Tabletop Size on Around-the-Table Collaboration with Personal Tablets , 2016, CHI.

[42]  Nicholas Chen,et al.  Graduate student use of a multi-slate reading system , 2013, CHI.

[43]  Alison Kidd,et al.  The marks are on the knowledge worker , 1994, CHI '94.

[44]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[45]  Carmelo Ardito,et al.  Interaction with Large Displays , 2015, ACM Comput. Surv..

[46]  Cesare Pautasso,et al.  Reusable decision space for mashup tool design , 2012, EICS '12.

[47]  Mark Pruett,et al.  Yahoo! pipes , 2007 .

[48]  Jun Rekimoto.  SyncTap: synchronous user operation for spontaneous network connection , 2004, Personal and Ubiquitous Computing.

[49]  Stacey D. Scott,et al.  Collaborative sensemaking on a digital tabletop and personal tablets: prioritization, comparisons, and tableaux , 2013, CHI.

[50]  Carmelo Ardito,et al.  Visual composition of data sources by end users , 2014, AVI.

[51]  Florian Daniel.  Live, Personal Data Integration Through UI-Oriented Computing , 2015, ICWE.

[52]  Bongshin Lee,et al.  Reducing legacy bias in gesture elicitation studies , 2014, INTR.

[53]  Jeff Sauro,et al.  The Factor Structure of the System Usability Scale , 2009, HCI.

[54]  Jun Rekimoto,et al.  Augmented surfaces: a spatially continuous work space for hybrid computing environments , 1999, CHI '99.

[55]  Hans-Werner Gellersen,et al.  How users associate wireless devices , 2011, CHI.

[56]  Daniel J. Wigdor,et al.  Panelrama: enabling easy specification of cross-device web applications , 2014, CHI.

[57]  Chris North,et al.  VisPorter: facilitating information sharing for collaborative sensemaking on multiple displays , 2014, Personal and Ubiquitous Computing.

[58]  Yvonne Rogers,et al.  Spatially-aware or Spatially-agnostic?: Elicitation and Evaluation of User-Defined Cross-Device Interactions , 2015, CHI.

[59]  James H. Aylor,et al.  Computer for the 21st Century , 1999, Computer.

[60]  Nikolay Mehandjiev,et al.  Alternative Representations for End User Composition of Service-Based Systems , 2011, IS-EUD.

[61]  Gregory D. Abowd,et al.  Human-Computer Interaction (3rd Edition) , 2003 .

[62]  Jason J. Jung.  ContextGrid: A contextual mashup-based collaborative browsing system , 2012, Inf. Syst. Frontiers.

[63]  Meredith Ringel Morris,et al.  User-defined gestures for surface computing , 2009, CHI.

[64]  Niklas Elmqvist,et al.  Branch-explore-merge: facilitating real-time revision control in collaborative visual exploration , 2012, ITS.

[65]  Austin Henderson,et al.  Interaction design: beyond human-computer interaction , 2002, UBIQ.

[66]  M. Prensky.  Digital Natives, Digital Immigrants Part 1 , 2001 .

[67]  Andrés Lucero,et al.  Collaborative use of mobile phones for brainstorming , 2010, Mobile HCI.

[68]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[69]  Thomas W. Malone,et al.  How do people organize their desks?: Implications for the design of office information systems , 1983, TOIS.

[70]  Ken Hinckley,et al.  Synchronous gestures for multiple persons and computers , 2003, UIST '03.

[71]  Ken Hinckley.  Bumping Objects Together as a Semantically Rich Way of Forming Connections between Ubiquitous Devices , 2016 .

[72]  Martin Porcheron,et al.  Co-curator: designing for mobile ideation in groups , 2016, MindTrek.

[73]  David Kirsh,et al.  The Intelligent Use of Space , 1995, Artif. Intell..

[74]  Yvonne Rogers,et al.  Interaction Design: Beyond Human-Computer Interaction , 2002 .

[75]  V. Braun,et al.  Using thematic analysis in psychology , 2006 .

[76]  Chiara Francalanci,et al.  DashMash: A Mashup Environment for End User Development , 2011, ICWE.

[77]  Alan J. Dix.  Human-Computer Interaction , 2018, Encyclopedia of Database Systems.

[78]  Brian P. Bailey,et al.  ARIS: An Interface for Application Relocation in an Interactive Space , 2004, Graphics Interface.

[79]  Giuseppe Desolda.  Enhancing Workspace Composition by Exploiting Linked Open Data as a Polymorphic Data Source , 2015 .

[80]  Carmelo Ardito,et al.  EFESTO: A Platform for the End-User Development of Interactive Workspaces for Data Exploration , 2015, RMC@ICWE.

[81]  Natalia Juristo Juzgado,et al.  Basics of Software Engineering Experimentation , 2010, Springer US.