Code space: touch + air gesture hybrid interactions for supporting developer meetings

We present Code Space, a system that contributes touch + air gesture hybrid interactions to support co-located, small group developer meetings by democratizing access, control, and sharing of information across multiple personal devices and public displays. Our system uses a combination of a shared multi-touch screen, mobile touch devices, and Microsoft Kinect sensors. We describe cross-device interactions, which use a combination of in-air pointing for social disclosure of commands, targeting and mode setting, combined with touch for command execution and precise gestures. In a formative study, professional developers were positive about the interaction design, and most felt that pointing with hands or devices and forming hand postures are socially acceptable. Users also felt that the techniques adequately disclosed who was interacting and that existing social protocols would help to dictate most permissions, but also felt that our lightweight permission feature helped presenters manage incoming content.

[1]  Richard A. Bolt,et al.  “Put-that-there”: Voice and gesture at the graphics interface , 1980, SIGGRAPH '80.

[2]  A BoltRichard,et al.  Put-that-there , 1980 .

[3]  Judith S. Olson,et al.  Small Group Design Meetings: An Analysis of Collaboration , 1992, Hum. Comput. Interact..

[4]  John C. Tang,et al.  Liveboard: a large interactive display supporting group meetings, presentations, and remote collaboration , 1992, CHI.

[5]  Michel Beaudouin-Lafon,et al.  Charade: remote control of objects using free-hand gestures , 1993, CACM.

[6]  Kenneth A. Pier,et al.  Toolglass and magic lenses: the see-through interface , 1993, SIGGRAPH.

[7]  William Buxton,et al.  The limits of expert performance using hierarchic marking menus , 1993, INTERCHI.

[8]  Ken Hinckley,et al.  A survey of design issues in spatial input , 1994, UIST '94.

[9]  Jun Rekimoto,et al.  Pick-and-drop: a direct manipulation technique for multiple computer environments , 1997, UIST '97.

[10]  Thomas Rist,et al.  Referring Phenomena in a Multimedia Context and their Computational Treatment , 1997 .

[11]  R.A. Brooks,et al.  The Intelligent Room project , 1997, Proceedings Second International Conference on Cognitive Technology Humanizing the Information Age.

[12]  Antonella De Angeli,et al.  Integration and synchronization of input modes during multimodal human-computer interaction , 1997, CHI.

[13]  Jun Rekimoto,et al.  A multiple device approach for supporting whiteboard-based interactions , 1998, CHI.

[14]  Jun Rekimoto,et al.  Augmented surfaces: a spatially continuous work space for hybrid computing environments , 1999, CHI '99.

[15]  Norbert A. Streitz,et al.  An interactive Landscape for Creativity and Innovation , 1999 .

[16]  Norbert A. Streitz,et al.  i-LAND: an interactive landscape for creativity and innovation , 1999, CHI '99.

[17]  Brad A. Myers.  Using multiple devices simultaneously for display and control , 2000, IEEE Wirel. Commun..

[18]  J. Krumm,et al.  Multi-camera multi-person tracking for EasyLiving , 2000, Proceedings Third IEEE International Workshop on Visual Surveillance.

[19]  Jun Rekimoto,et al.  Multiple-computer user interfaces: "beyond the desktop" direct manipulation environments , 2000, CHI Extended Abstracts.

[20]  Armando Fox,et al.  The Interactive Workspaces Project: Experiences with Ubiquitous Computing Rooms , 2002, IEEE Pervasive Comput..

[21]  William Buxton,et al.  Sentient Data Access via a Diverse Society of Devices , 2003, ACM Queue.

[22]  Yvonne Rogers,et al.  Dynamo: a public interactive surface supporting the cooperative sharing and exchange of media , 2003, UIST '03.

[23]  Patrick Baudisch,et al.  Stitching: pen gestures that span multiple displays , 2004, AVI.

[24]  Daniel Vogel,et al.  Interactive public ambient displays: transitioning from implicit to explicit, public to personal, interaction with multiple users , 2004, UIST '04.

[25]  Michael Boyle,et al.  PDAs and shared public displays: Making personal information public, and public information personal , 1999, Personal Technologies.

[26]  Andreas Paepcke,et al.  Cooperative gestures: multi-user gestural interactions for co-located groupware , 2006, CHI.

[27]  Mary Czerwinski,et al.  FASTDash: a visual dashboard for fostering awareness in software teams , 2007, CHI.

[28]  Andrew D. Wilson,et al.  BlueTable: connecting wireless mobile devices on interactive surfaces using vision-based handshaking , 2007, GI '07.

[29]  André van der Hoek,et al.  Calico: a prototype sketching tool for modeling in early design , 2008, MiSE '08.

[30]  Antonio Krüger,et al.  Using Mobile Phones to Spontaneously Authenticate and Interact with Multi-Touch Surfaces , 2008 .

[31]  Joseph J. LaViola,et al.  GestureBar: improving the approachability of gesture-based interfaces , 2009, CHI.

[32]  Carl Gutwin,et al.  There and Back Again: Cross-Display Object Movement in Multi-Display Environments , 2009, Hum. Comput. Interact..

[33]  Khai N. Truong,et al.  Virtual shelves: interactions with orientation aware devices , 2009, UIST '09.

[34]  Hao Jiang,et al.  WeSpace: the design development and deployment of a walk-up and share multi-surface visual collaboration system , 2009, CHI.

[35]  Pourang Irani,et al.  Chucking: A One-Handed Document Sharing Technique , 2009, INTERACT.

[36]  Robert C. Zeleznik,et al.  Hands-on math: a page-based multi-touch and pen desktop for technical work and problem solving , 2010, UIST.

[37]  Dominik Schmidt.  PhoneTouch: a technique for direct phone interaction on surfaces , 2010, ITS '10.

[38]  Stephen A. Brewster,et al.  Usable gestures for mobile interfaces: evaluating social acceptability , 2010, CHI.

[39]  Saul Greenberg,et al.  Proxemic interaction: designing for a proximity and orientation-aware environment , 2010, ITS '10.

[40]  Joseph J. LaViola,et al.  Code bubbles: a working set-based interface for code understanding and maintenance , 2010, CHI.

[41]  Spencer Rugaber,et al.  CodePad: interactive spaces for maintaining concentration in programming environments , 2010, SOFTVIS '10.

[42]  Hrvoje Benko,et al.  Combining multiple depth cameras and projectors for interactions on, above and between surfaces , 2010, UIST.

[43]  Andrew Bragdon,et al.  Gesture select:: acquiring remote targets on large displays without pointing , 2011, CHI.