SocioCon: A Social Circle for Your Interactive Devices

Our everyday lives with technology are increasingly characterised by the use of multiple interactive devices. We are seeing more and more variety of specialised input and output devices that can connect with other devices to help the user perform a particular task in a more optimal or other beneficial way. When multiple devices are used together within a session, owing to the diversity of native interaction modalities, content representation and capabilities of each device, there are usability issues arising before, after or at the transition of inter-connectivity. In such a device-sharing situation, one device that could temporarily borrow the input or output modalities from other devices must know which devices are available or allowed to offer such connections, and how their combined use should push yet another way of end-user interactivity. In this paper, we envision a few typical usage scenarios where such multi-device interaction is expected or desirable, then from these develop an overarching concept of usability in a multi-device environment. Simple social heuristics in sharing the interactivity and resources amongst devices are employed to shape how the overall usability in these situations could be addressed and framed. We develop two multi-device interaction prototypes that correspond to some of these scenarios, conduct usability testing, report the findings and discuss further insights.

[1]  Andrés Lucero,et al.  Collaborative use of mobile phones for brainstorming , 2010, Mobile HCI.

[2]  Richard A. Bolt,et al.  “Put-that-there”: Voice and gesture at the graphics interface , 1980, SIGGRAPH '80.

[3]  Suguru Higashino,et al.  C-blink: a hue-difference-based light signal marker for large screen interaction via any mobile terminal , 2004, UIST '04.

[4]  Mikael Wiberg,et al.  Conceptually Advancing "Application Mobility" Towards Design: Applying a Concept-Driven Approach to the Design of Mobile IT for Home Care Service Groups , 2012, Int. J. Ambient Comput. Intell..

[5]  Parth H. Pathak,et al.  Finger-writing with Smartwatch: A Case for Finger and Hand Gesture Recognition using Smartwatch , 2015, HotMobile.

[6]  Jun Rekimoto,et al.  Pick-and-drop: a direct manipulation technique for multiple computer environments , 1997, UIST '97.

[7]  Jun Rekimoto,et al.  Augmented surfaces: a spatially continuous work space for hybrid computing environments , 1999, CHI '99.

[8]  Norbert A. Streitz,et al.  i-LAND: an interactive landscape for creativity and innovation , 1999, CHI '99.

[9]  Elena Mugellini,et al.  A Smart Watch with Embedded Sensors to Recognize Objects, Grasps and Forearm Gestures , 2012 .

[10]  Brad A. Myers,et al.  Using handhelds and PCs together , 2001, CACM.

[11]  Hyowon Lee,et al.  'Hop-to-select' traverse with gestural input in an eye-off interaction , 2017, OZCHI.

[12]  Moira C. Norrie,et al.  PresiShare: opportunistic sharing and presentation of content using public displays and QR codes , 2013, PerDis '13.

[13]  Jukka Riekki,et al.  Multimodal interaction with speech and physical touch interface in a media center application , 2009, Advances in Computer Entertainment Technology.

[14]  Jesper Kjeldskov,et al.  The interaction space of a multi-device, multi-user music experience , 2012, NordiCHI.

[15]  John J. Leggett,et al.  Interaction styles and input/output devices , 1993, Behav. Inf. Technol..

[16]  Jian Lu,et al.  FollowMe: on research of pluggable infrastructure for context-awareness , 2006, 20th International Conference on Advanced Information Networking and Applications - Volume 1 (AINA'06).

[17]  Patrick Baudisch,et al.  Stitching: pen gestures that span multiple displays , 2004, AVI.

[18]  Jun Rekimoto,et al.  GestureWrist and GesturePad: unobtrusive wearable interaction devices , 2001, Proceedings Fifth International Symposium on Wearable Computers.

[19]  Tero Jokela,et al.  A Diary Study on Combining Multiple Information Devices in Everyday Activities and Tasks , 2015, CHI.

[20]  Gregory R. Crane,et al.  Drudgery and deep thought , 2001, CACM.

[21]  Clifton Forlines,et al.  MultiSpace: enabling electronic document micro-mobility in table-centric, multi-device environments , 2006, First IEEE International Workshop on Horizontal Interactive Human-Computer Systems (TABLETOP '06).

[22]  Ki-Won Yeom,et al.  Select-and-point: a novel interface for multi-device connection and control based on simple hand gestures , 2008, CHI Extended Abstracts.

[23]  Jun Rekimoto,et al.  Proximal Interactions: A Direct Manipulation Technique for Wireless Networking , 2003, INTERACT.

[24]  Alexander De Luca,et al.  A privacy-respectful input method for public terminals , 2008, NordiCHI.

[25]  Keith Cheverst,et al.  Exploring bluetooth based mobile phone interaction with the hermes photo display , 2005, Mobile HCI.

[26]  Buntarou Shizuki,et al.  Exploring Context-Aware User Interfaces for Smartphone-Smartwatch Cross-Device Interaction , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[27]  Jun Rekimoto,et al.  SyncTap: An Interaction Technique for Mobile Networking , 2003, Mobile HCI.

[28]  Suranga Nanayakkara,et al.  Visual Field Visualizer: Easier & Scalable way to be Aware of the Visual Field , 2018, AH.

[29]  Joohee Kim,et al.  TakeOut: drawing application using distributed user interface for being close to real experience , 2016, UbiComp Adjunct.

[30]  Xiang 'Anthony' Chen,et al.  Duet: exploring joint interactions on a smart phone and a smart watch , 2014, CHI.

[31]  Michael Rohs,et al.  Sweep and point and shoot: phonecam-based interactions for large public displays , 2005, CHI Extended Abstracts.

[32]  Ken Hinckley,et al.  Synchronous gestures for multiple persons and computers , 2003, UIST '03.

[33]  Michael Boyle,et al.  PDAs and shared public displays: Making personal information public, and public information personal , 1999, Personal Technologies.

[34]  Riku Suomela,et al.  Enhancing end-user experience in a multi-device ecosystem , 2005, MUM '05.

[35]  Jesper Kjeldskov,et al.  The 4C framework: principles of interaction in digital ecosystems , 2014, UbiComp.

[36]  Hideki Koike,et al.  3-D interaction with a large wall display using transparent markers , 2010, AVI.

[37]  Alireza Sahami Shirazi,et al.  Interaction techniques for creating and exchanging content with public displays , 2013, CHI.

[38]  Marita Franzke,et al.  Dual device user interface design: PDAs and interactive television , 1996, CHI.