Experimenting with Large Displays and Gestural Interaction in the Smart Factory

In this paper we present the results of two experimental evaluations carried out in the broader context of a research project on Industry 4.0. We envision in large displays operated through gestural interaction one of the key technologies for enabling real-time management of production processes and supporting decision-making. We developed prototype applications with the twofold aim of: (1) designing a vocabulary of one-handed, touchless gestures meant for interaction in a CAD-like environment, and (2) investigating the role of the cursor feedback. Our results provide a basis for the development of a new integrated wearable sensor which will be assessed with industry workers and in a more realistic setting, for better ecological validity.

[1]  Anthony Collins,et al.  Investigating intuitiveness and effectiveness of gestures for free spatial interaction with large displays , 2012, PerDis '12.

[2]  David Ott,et al.  Web on the Wall Reloaded: Implementation, Replication and Refinement of User-Defined Interaction Sets , 2014, ITS '14.

[3]  Joana Vieira,et al.  Gestures while driving : A guessability approach for a surface gestures taxonomy for in-vehicle indirect interaction , 2016 .

[4]  Thomas Grechenig,et al.  Comparing the immediate usability of graffiti 2 and virtual keyboard , 2004, CHI EA '04.

[5]  Gerrit Meixner,et al.  User interaction evolution in the SmartFactoryKL , 2010, BCS HCI.

[6]  Yang Zhang,et al.  Wall++: Room-Scale Interactive and Context-Aware Sensing , 2018, CHI.

[7]  Christian Diedrich,et al.  Cyber-physical systems alter automation architectures , 2014, Annu. Rev. Control..

[8]  Gierad Laput,et al.  AuraSense: Enabling Expressive Around-Smartwatch Interactions with Electric Field Sensing , 2016, UIST.

[9]  Sukeshini A. Grandhi,et al.  Understanding naturalness and intuitiveness in gesture production: insights for touchless gestural interfaces , 2011, CHI.

[10]  Markus Funk,et al.  An augmented workplace for enabling user-defined tangibles , 2014, CHI Extended Abstracts.

[11]  Robert Xiao,et al.  Gaze+Gesture: Expressive, Precise and Targeted Free-Space Interactions , 2015, ICMI.

[12]  Anton Nijholt,et al.  Gestures for Large Display Control , 2009, Gesture Workshop.

[13]  Brad A. Myers,et al.  Maximizing the guessability of symbolic input , 2005, CHI Extended Abstracts.

[14]  Dan Saffer.  Designing gestural interfaces , 2009 .

[15]  Franca Garzotto,et al.  Personalized interaction on large displays: the StreetSmart project approach , 2014, AVI.

[16]  Radu-Daniel Vatavu,et al.  User-defined gestures for free-hand TV control , 2012, EuroITV.

[17]  Bruce C. Straits,et al.  Approaches to social research , 1993 .

[18]  Meredith Ringel Morris,et al.  Web on the wall: insights from a multimodal interaction elicitation study , 2012, ITS.

[19]  Carmelo Ardito,et al.  Interaction with Large Displays , 2015, ACM Comput. Surv..

[20]  Yang Li,et al.  User-defined motion gestures for mobile interaction , 2011, CHI.

[21]  Mark Billinghurst,et al.  User Defined Gestures for Augmented Virtual Mirrors: A Guessability Study , 2015, CHI Extended Abstracts.

[22]  Carlos Duarte,et al.  A Study on the use of Gestures for Large Displays , 2009, ICEIS.

[23]  Alessio Malizia,et al.  Touchless Interfaces For Public Displays: Can We Deliver Interface Designers From Introducing Artificial Push Button Gestures? , 2016, AVI.

[24]  Jiafu Wan,et al.  Implementing Smart Factory of Industrie 4.0: An Outlook , 2016, Int. J. Distributed Sens. Networks.

[25]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[26]  Susan T. Dumais,et al.  The vocabulary problem in human-system communication , 1987, CACM.

[27]  Susan Wiedenbeck,et al.  The use of icons and labels in an end user application program: An empirical study of learning and retention , 1999, Behav. Inf. Technol..

[28]  Silvia Appendino,et al.  EMG-based biofeedback system for motor rehabilitation: A pilot study , 2017, 2017 IEEE Biomedical Circuits and Systems Conference (BioCAS).