Smartwatch-Enhanced Interaction with an Advanced Troubleshooting System for Industrial Machines

Abstract Smartwatches are unobtrusive everyday devices which can be also exploited for effective gesture-based human-machine interaction. In this paper, we propose the use of a smartwatch to interact with an advanced troubleshooting application to be used in industrial environment. The application is a hypermedia information system aiming at assisting workers in performing preventive and corrective machine maintenance. The smartwatch allows a handsfree interaction, thus facilitating the use of the whole system when wearing personal protective equipment such as gloves or having fingers greased with oil or dust, which prevent operating touch screens. The algorithm for gesture recognition we have devised, which is based on template matching, is described in the paper, together with its experimental validation. Finally, we present a preliminary usability assessment of the overall system, meant as integration of the smartwatch with the hypermedia system.

[1]  Richard Greenough,et al.  An interactive electronic technical manual for an advanced aerospace assembly machine , 2007 .

[2]  Sheikh Iqbal Ahamed,et al.  Gesthaar: An accelerometer-based gesture recognition method and its application in NUI driven pervasive healthcare , 2012, 2012 IEEE International Conference on Emerging Signal Processing Applications.

[3]  Jeremy P. Gray,et al.  Control solutions for robots using Android and iOS devices , 2012, Defense, Security, and Sensing.

[4]  Zhen Wang,et al.  uWave: Accelerometer-based Personalized Gesture Recognition and Its Applications , 2009, PerCom.

[5]  Lee Garber.  Gestural Technology: Moving Interfaces in a New Direction , 2013, Computer.

[6]  Lorenzo Porzi,et al.  A smart watch-based gesture recognition system for assisting people with visual impairments , 2013, IMMPD '13.

[7]  Thomas Kirste,et al.  Ambient interaction by smart watches , 2012, PETRA '12.

[8]  Shahrokh Valaee,et al.  Accelerometer-based gesture recognition via dynamic-time warping, affinity propagation, & compressive sensing , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[9]  Cesare Fantuzzi,et al.  MyAID: a Troubleshooting Application for Supporting Human Operators in Industrial Environment , 2016 .

[10]  Thomas B. Moeslund,et al.  A Survey of Computer Vision-Based Human Motion Capture , 2001, Comput. Vis. Image Underst..

[11]  Rossitza Setchi,et al.  The development of a hypermedia maintenance manual for an advanced manufacturing company , 2003 .

[12]  Gary B. Wills,et al.  Evaluation of a hypermedia maintenance support application , 2003, Comput. Ind..

[13]  Lama Nachman,et al.  Don't slow me down: Bringing energy efficiency to continuous gesture recognition , 2010, International Symposium on Wearable Computers (ISWC) 2010.

[14]  Richard Greenough,et al.  User‐interface design heuristics for developing usable industrial hypermedia applications , 2002 .

[15]  S. Mitra,et al.  Gesture Recognition: A Survey , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[16]  H. Yanco,et al.  Analysis of natural gestures for controlling robot teams on multi-touch tabletop surfaces , 2009, ITS '09.

[17]  Paul Lukowicz,et al.  Gesture spotting with body-worn inertial sensors to detect user activities , 2008, Pattern Recognit..