Image-based gesture recognition for user interaction with mobile companion-based assistance systems

In this paper, we present image-based methods for robust recognition of static and dynamic hand gestures in real-time. These methods are used for an intuitive interaction with an assistance-system in which the skin-tones are used to segment the hands. The segmentation builds the basis of feature extraction for the static and dynamic gestures. In the static gestures, the activation of particular region leads us to associated actions whereas HMM classifier is used to extract the dynamic gestures dependent upon the flow. The assistance-system supports the workers in manual working tasks in the context of assembling complex products. This paper is focused on the interaction of the user with this system and describes the work in progress with the initial results from an application scenario.

[1]  Ayoub Al-Hamadi,et al.  Utilizing Invariant Descriptors for Finger Spelling American Sign Language Using SVM , 2010, ISVC.

[2]  Dieter Schmalstieg,et al.  Robust and unobtrusive marker tracking on mobile phones , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.

[3]  Mokhtar M. Hasan,et al.  Hand Gesture Modeling and Recognition using Geometric Features: A Review , 2012 .

[4]  Rafiqul Zaman Khan,et al.  Survey on Various Gesture Recognition Technologies and Techniques , 2012 .

[5]  Thomas Alexander,et al.  Display Devices for Virtual Environments: Impact on Performance, Workload, and Simulator Sickness , 2008, IPT/EGVE.

[6]  Shamus P. Smith,et al.  Citation for Published Item: Use Policy Evaluating Distributed Cognitive Resources for Wayfinding in a Desktop Virtual Environment , 2022 .

[7]  Alex Waibel,et al.  Readings in speech recognition , 1990 .

[8]  Allan Hanbury,et al.  Color based skin classification , 2012, Pattern Recognit. Lett..

[9]  Willibald A. Günthner,et al.  Pick-by-Vision: A first stress test , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[10]  Gunnar Farnebäck,et al.  Two-Frame Motion Estimation Based on Polynomial Expansion , 2003, SCIA.

[11]  Deborah Hix,et al.  Active Text Drawing Styles for Outdoor Augmented Reality: A User-Based Study and Design Implications , 2007, 2007 IEEE Virtual Reality Conference.

[12]  Bernd Fröhlich,et al.  The GlobeFish and the GlobeMouse: two new six degree of freedom input devices for graphics applications , 2006, CHI.

[13]  Vladimir Vezhnevets,et al.  A Survey on Pixel-Based Skin Color Detection Techniques , 2003 .

[14]  Ayoub Al-Hamadi,et al.  A Novel System for Automatic Hand Gesture Spotting and Recognition in Stereo Color Image , 2009, J. WSCG.

[15]  Rüdiger Mecke,et al.  Mobile Augmented Reality in industrial applications: Approaches for solution of user-related issues , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.

[16]  J. Bhattacharya,et al.  Hand Gesture Recognition for Sign Language : A New Hybrid Approach , 2011 .

[17]  Georgia Albuquerque,et al.  Tangible 3D: hand gesture interaction for immersive 3D modeling , 2005, EGVE'05.

[18]  Frank Dürr,et al.  Efficient real-time trajectory tracking , 2011, The VLDB Journal.