FM: Flexible mapping from one gesture to multiple semantics

Abstract An algorithm for flexible mapping (FM) from one gesture to multiple semantics in the same situational context is presented for the first time to reduce the cognitive and operating loads of an operator by using gesture commands. First, the foundation of the FM is built with the semantic-oriented difference features of behavior model (SDFBM) of the operator as implicit inputs. Second, the gesture–semantics FM algorithm and its extended process, namely, attribute classification for FM are proposed and implemented. Third, five commonly used gestures are designed to demonstrate the way a gesture is mapped to multiple semantics. Finally, several comparative experimental results are provided to demonstrate the superiority of the proposed methods to the state of the art. The main innovation of this study is that the same gesture in the same context can be mapped to several different semantics through the SDFBM feature recognition. This study provides an intelligent and natural interaction interface model for 3D platforms and key support for gesture-based implicit interaction design. The proposed algorithms are also tested or used in several applications, such as smart teaching interface and onboard vehicular systems and intelligent TV.

[1]  Zhiquan Feng,et al.  An HCI paradigm fusing flexible object selection and AOM-based animation , 2016, Inf. Sci..

[2]  Tiago Dias,et al.  Gesture interaction system for social web applications on smart TVs , 2013, OAIR.

[3]  Ugur Güdükbay,et al.  A hand gesture recognition technique for human-computer interaction , 2015, J. Vis. Commun. Image Represent..

[4]  Radu-Daniel Vatavu,et al.  Gestures as point clouds: a $P recognizer for user interface prototypes , 2012, ICMI '12.

[5]  Joochan Sohn,et al.  Long-range touch gesture interface for Smart TV , 2011, 2011 6th International Conference on Computer Sciences and Convergence Information Technology (ICCIT).

[6]  Dmitry B. Goldgof,et al.  Gesture recognition using Bezier curves for visualization navigation from registered 3-D data , 2004, Pattern Recognit..

[7]  Radu-Daniel Vatavu,et al.  Leap gestures for TV: insights from an elicitation study , 2014, TVX.

[8]  Xindong Wu,et al.  RIEVL: Recursive Induction Learning in Hand Gesture Recognition , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Daniel P. Huttenlocher,et al.  Comparing Images Using the Hausdorff Distance , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Thomas Mitchell,et al.  SoundGrasp: A Gestural Interface for the Performance of Live Music , 2011, NIME.

[11]  Ming C. Leu,et al.  Human-Computer Interaction System with Artificial Neural Network Using Motion Tracker and Data Glove , 2005, PReMI.

[12]  Daniel Thalmann,et al.  Model-based hand pose estimation via spatial-temporal hand parsing and 3D fingertip localization , 2013, The Visual Computer.

[13]  KwangYun Wohn,et al.  Recognition of space-time hand-gestures using hidden Markov model , 1996, VRST.

[14]  Luca Benini,et al.  Reconfigurable natural interaction in smart environments: approach and prototype implementation , 2011, Personal and Ubiquitous Computing.

[15]  Gerhard Rigoll,et al.  High Performance Real-Time Gesture Recognition Using Hidden Markov Models , 1997, Gesture Workshop.

[16]  Wendy Yee.  Potential Limitations of Multi-touch Gesture Vocabulary: Differentiation, Adoption, Fatigue , 2009, HCI.

[17]  Zhi Quan Feng,et al.  A Gesture Recognition Algorithm Using Hausdorff-Like Distance Template Matching Based on the Main Direction of Gesture , 2015 .

[18]  Sebastian Möller,et al.  I'm home: Defining and evaluating a gesture set for smart-home control , 2011, Int. J. Hum. Comput. Stud..

[19]  Donghun Lee,et al.  Towards successful user interaction with systems: focusing on user-derived gestures for smart home systems. , 2014, Applied ergonomics.

[20]  박원규.  A Multi-touch Gesture Vocabulary Design Methodology for Mobile Devices , 2012 .

[21]  Claudio S. Pinhanez,et al.  Dynamically reconfigurable vision-based user interfaces , 2004, Machine Vision and Applications.

[22]  Wu-Chih Hu,et al.  Gabor filter-based hand-pose angle estimation for hand gesture recognition under varying illumination , 2011, Expert Syst. Appl..

[23]  Mohammed Yeasin,et al.  Visual understanding of dynamic hand gestures , 2000, Pattern Recognit..

[24]  Radu-Daniel Vatavu,et al.  User-defined gestures for free-hand TV control , 2012, EuroITV.

[25]  Martin Saerbeck,et al.  Recent methods and databases in vision-based hand gesture recognition: A review , 2015, Comput. Vis. Image Underst..

[26]  Thi Thanh Mai Nguyen,et al.  A fully automatic hand gesture recognition system for human-robot interaction , 2011, SoICT.

[27]  Yang Li,et al.  Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes , 2007, UIST.

[28]  Jan Alexandersson,et al.  A Gesture Based System for Context – Sensitive Interaction with Smart Homes , 2011 .

[29]  Alexander Zelinsky,et al.  Visual gesture interfaces for virtual environments , 2002, Interact. Comput..

[30]  Radu-Daniel Vatavu.  A comparative study of user-defined handheld vs. freehand gestures for home entertainment environments , 2013, J. Ambient Intell. Smart Environ..

[31]  Robert J. K. Jacob,et al.  Eye Movement-Based Human-Computer Interaction Techniques: Toward Non-Command Interfaces , 2003 .

[32]  Jin-Hyung Kim,et al.  An HMM-Based Threshold Model Approach for Gesture Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Gerhard Rigoll,et al.  Hidden Markov model based continuous online gesture recognition , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[34]  David Rempel,et al.  The design of hand gestures for human-computer interaction: Lessons from sign language interpreters , 2014, Int. J. Hum. Comput. Stud..

[35]  Jean-Claude Martin,et al.  The effects of speech-gesture cooperation in animated agents' behavior in multimedia presentations , 2007, Interact. Comput..

[36]  Sang-Heon Lee,et al.  Smart TV interaction system using face and hand gesture recognition , 2013, 2013 IEEE International Conference on Consumer Electronics (ICCE).

[37]  Niels Henze,et al.  Gesture recognition with a Wii controller , 2008, TEI.

[38]  Sébastien Marcel,et al.  Hand gesture recognition using input-output hidden Markov models , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[39]  Ivan Poupyrev,et al.  Virtual object manipulation on a table-top AR environment , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[40]  Yimin Zhou,et al.  A novel finger and hand pose estimation technique for real-time hand gesture recognition , 2016, Pattern Recognit..

[41]  Chung-Lin Huang,et al.  Hand gesture recognition using a real-time tracking method and hidden Markov models , 2003, Image Vis. Comput..

[42]  Helge J. Ritter,et al.  Visual recognition of continuous hand postures , 2002, IEEE Trans. Neural Networks.