Designing Hand Gesture Vocabularies for Natural Interaction by Combining Psycho-Physiological and Recognition Factors

A need exists for intuitive hand gesture machine interaction in which the machine not only recognizes gestures, but also the human feels comfortable and natural in their execution. The gesture vocabulary design problem is rigorously formulated as a multi-objective optimization problem. Psycho-physiological measures (intuitiveness, comfort) and gesture recognition accuracy are taken as the multi-objective factors. The hand gestures are static and recognized by a vision based fuzzy c-means classifier. A meta-heuristic approach decomposes the problem into two sub-problems: finding the subsets of gestures that meet a minimal accuracy requirement, and matching gestures to commands to maximize the human factors objective. The result is a set of Pareto optimal solutions in which no objective may be increased without a concomitant decrease in another. Several solutions from the Pareto set are selected by the user using prioritized objectives. Software programs are developed to automate the collection of intuitive and stress indices. The method is tested for a simulated car — maze navigation task. Validation tests were conducted to substantiate the claim that solutions that maximize intuitiveness, comfort, and recognition accuracy performance measures can be used as proxies for the minimization task time objective. Learning and memorability were also tested.

[1]  Yael Edan,et al.  PARAMETER CALIBRATION FOR RECONFIGURATION OF A HAND GESTURE TELE-ROBOTIC CONTROL SYSTEM , 2004 .

[2]  Jr. Joseph J. LaViola,et al.  Whole-Hand and Speech Input in Virtual Environments , 1999 .

[3]  Mohamed S. Kamel,et al.  Knowledge-based disambiguation of hand gestures , 2002, IEEE International Conference on Systems, Man and Cybernetics.

[4]  Manolis I. A. Lourakis,et al.  Vision-Based Interpretation of Hand Gestures for Remote Control of a Computer Mouse , 2006, ECCV Workshop on HCI.

[5]  J. P. Foley,et al.  Gesture and Environment , 1942 .

[6]  Thomas B. Moeslund,et al.  A Procedure for Developing Intuitive and Ergonomic Gesture Interfaces for HCI , 2003, Gesture Workshop.

[7]  Karin Husballe Munk.  Development of a Gesture Plug-In for Natural Dialogue Interfaces , 2001, Gesture Workshop.

[8]  K N An,et al.  Normative model of human hand for biomechanical analysis. , 1979, Journal of biomechanics.

[9]  Alex Pentland,et al.  Invariant features for 3-D gesture recognition , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[10]  Rick Kjeldsen,et al.  Design issues for vision-based computer interaction systems , 2001, PUI '01.

[11]  Michel Beaudouin-Lafon,et al.  Charade: remote control of objects using free-hand gestures , 1993, CACM.

[12]  Roberto Cipolla,et al.  Human-robot interface by pointing with uncalibrated stereo vision , 1996, Image Vis. Comput..

[13]  G. Borg.  Perceived exertion as an indicator of somatic stress. , 2019, Scandinavian journal of rehabilitation medicine.

[14]  Jochen Triesch,et al.  Robotic Gesture Recognition by Cue Combination , 1998, GI Jahrestagung.

[15]  W. Stokoe,et al.  Semiotics and Human Sign Languages , 1972 .

[16]  William T. Freeman,et al.  Orientation Histograms for Hand Gesture Recognition , 1995 .

[17]  July , 1890, The Hospital.

[18]  R. J. KohlerInformatik.  System Architecture and Techniques for Gesture Recognition in Unconstraint Environments , 1997 .

[19]  David Connolly.  An improved annealing scheme for the QAP , 1990 .

[20]  A. Lecours,et al.  The Biological foundations of gestures : motor and semiotic aspects , 1986 .

[21]  T. Koopmans,et al.  Assignment Problems and the Location of Economic Activities , 1957 .

[22]  Markus Kohler.  System architecture and techniques for gesture recognition in unconstrained environments , 1997, Proceedings. International Conference on Virtual Systems and MultiMedia VSMM '97 (Cat. No.97TB100182).

[23]  T. P. Wright,et al.  Factors affecting the cost of airplanes , 1936 .

[24]  Marcelo Knörich Zuffo,et al.  On the usability of gesture interfaces in virtual reality environments , 2005, CLIHC '05.

[25]  Ying Wu,et al.  Vision-Based Gesture Recognition: A Review , 1999, Gesture Workshop.

[26]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..