Gesture Recognition System for Hand and Arm Signals

Abstract : This report describes an evaluation of a computer system for recognizing human hand and arm signals as a means of interacting with virtual environments. The system consists of two video cameras, software to track the positions of the gesturer's head and hands. and software to recognize gestures by analyzing the position and movement of the hands. The software was hosted on a standard PC. A set of 14 gestures from Army Field Manual 2l-6O. Visual Signals. was used to test the system. Ten participants individually performed each gesture twice as discrete trials. with a brief rest period between each trial. The average recognition rate was 68%. The highest average recognition rate for an individual was 86%; the lowest was 57%. Three of the 14 gestures were always recognized correctly. and one was never recognized correctly. While no tracking failures occurred for four ofthe gestures, tracking failures ranged from 10% to 100% for the other ten. The system's capabilities for untagged optical tracking and recognition of gestures involving certain types of repetitive motion advance the state-ofthe-art in computer-based gesture recognition. However. for training applications. substantial improvements are needed in tracking reliability and recognition of gestures involving the depth dimension.