Gesture recognition using an acceleration sensor and its application to musical performance control

It is understood that nonverbal means of communication are more important than verbal communication in the transmission of intention or emotion. Gestures using body or hand are typical nonverbal means of communication. Human intention or emotion appears to be expressed to a greater extent by the force applied to the body than by the position of the hand. The authors noted that the force acting during motion can be sensed as an acceleration and therefore attempted recognition of gestures with a three-dimensional acceleration sensor. In the experiments using this system, the changes of the acceleration, the rotational force, and the directional distribution of the acceleration are determined from the sequential three-dimensional acceleration data, as features of the motion. By examining the extent of matching to standard patterns, ten kinds of gestures can be discriminated with a nearly 100 percent recognition rate. A real-time music control system is also constructed by applying the proposed method. Compared to the conventional methods using image processing or the data gloves, in the new system the delay in tempo detection is shorter and the mechanism is simpler. The system gives a clue to the design of flexible and sensitive man-machine interfaces based on gesture. © 1997 Scripta Technica, Inc. Electron Comm Jpn Pt 3, 80(5): 9–17, 1997