A Speech-Driven Hand Gesture Generation Method and Evaluation in Android Robots

Hand gestures commonly occur in daily dialogue interactions, and have important functions in communication. We first analyzed a multimodal human–human dialogue data and found relations between the occurrence of hand gestures and dialogue act categories. We also conducted a clustering analysis on gesture motion data, and associated text information with the gesture motion clusters through gesture function categories. Using the analysis results, we proposed a speech-driven gesture generation method by taking text, prosody, and dialogue act information into account. We then implemented a hand motion control to an android robot, and evaluated the effectiveness of the proposed gesture generation method through subjective experiments. The gesture motions generated by the proposed method were judged to be relatively natural even under the robot hardware constraints.