Speech-driven embodied entrainment character system with hand motion input in mobile environment

InterActor is a speech-input-driven CG-embodied interaction character that can generate communicative movements and actions for entrained interactions. InterPuppet, on the other hand, is an embodied interaction character that is driven by both speech input-similar to InterActor-and hand motion input, like a puppet. Therefore, humans can use InterPuppet to effectively communicate by using deliberate body movements and natural communicative movements and actions. In this paper, an advanced InterPuppet system that uses a cellular-phone-type device is developed, which can be used in a mobile environment. The effectiveness of the system is demonstrated by performing a sensory evaluation experiment in an actual remote communication scenario.

[1]  Tomio Watanabe,et al.  Development of an embodied interaction system with InterActor by speech and hand motion input , 2005, ROMAN 2005. IEEE International Workshop on Robot and Human Interactive Communication, 2005..

[2]  Bruce Blumberg,et al.  Sympathetic interfaces: using a plush toy to direct synthetic characters , 1999, CHI '99.

[3]  Tomoko Yonezawa,et al.  HandySinger: Expressive Singing Voice Morphing using Personified Hand-puppet Interface , 2005, NIME.

[4]  Youn-Kyung Lim,et al.  Physical interaction and multi-aspect representation for information intensive environments , 2000, Proceedings 9th IEEE International Workshop on Robot and Human Interactive Communication. IEEE RO-MAN 2000 (Cat. No.00TH8499).

[5]  Tomio Watanabe,et al.  InterActor: Speech-Driven Embodied Interactive Actor , 2004, Int. J. Hum. Comput. Interact..