This paper explores the use of fast, simple computer vision techniques to add compelling visual capabilities to social user interfaces. Social interfaces involve the user in natural dialog with animated, "lifelike" characters. However, current systems employ spoken language as the only input modality. Used effectively, vision can greatly enhance the user's experience interacting with these characters. In addition, vision can provide key information to help manage the dialog and to aid the speech recognition process. We describe constraints imposed by the conversational environment and present a set of "interactive-time" vision routines that begin to support the user's expectations of a seeing character. A control structure is presented which chooses among the vision routines based on the current state of the character, the conversation and the visual environment. These capabilities are beginning to be integrated into the Persona lifelike character project.
[1]
Alex Pentland,et al.
Pfinder: Real-Time Tracking of the Human Body
,
1997,
IEEE Trans. Pattern Anal. Mach. Intell..
[2]
Markus A. Stricker,et al.
Similarity of color images
,
1995,
Electronic Imaging.
[3]
Matthew Stone,et al.
Modeling the Interaction between Speech and Gesture.
,
1994
.
[4]
Ramesh Jain,et al.
Storage and Retrieval for Image and Video Databases III
,
1995
.
[5]
Alex Pentland,et al.
Interactive-time vision: face recognition as a visual behavior
,
1991
.
[6]
M. Turk,et al.
Eigenfaces for Recognition
,
1991,
Journal of Cognitive Neuroscience.