Dialogue control in social interface agents

Interface agents are computational entities that form a focal point for communication at the interface; social inter-ace agents are familiar with the conventions of personal interaction, This paper outlines a prototype social interface agent, called J. Jr., that integrates various channels of information about the user to control its real-time behavior in the social setting. Information about the user’s gaze and hand gestures is provided by a human observer; data about intonation in the user’s speech is obtained with automatic frequency analysis. This data is in turn used to control the gaze of the agent’s on-screen face, its back-channel paraverbals, and turn-taking behavior. Results show that by choosing the appropriate variables for dialogue control, a relatively convincing social behavior can be achieved in the agent.