Situated facial displays: towards social interaction

Most interactive programs have been assuming interaction with a single user. We propose the notion of "Social Interaction" as a new interaction paradigm between multiple humans and computers. Social interaction requires that first a computer has the multiple participants model, second its behaviors are not only determined by internal logic but also affected by perceived external situations, and finally it actively joins the interaction. An experimental system with these features was developed. It consists of three subsystems, a vision subsystem that processes motion video input to examine an external situation, an action/reaction subsystem that generates an action based on internal logic of a task and a situated reaction triggered by perceived external situation, and a facial animation subsystem that generates a threedimensional face capable of various facial displays. From the experiment using the system with a number of subjects, we found that subjects generally tended to try to interpret facial displays of the computer. Such involvement prevented them from concentrating on a task. We also found that subjects never recognized situated reactions of the computer that were unrelated to the task although they unconsciously responded to them. These findings seem to imply subliminal involvement of the subjects caused by facial displays and situated