Towards tracking interaction between people

During face-to-face communication people make use of both verbal and visual behaviors. In this paper we address the problem of tracking visual cues between people. We propose a hybrid approach to tracking who is looking at whom during a discussion or meeting situation. A neural network and a model based gaze tracker are combined to track gaze directions of participants in a meeting. The neural network serves as two functions. First, the neural network coarsely detects gaze direction of a person, i.e., determines if the person is looking at front, or left, or right, or down at the table. Second, the neural network initializes a model based gaze tracker to find out more precise gaze information when a person is in a near front view. The feasibility of the proposed approach has been demonstrated by experiments. The trained neural network has achieved classification accuracy between 82% and 97% for different people. The experimental results have shown significant improvement of robustness for the model based gaze tracker initialized by the neural network.