Initial Assessment of Job Interview Training System using Multimodal Behavior Analysis

The COVID-19 pandemic has had a significant socio-economic impact on the world. Specifically, social distancing has impacted many activities that were previously conducted face-to-face. One of these is the training that students receive for job interviews. Thus, we introduce a job interview training system that will give students the ability to continue receiving this type of training. Our system recognizes the nonverbal behaviors of an interviewee, namely gaze, facial expression, and posture using a Tobii eye tracker and cameras. The system compares the recognition results with those of models of exemplary nonverbal behaviors of an interviewee and highlights the behaviors that need improvement while playing back the interview recording. Most current interview training systems require high-end Hardware and Software and are not designed for general users, and there are few systems using CG agents to give feedback. The purpose of this study was to conduct an initial evaluation of the usefulness of this system and to identify areas for improvement. The results of the initial evaluation of the system indicate that improvements in the recognition accuracy of nonverbal behaviors and the quality of the interaction with the CG agent are needed.