Multi-modal Person Identification

Multi-modal Person Identification (MMPI) is a software technology for multi-modal recognition of persons according to their faces and the way they walk. The recognition process is based on analysis of detected face images and motion capture data — both acquired by motion capturing devices, including popular Microsoft Kinect and ASUS Xtion. The acquired data are firstly preprocessed by (1) detecting face images and extracting MPEG-7 face features and (2) detecting walk cycles and extracting movement features in form of relative velocities of the specific joints for each walk cycle. Both the face and walk-cycle features extracted from a query motion are used to retrieve two independent sets of the most similar faces and walk cycles. These retrieved sets are processed by a multi-modal classification method to recognize the query person identity. The MMPI technology is demonstrated via a web application that allows users to select a query motion and verify whether the technology recognizes the query person correctly.