Automatic real-time FACS-coder to anonymise drivers in eye tracker videos

Driver's face is a rich source of information for understanding driver behaviour. From the driver's face, one could get an idea of the driver's emotional state and where s/he looks at. In recent years, naturalistic driving studies and field operational tests have been conducted to collect driver behavioural data, which often includes video of the driver, from many drivers driving for an extended period of time. Due to the Data Privacy Act, it is desirable to make the driver video anonymous, while preserving the original facial expressions. This paper describes our attempt to make a system that could do so. The system is a combination of an automatic Facial Action Coding System (FACS) coder based on Active Appearance Models (AAMs), a classifier that analyses local deformations in the AAM shape mesh and a 3D visualisation. The image acquisition hardware is based on a SmartEye eye tracker installed in a vehicle. The eye tracker we used provides a constant image quality independent of external illumination, which is a precondition for deploying the system in a vehicle environment. While the system uses Action Unit (AU) activations internally, the evaluation was done using the six basic emotions.

[1]  Bernd Fritzke,et al.  A Growing Neural Gas Network Learns Topologies , 1994, NIPS.

[2]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[3]  Aggelos K. Katsaggelos,et al.  Automatic facial expression recognition using facial animation parameters and multistream HMMs , 2006, IEEE Transactions on Information Forensics and Security.

[4]  Thomas S. Huang,et al.  Capturing subtle facial motions in 3D face tracking , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[5]  Marian Stewart Bartlett,et al.  Classifying Facial Actions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Timothy F. Cootes,et al.  Active Appearance Models , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  J. Cohn,et al.  Automated face analysis by feature point tracking has high concurrent validity with manual FACS coding. , 1999, Psychophysiology.

[8]  Takeo Kanade,et al.  Recognizing Action Units for Facial Expression Analysis , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[10]  Anastasios A. Economides,et al.  Measuring instant emotions during a self-assessment test: the use of FaceReader , 2010, MB '10.

[11]  Hichem Sahli,et al.  Automatic recognition of lower facial action units , 2010, MB '10.

[12]  M. Pantic,et al.  Facial action unit recognition using temporal templates , 2004, RO-MAN 2004. 13th IEEE International Workshop on Robot and Human Interactive Communication (IEEE Catalog No.04TH8759).

[13]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[14]  Simon Baker,et al.  Active Appearance Models Revisited , 2004, International Journal of Computer Vision.

[15]  Ioannis Pitas,et al.  Facial Expression Recognition in Image Sequences Using Geometric Deformation Features and Support Vector Machines , 2007, IEEE Transactions on Image Processing.