Modelling of emotional facial expressions during speech in synthetic talking heads using a hybrid approach

Talking heads and virtual characters, able to commu nicate complex information with human-like expressiveness and naturalness, should be able to display emotional fa cial expressions. In recent years several works, both ru le-based and statistical, have obtained important results in the modelling of emotional facial expressions to be use d in synthetic Talking Heads. However, most of the rulebased systems suffer from the drawback of static generati on, due to the fact that the set of rules and their combinatio ns are limited. On the other hand, most of works on synthe sis using statistical approaches are restricted to speech and lip movements. This paper presents a modelling of the dynamics of emotional facial expressions based on a hybrid statistical/machine learning approach. This approac h combines Hidden Markov Models (HMMs) and Recurrent Neural Networks (RNNs), aiming at benefiting from t he advantages of both paradigms and overcoming their o wn limitations.