Real time feature point tracking with automatic model selection

We present an efficient and accurate algorithm for face tracking using a set of Active Appearance Models (AAMs). We observe that a single AAM, trained at a particular model resolution and a particular range of displacements, has a “sweet spot” - a range of displacements for which it is most accurate. A common approach to increasing the range of convergence is to use a multi-resolution model, or a sequence of AAMs trained on smaller and smaller displacements. However, during tracking it is inefficient to run the whole sequence at every frame. If there has been little movement since the previous frame, it is sufficient to only run one step of a single higher resolution AAM. In this paper we show that we can use a non-linear regressor to estimate the magnitude of the displacement from the optimal position in the current frame, and use this to select a model which has been tuned to work well at that displacement. This is significantly more efficient than running a complete sequence of models at every frame. We describe the method in detail and demonstrate its performance on several datasets.