Estimating Image Motion in Layers: The “Skin and Bones” Model

This paper describes a method for estimating optical flow that strikes a balance between the flexibility of regularization techniques and the robustness and accuracy of area-based regression techniques. The approach assumes that image motion can be represented by an affine flow model within local image patchs. Since some image regions may not have sufficient information to estimate an affine motion model robustly, we define a spatial smoothness constraint on the affine flow parameters of neighboring patches. We refer to this as a “Skin and Bones” model in which the affine patches can be thought of as rigid patches of “bone” connected by a flexible “skin.” Since local image patches may contain multiple motions we use a layered representation for the affine bones. With the possibility of multiple motions at a given point, standard regularization schemes cannot be used to smooth the multiple sets of affine parameters. We therefore develop a new framework for regularization with transparency that can applied to produce a smoothed layered motion representation. The motion estimation problem, with layered locally affine bones and transparent regularization, is formulated as an objective function that is minimized using a variant of the EM-algorithm. Experiments with synthetic and natural images are provided throughout the paper to illustrate the method. Submitted as a Regular Paper.