Skin and bones: multi-layer, locally affine, optical flow and regularization with transparency

This paper describes a new method for estimating optical flow that strikes a balance between the flexibility of local dense computations and the robustness and accuracy of global parameterized flow models. An affine model of image motion is used within local image patches while a spatial smoothness constraint on the affine flow parameters of neighboring patches enforces continuity of the motion. We refer to this as a "Skin and Bones" model in which the affine patches can be thought of as rigid "bones" connected by a flexible "skin". Since local image patches may contain multiple motions we use a layered representation for the affine bones. To regularize this layered motion representation we develop a new framework for regularization with transparency.

[1]  Gilad Adiv,et al.  Determining Three-Dimensional Motion and Structure from Optical Flow Generated by Several Moving Objects , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  David J. Hand,et al.  Mixture Models: Inference and Applications to Clustering , 1989 .

[3]  D. N. Geary.  Mixture Models: Inference and Applications to Clustering , 1989 .

[4]  Shmuel Peleg,et al.  Computing two motions from three frames , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[5]  A. Pentland,et al.  Robust estimation of a multi-layered motion representation , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[6]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.

[7]  Richard Szeliski,et al.  Surface modeling with oriented particle systems , 1992, SIGGRAPH.

[8]  Michael J. Black,et al.  A framework for the robust estimation of optical flow , 1993, 1993 (4th) International Conference on Computer Vision.

[9]  Michael J. Black,et al.  Mixture models for optical flow computation , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Daniel J. Kersten,et al.  Multi-layer surface segmentation using energy minimization , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Edward H. Adelson,et al.  Representing moving images with layers , 1994, IEEE Trans. Image Process..

[12]  Michael J. Black.  Recursive Non-Linear Estimation of Discontinuous Flow Fields , 1994, ECCV.

[13]  A. Jepson,et al.  Estimating multiple independent motions in segmented images using parametric models with local deformations , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[14]  Serge LAyrr.  Layered Representation of Motion Video using Robust Maximum-Likelihood Estimation of Mixture Models and MDL Encoding , 1995 .

[15]  Harpreet S. Sawhney,et al.  Layered representation of motion video using robust maximum-likelihood estimation of mixture models and MDL encoding , 1995, Proceedings of IEEE International Conference on Computer Vision.

[16]  Richard Szeliski,et al.  Motion Estimation with Quadtree Splines , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..