The generalization, optimization, and information-theoretic justification of filter-based and autocovariance-based motion estimation

We discuss the theoretical foundations of measuring motion in video data, and relate this strongly to statistical estimation theory. A very general class of motion estimation methods is characterized by determining second order moments of filter bank outputs. These moments are represented in tensors, and motion estimation boils down to analyzing their eigensystems. An alternative approach is to directly estimate and analyze the autocorrelation of the given signal. We provide motivation for developing these approaches further towards directional entropy rate criteria rather than rely on conventional directional smoothness criteria. This pa- per emphasizes that prior knowledge on the video signal (e.g. spatial autocovariance, distribution of expected motion speed, noise spectrum,...) should be integrated into the motion estimation procedure. Relations between different classes of motion algorithms (differential, tensor-based, steerable filters...) are discussed and perspectives for a unification and enhancement of such procedures are presented.

[1]  J. Bigun,et al.  Optimal Orientation Detection of Linear Symmetry , 1987, ICCV 1987.

[2]  Bernd Girod,et al.  Motion-compensating prediction with fractional-pel accuracy , 1993, IEEE Trans. Commun..

[3]  Athanasios Papoulis,et al.  Probability, Random Variables and Stochastic Processes , 1965 .

[4]  Björn Johansson,et al.  A Theoretical Comparison of Different Orientation Tensors , 2002 .

[5]  Carl-Fredrik Westin,et al.  Normalized and differential convolution , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Rudolf Mester,et al.  Subspace Methods and Equilibration in Computer Vision , 1999 .

[7]  C. Westin,et al.  Normalized and differential convolution , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[8]  S. Lippman,et al.  The Scripps Institution of Oceanography , 1959, Nature.

[9]  David J. Fleet,et al.  Likelihood functions and confidence bounds for total-least-squares problems , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[10]  Eero P. Simoncelli Design of multi-dimensional derivative filters , 1994, Proceedings of 1st International Conference on Image Processing.

[11]  Hans Knutsson,et al.  Signal processing for computer vision , 1994 .

[12]  Hagen Spies,et al.  Motion , 2000, Computer Vision and Applications.

[13]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[14]  Maria Huhtala,et al.  Random Variables and Stochastic Processes , 2021, Matrix and Tensor Decompositions in Signal Processing.