A generalized motion model for estimating optical flow using 3-D Hermite polynomials

Classic optical flow algorithms assume local image translational motion and apply some primitive image smoothing. Recent studies have taken two separate approaches toward improving accuracy: the application of spatio-temporal filtering schemes and the use of generalized motion models such as the affine model. Each has achieved improvement in its specialized situations. We analyze the interdependency between them and propose a unified theory. The generalized motion we adopt models arbitrary 3D steady motion. Under perspective projection, we derive an image motion equation that describes the spatio-temporal relation in an image sequence, thus making 3D spatio-temporal filtering possible. Hence we establish a theory of Hermite polynomial differentiation filters, whose orthogonality and Gaussian derivative properties ensure numerical stability. The use of higher order motion constraint equations to accommodate more complex motion is justified by the algorithm's reliable performance, as demonstrated by evaluating our algorithm in the scheme established by Barron, et al. (1994).