Computing optical flow with physical models of brightness variation

This paper exploits physical models of time-varying brightness in image sequences to estimate optical flow and physical parameters of the scene. Previous approaches handled violations of brightness constancy with the use of robust statistics or with generalized brightness constancy constraints that allow generic types of contrast and illumination changes. We consider models of brightness variation that have time-dependent physical causes, namely, changing surface orientation with respect to a directional illuminant, motion of the illuminant, and physical models of heat transport in infrared images. We simultaneously estimate the optical flow and the relevant physical parameters. The estimation problem is formulated using total least squares (TLS), with confidence bounds on the parameters.

[1]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[2]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[3]  Takeo Kanade,et al.  Adapting optical-flow to measure object motion in reflectance and x-ray image sequences (abstract only) , 1984, COMG.

[4]  W. Hauser.  On planetary motion , 1985 .

[5]  Berthold K. P. Horn.  Robot vision , 1986, MIT electrical engineering and computer science series.

[6]  HANS-HELLMUT NAGEL,et al.  On a Constraint Equation for the Estimation of Displacement Rates in Image Sequences , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Tomaso A. Poggio,et al.  Motion Field and Optical Flow: Qualitative Properties , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  David W. Murray,et al.  Experiments in the machine interpretation of visual motion , 1990 .

[9]  Naoki Mukawa.  Estimation of shape, reflection coefficients and illuminant direction from image sequences , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[10]  Hans-Hellmut Nagel,et al.  Extending the 'Oriented Smoothness Constraint' into the Temporal Domain and the Estimation of Derivatives of Optical Flow , 1990, ECCV.

[11]  Sabine Van Huffel,et al.  Total least squares problem - computational aspects and analysis , 1991, Frontiers in applied mathematics.

[12]  Michael J. Black,et al.  Robust dynamic motion estimation over time , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.

[14]  Vishal Markandey,et al.  Total least squares fitting spatiotemporal derivatives to smooth optical flow fields , 1992, Defense, Security, and Sensing.

[15]  David J. Fleet,et al.  Stability of Phase Information , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  David J. Fleet,et al.  Recursive Filters for Optical Flow , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  G. Stewart.  Errors in variables for numerical analysts , 1997 .

[18]  Nagaraj Nandhakumar,et al.  Thermophysical Algebraic Invariants from Infrared Imagery for Object Recognition , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Naoya Ohta.  Optical Flow Detection Using a General Noise Model for Gradient Constraint , 1997, CAIP.

[20]  Hanno Scharr,et al.  Study of Dynamical Processes with Tensor-Based Spatiotemporal Image Processing Techniques , 1998, ECCV.

[21]  Rudolf Mester,et al.  The Role of Total Least Squares in Motion Analysis , 1998, ECCV.

[22]  Shahriar Negahdaripour,et al.  Revised Definition of Optical Flow: Integration of Radiometric and Geometric Cues for Dynamic Scene Analysis , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Gregory D. Hager,et al.  Efficient Region Tracking With Parametric Models of Geometry and Illumination , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Shree K. Nayar,et al.  Vision in bad weather , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[25]  B. Jähne,et al.  Handbook of Computer Vision and Applications: Volume 1: From Scenes to Images , 1999 .

[26]  Lihi Zelnik-Manor,et al.  Multi-frame alignment of planes , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[27]  W. James MacLean.  Removal of translation bias when using subspace methods , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[28]  Motion , 2000, Computer Vision and Applications.

[29]  David J. Fleet,et al.  Likelihood functions and confidence bounds for total-least-squares problems , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[30]  H. Haussecker.  Simultaneous estimation of optical flow and heat transport in infrared image sequences , 2000, Proceedings IEEE Workshop on Computer Vision Beyond the Visible Spectrum: Methods and Applications (Cat. No.PR00640).

[31]  Richard P. Wildes,et al.  Recovering Estimates of Fluid Flow from Image Sequence Data , 2000, Comput. Vis. Image Underst..

[32]  Peter Meer,et al.  A general method for Errors-in-Variables problems in computer vision , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[33]  David J. Fleet,et al.  Robustly Estimating Changes in Image Appearance , 2000, Comput. Vis. Image Underst..