Range Flow in Varying Illumination: Algorithms and Comparisons

We extend estimation of range flow to handle brightness changes in image data caused by inhomogeneous illumination. Standard range flow computes 3D velocity fields using both range and intensity image sequences. Toward this end, range flow estimation combines a depth change model with a brightness constancy model. However, local brightness is generally not preserved when object surfaces rotate relative to the camera or the light sources, or when surfaces move in inhomogeneous illumination. We describe and investigate different approaches to handle such brightness changes. A straightforward approach is to prefilter the intensity data such that brightness changes are suppressed, for instance, by a highpass or a homomorphic filter. Such prefiltering may, though, reduce the signal-to-noise ratio. An alternative novel approach is to replace the brightness constancy model by 1) a gradient constancy model, or 2) by a combination of gradient and brightness constancy constraints used earlier successfully for optical flow, or 3) by a physics-based brightness change model. In performance tests, the standard version and the novel versions of range flow estimation are investigated using prefiltered or nonprefiltered synthetic data with available ground truth. Furthermore, the influences of additive Gaussian noise and simulated shot noise are investigated. Finally, we compare all range flow estimators on real data.

[1]  Hanno Scharr,et al.  Simultaneous motion, depth and slope estimation with a camera-grid , 2006 .

[2]  Johan Wiklund,et al.  Multidimensional Orientation Estimation with Applications to Texture Analysis and Optical Flow , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Olivier D. Faugeras,et al.  Multi-View Stereo Reconstruction and Scene Flow Estimation with a Global Image-Based Matching Score , 2007, International Journal of Computer Vision.

[4]  Rui Li,et al.  Multi-Scale 3D Scene Flow from Binocular Stereo Sequences , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[5]  Ye Zhang,et al.  Integrated 3D scene flow and structure recovery from multiview image sequences , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[6]  Jerry L. Prince,et al.  Optimal brightness functions for optical flow estimation of deformable motion , 1994, IEEE Trans. Image Process..

[7]  Bernd Jähne,et al.  Range Flow Estimation , 2002, Comput. Vis. Image Underst..

[8]  Takeo Kanade,et al.  Three-dimensional scene flow , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[10]  Hanno Scharr,et al.  Towards a Multi-camera Generalization of Brightness Constancy , 2004, IWCM.

[11]  Frederic Devernay,et al.  A Variational Method for Scene Flow Estimation from Stereo Sequences , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[12]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..

[13]  David J. Fleet,et al.  Likelihood functions and confidence bounds for total-least-squares problems , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[14]  Hanno Scharr,et al.  Optimal Filters for Extended Optical Flow , 2004, IWCM.

[15]  Til Aach,et al.  Illumination-invariant change detection , 2000, 4th IEEE Southwest Symposium on Image Analysis and Interpretation.

[16]  Bernd Jähne,et al.  Surface Expansion from Range Data Sequences , 2001, DAGM-Symposium.

[17]  Hanno Scharr,et al.  Simultaneous Estimation of Surface Motion, Depth and Slopes Under Changing Illumination , 2007, DAGM-Symposium.

[18]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[19]  Hamid Gharavi,et al.  3-D Motion Estimation Using Range Data , 2007, IEEE Transactions on Intelligent Transportation Systems.

[20]  John F. Walkup,et al.  Image Processing In Signal-Dependent Noise , 1974 .

[21]  Michael J. Black,et al.  Skin and bones: multi-layer, locally affine, optical flow and regularization with transparency , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Bernd Jähne,et al.  Differential Range Flow Estimation , 1999, DAGM-Symposium.

[23]  David J. Fleet,et al.  Computing Optical Flow with Physical Models of Brightness Variation , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  A. Oppenheim,et al.  Nonlinear filtering of multiplied and convolved signals , 1968 .

[25]  Hanno Scharr,et al.  Range Flow for Varying Illumination , 2008, ECCV.

[26]  Joachim Weickert,et al.  Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods , 2005, International Journal of Computer Vision.

[27]  Thomas Brox,et al.  Universität Des Saarlandes Fachrichtung 6.1 – Mathematik Highly Accurate Optic Flow Computation with Theoretically Justified Warping Highly Accurate Optic Flow Computation with Theoretically Justified Warping , 2022 .

[28]  Marc Rioux,et al.  Direct Estimation of Range Flow on Deformable Shape From a Video Rate Range Camera , 1993, IEEE Trans. Pattern Anal. Mach. Intell..