Point matching under large image deformations and illumination changes

To solve the general point correspondence problem in which the underlying transformation between image patches is represented by a homography, a solution based on extensive use of first order differential techniques is proposed. We integrate in a single robust M-estimation framework the traditional optical flow method and matching of local color distributions. These distributions are computed with spatially oriented kernels in the 5D joint spatial/color space. The estimation process is initiated at the third level of a Gaussian pyramid, uses only local information, and the illumination changes between the two images are also taken into account. Subpixel matching accuracy is achieved under large projective distortions significantly exceeding the performance of any of the two components alone. As an application, the correspondence algorithm is employed in oriented tracking of objects.

[1]  Reinhard Klette,et al.  Quantitative color optical flow , 2002, Object recognition supported by user interaction for service robots.

[2]  Philip H. S. Torr,et al.  IMPSAC: Synthesis of Importance Sampling and Random Sample Consensus , 2000, ECCV.

[3]  Peter Meer,et al.  Registration via direct methods: a statistical approach , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[4]  Jonathan W. Brandt,et al.  Improved Accuracy in Gradient-Based Optical Flow Estimation , 1997, International Journal of Computer Vision.

[5]  Luc Van Gool,et al.  Comparing Intensity Transformations and Their Invariants in the Context of Color Pattern Recognition , 2002, ECCV.

[6]  Simon Baker,et al.  Equivalence and efficiency of image alignment algorithms , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[7]  Andrew Zisserman,et al.  Multi-view Matching for Unordered Image Sets, or "How Do I Organize My Holiday Snaps?" , 2002, ECCV.

[8]  P. Anandan,et al.  Robust multi-sensor image alignment , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[9]  Wojciech Chojnacki,et al.  From FNS to HEIV: a link between two vision parameter estimation methods , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[11]  Bing Wang,et al.  The Localized Consistency Principle for Image Matching under Non-uniform Illumination Variation and Affine Distortion , 2002, ECCV.

[12]  Kevin Nickels,et al.  Estimating uncertainty in SSD-based feature tracking , 2002, Image Vis. Comput..

[13]  G D Finlayson,et al.  Color constancy at a pixel. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[14]  Josef Kittler,et al.  An active mesh based tracker for improved feature correspondences , 2002, Pattern Recognit. Lett..

[15]  Venu Madhav Govindu,et al.  Alignment Using Distributions of Local Geometric Properties , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Stefan Carlsson,et al.  Wide Baseline Point Matching Using Affine Invariants Computed from Intensity Profiles , 2000, ECCV.

[17]  Kenichi Kanatani,et al.  Do we really have to consider covariance matrices for image features? , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[18]  Luc Van Gool,et al.  Noncombinatorial Detection of Regular Repetitions under Perspective Skew , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Peter J. Rousseeuw,et al.  Robust regression and outlier detection , 1987 .

[20]  Azriel Rosenfeld,et al.  Computer Vision , 1988, Adv. Comput..

[21]  Jitendra Malik,et al.  Robust computation of optical flow in a multi-scale differential framework , 2005, International Journal of Computer Vision.

[22]  Gregory D. Hager,et al.  Real-time tracking of image regions with changes in geometry and illumination , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Jie Wei,et al.  Illumination-invariant image retrieval and video segmentation , 1999, Pattern Recognit..

[24]  Cordelia Schmid,et al.  3D object modeling and recognition using affine-invariant patches and multi-view spatial constraints , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[25]  Luc Van Gool,et al.  Wide Baseline Stereo Matching based on Local, Affinely Invariant Regions , 2000, BMVC.

[26]  Arnold W. M. Smeulders,et al.  Content-based image retrieval by viewpoint-invariant color indexing , 1999, Image Vis. Comput..

[27]  Yiannis Aloimonos,et al.  The Statistics of Optical Flow , 2001, Comput. Vis. Image Underst..

[28]  Dorin Comaniciu,et al.  Real-time tracking of non-rigid objects using mean shift , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[29]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[30]  Stepán Obdrzálek,et al.  Local affine frames for wide-baseline stereo , 2002, Object recognition supported by user interaction for service robots.

[31]  J. Odobez,et al.  Robust mul-tiresolution estimation of parametric motion models applied to complex scenes , 1994 .

[32]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Kenichi Kanatani,et al.  Automatic Thresholding For Correspondence Detection , 2004, Int. J. Image Graph..

[34]  Kohji Fukunaga,et al.  Introduction to Statistical Pattern Recognition-Second Edition , 1990 .

[35]  Patrick Bouthemy,et al.  Computation and analysis of image motion: A synopsis of current problems and methods , 1996, International Journal of Computer Vision.

[36]  David Suter,et al.  Robust Optic Flow Computation , 1998, International Journal of Computer Vision.

[37]  Andrew Zisserman,et al.  Wide baseline stereo matching , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[38]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.

[39]  Jean-Marc Odobez,et al.  Robust Multiresolution Estimation of Parametric Motion Models , 1995, J. Vis. Commun. Image Represent..

[40]  Gérard G. Medioni,et al.  Perceptual Grouping from Motion Cues Using Tensor Voting in 4-D , 2002, ECCV.

[41]  P. J. Green,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[42]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[43]  Pavel Pudil,et al.  Introduction to Statistical Pattern Recognition , 2006 .

[44]  Richard Szeliski,et al.  Spline-Based Image Registration , 1997, International Journal of Computer Vision.

[45]  Wojciech Chojnacki,et al.  What Value Covariance Information in Estimating Vision Parameters? , 2001, ICCV.

[46]  Jitendra Malik,et al.  Geometric blur for template matching , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[47]  F. Mosteller,et al.  Exploring Data Tables, Trends and Shapes. , 1988 .

[48]  P. J. Huber.  Robust Statistical Procedures , 1977 .

[49]  Pascal Fua,et al.  Self-Consistency and MDL: A Paradigm for Evaluating Point-Correspondence Algorithms, and Its Application to Detecting Changes in Surface Elevation , 2004, International Journal of Computer Vision.

[50]  Isaac Weiss,et al.  Smoothed differentiation filters for images , 1992, J. Vis. Commun. Image Represent..

[51]  Stefan Carlsson,et al.  Combining Appearance and Topology for Wide Baseline Matching , 2002, ECCV.

[52]  Cordelia Schmid,et al.  An Affine Invariant Interest Point Detector , 2002, ECCV.

[53]  Peter J. Huber,et al.  Robust Statistical Procedures: Second Edition , 1996 .

[54]  Andrew Zisserman,et al.  MLESAC: A New Robust Estimator with Application to Estimating Image Geometry , 2000, Comput. Vis. Image Underst..

[55]  Emanuele Trucco,et al.  Making good features track better , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[56]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[57]  Hans-Hellmut Nagel,et al.  Optical flow estimation and the interaction between measurement errors at adjacent pixel positions , 1995, International Journal of Computer Vision.

[58]  Joe Brewer,et al.  Kronecker products and matrix calculus in system theory , 1978 .

[59]  Cordelia Schmid,et al.  Evaluation of Interest Point Detectors , 2000, International Journal of Computer Vision.

[60]  Nikolas P. Galatsanos,et al.  Optical flow estimation from noisy data using differential techniques , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[61]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[62]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[63]  P. Meer,et al.  Heteroscedastic errors-in-variables models in computer vision , 2001 .

[64]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..

[65]  Peter Meer,et al.  A general method for Errors-in-Variables problems in computer vision , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[66]  金谷 健一.  Statistical optimization for geometric computation : theory and practice , 2005 .

[67]  Andrew Zisserman,et al.  Multiple view geometry in computer visiond , 2001 .

[68]  Emanuele Trucco,et al.  Improving Feature Tracking with Robust Statistics , 1999, Pattern Analysis & Applications.

[69]  Victor Solo,et al.  Errors-in-variables modeling in optical flow estimation , 2001, IEEE Trans. Image Process..