Uncertainty Estimation for KLT Tracking

The Kanade-Lucas-Tomasi tracker (KLT) is commonly used for tracking feature points due to its excellent speed and reasonable accuracy. It is a standard algorithm in applications such as video stabilization, image mosaicing, egomotion estimation, structure from motion and Simultaneous Localization and Mapping (SLAM). However, our understanding of errors in the output of KLT tracking is incomplete. In this paper, we perform a theoretical error analysis of KLT tracking. We first focus our analysis on the standard KLT tracker and then extend it to the pyramidal KLT tracker and multiple frame tracking. We show that a simple local covariance estimate is insufficient for error analysis and a Gaussian Mixture Model is required to model the multiple local minima in KLT tracking. We perform Monte Carlo simulations to verify the accuracy of the uncertainty estimates.

[1]  J.-Y. Bouguet,et al.  Pyramidal implementation of the lucas kanade feature tracker , 1999 .

[2]  Simon Baker,et al.  Lucas-Kanade 20 Years On: A Unifying Framework , 2004, International Journal of Computer Vision.

[3]  M. Brooks,et al.  What value covariance information in estimating vision parameters? , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[4]  Stefan K. Gehrig,et al.  Exploiting the Power of Stereo Confidences , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Andrej Pázman,et al.  Nonlinear Regression , 2019, Handbook of Regression Analysis With Applications in R.

[7]  Fredrik Gustafsson,et al.  Statistical Characteristics of Harris Corner Detector , 2007, 2007 IEEE/SP 14th Workshop on Statistical Signal Processing.

[8]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[9]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[10]  Nassir Navab,et al.  Estimation of Location Uncertainty for Scale Invariant Features Points , 2009, BMVC.

[11]  Kevin Nickels,et al.  Estimating uncertainty in SSD-based feature tracking , 2002, Image Vis. Comput..

[12]  X. Rong Li,et al.  Practical Measures and Test for Credibility of an Estimator , 2001 .

[13]  Kenichi Kanatani,et al.  Do we really have to consider covariance matrices for image features? , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[14]  S. Beucher,et al.  Morphological segmentation , 1990, J. Vis. Commun. Image Represent..