Robust Tracking in Low Light and Sudden Illumination Changes

We present an algorithm for robust and real-time visual tracking under challenging illumination conditions characterized by poor lighting as well as sudden and drastic changes in illumination. Robustness is achieved by adapting illumination-invariant binary descriptors to dense image alignment using the Lucas and Kanade algorithm. The proposed adaptation preserves the Hamming distance under least-squares minimization, thus preserving the photometric invariance properties of binary descriptors. Due to the compactness of the descriptor, the algorithm runs in excess of 400 fps on laptops and 100 fps on mobile devices.

[1]  Richard Bowden,et al.  Mutual Information for Lucas-Kanade Tracking (MILK): An Inverse Compositional Formulation , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Simon Lucey,et al.  In Defense of Gradient-Based Alignment on Densely Sampled Sparse Features , 2016 .

[3]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[4]  Matti Pietikäinen,et al.  Unsupervised texture segmentation using feature distributions , 1997, Pattern Recognit..

[5]  P. Anandan,et al.  Robust multi-sensor image alignment , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[6]  G. Schweighofer.  Groupwise Geometric and Photometric Direct Image Registration , 2006 .

[7]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[8]  Stefanos Zafeiriou,et al.  Feature-Based Lucas–Kanade and Active Appearance Models , 2015, IEEE Transactions on Image Processing.

[9]  Michael J. Black,et al.  Optical Flow Estimation with Channel Constancy , 2014, ECCV.

[10]  Georgios D. Evangelidis,et al.  Parametric Image Alignment Using Enhanced Correlation Coefficient Maximization , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[12]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[13]  Tobias Höllerer,et al.  Evaluation of Interest Point Detectors and Feature Descriptors for Visual Tracking , 2011, International Journal of Computer Vision.

[14]  Vincent Lepetit,et al.  Robust 3D Tracking with Descriptor Fields , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  TurkMatthew,et al.  Evaluation of Interest Point Detectors and Feature Descriptors for Visual Tracking , 2011 .

[16]  Rudolf Mester,et al.  Illumination-Robust Dense Optical Flow Using Census Signatures , 2011, DAGM-Symposium.

[17]  Richard W. Hamming,et al.  Error detecting and error correcting codes , 1950 .

[18]  M. Chiani.  Error Detecting and Error Correcting Codes , 2012 .

[19]  Tomasz Kornuta,et al.  Performance Evaluation of Binary Descriptors of Local Features , 2014, ICCVG.

[20]  Roland Siegwart,et al.  BRISK: Binary Robust invariant scalable keypoints , 2011, 2011 International Conference on Computer Vision.

[21]  Jitendra Malik,et al.  Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Takeo Kanade,et al.  Realtime affine-photometric KLT feature tracker on GPU in CUDA framework , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[23]  Jan-Michael Frahm,et al.  Fast gain-adaptive KLT tracking on the GPU , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[24]  Jian Sun,et al.  Face Alignment at 3000 FPS via Regressing Local Binary Features , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Heiko Hirschmüller,et al.  Evaluation of Stereo Matching Costs on Images with Radiometric Differences , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Erkan Bostanci.  Is Hamming distance the only way for matching binary image feature descriptors? , 2015, ArXiv.

[27]  Geraldo F. Silveira,et al.  Real-time Visual Tracking under Arbitrary Illumination Changes , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[29]  Timothy F. Cootes,et al.  Active Appearance Models , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Ramin Zabih,et al.  Non-parametric Local Transforms for Computing Visual Correspondence , 1994, ECCV.

[31]  John Iselin Woodfill,et al.  Tyzx DeepSea High Speed Stereo Vision System , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[32]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[33]  Alois Knoll,et al.  Mutual Information-Based 3D Object Tracking , 2008, International Journal of Computer Vision.

[34]  Fridtjof Stein,et al.  Efficient Computation of Optical Flow Using the Census Transform , 2004, DAGM-Symposium.

[35]  Laura Sevilla-Lara,et al.  Distribution fields for tracking , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Joachim Weickert,et al.  Why Is the Census Transform Good for Robust Optic Flow Computation? , 2013, SSVM.

[37]  Krista A. Ehinger,et al.  SUN database: Large-scale scene recognition from abbey to zoo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[38]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[39]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[40]  Konrad Schindler,et al.  An Evaluation of Data Costs for Optical Flow , 2013, GCPR.

[41]  Jan-Michael Frahm,et al.  Comparative Evaluation of Binary Features , 2012, ECCV.

[42]  P. Anandan,et al.  About Direct Methods , 1999, Workshop on Vision Algorithms.

[43]  Ethan Rublee,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[44]  Simon Baker,et al.  Lucas-Kanade 20 Years On: A Unifying Framework , 2004, International Journal of Computer Vision.

[45]  Éric Marchand,et al.  Accurate real-time tracking using mutual information , 2010, 2010 IEEE International Symposium on Mixed and Augmented Reality.