Gyro-aided visual tracking using iterative Earth Mover's Distance

An efficient gyro-aided iterative Earth Mover's Distance (iEMD) algorithm for visual tracking is proposed in this paper. The Earth Mover's Distance (EMD) is used as the similarity measure to search the optimal template candidates in color-spatial space in a video sequence. The computation of the EMD is formulated as the transportation problem from linear programming. The efficiency of this optimization problem limits its use for visual tracking. To efficiently track a target, a monotonically convergent iterative optimization algorithm is developed. Based on the current location of the template candidate, the EMD is calculated and reformulated as the function of the weights of the template candidate. Then the derivative of the EMD with respect to the template displacement is calculated to search for the new position of the target. The iEMD tracking algorithm assumes small inter-frame movement in order to guarantee convergence. When the camera is mounted on a moving robot, e.g., a flying quadcopter, the camera could experience sudden and rapid motion leading to large inter-frame movements. In order to ensure that the tracking algorithm converges, synchronized gyroscope information is utilized to compensate for the rotation of the camera. Three publicly available datasets are used to validate the proposed algorithm. This algorithm is compared with the Mutual Information tracker and the kernel-based Mean-shift tracker by tracking an object undergoing severe illumination changes. The iEMD algorithm outperforms the others in terms of accuracy. The robustness of this algorithm to the ego-motion of the camera is also illustrated.

[1]  Joseph Cohen,et al.  Deep Convolutional Neural Network For Human Detection And Tracking In FLIR Videos , 2016 .

[2]  Kwangsoo Kim,et al.  A novel line of sight control system for a robot vision tracking system, using vision feedback and motion-disturbance feedforward compensation , 2012, Robotica (Cambridge. Print).

[3]  Olivier Stasse,et al.  MonoSLAM: Real-Time Single Camera SLAM , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Shai Avidan,et al.  Ensemble Tracking , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Siddhartha S. Mehta,et al.  Information fusion in human-robot collaboration using neural network representation , 2014, 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[6]  Dorin Comaniciu,et al.  Kernel-Based Object Tracking , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Michael Werman,et al.  Fast and robust Earth Mover's Distances , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[8]  Peter Willett,et al.  Shooting two birds with two bullets: How to find Minimum Mean OSPA estimates , 2010, 2010 13th International Conference on Information Fusion.

[9]  Takeo Kanade,et al.  Gyro-aided feature tracking for a moving camera: fusion, auto-calibration and GPU implementation , 2011, Int. J. Robotics Res..

[10]  Luc Van Gool,et al.  Beyond semi-supervised tracking: Tracking should be as simple as detection, but not simpler than recognition , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[11]  Leonidas J. Guibas,et al.  The Earth Mover's Distance as a Metric for Image Retrieval , 2000, International Journal of Computer Vision.

[12]  George B. Dantzig,et al.  Linear Programming 1: Introduction , 1997 .

[13]  Aristidis Likas,et al.  Visual tracking using the Earth Mover's Distance between Gaussian mixtures and Kalman filtering , 2011, Image Vis. Comput..

[14]  Shai Avidan,et al.  Locally Orderless Tracking , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Ashwin P. Dani,et al.  Human intention inference through interacting multiple model filtering , 2015, 2015 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI).

[16]  Warren E. Dixon,et al.  Single Camera Structure and Motion , 2012, IEEE Transactions on Automatic Control.

[17]  Éric Marchand,et al.  Accurate real-time tracking using mutual information , 2010, 2010 IEEE International Symposium on Mixed and Augmented Reality.

[18]  Warren E. Dixon,et al.  Range and Motion Estimation of a Monocular Camera Using Static and Moving Objects , 2016, IEEE Transactions on Control Systems Technology.

[19]  Jeffery R Gray,et al.  Deeply-Integrated Feature Tracking for Embedded Navigation , 2009 .

[20]  Ben J. A. Kröse,et al.  Approximate Bayesian methods for kernel-based object tracking , 2009, Comput. Vis. Image Underst..

[21]  Zdenek Kalal,et al.  Tracking-Learning-Detection , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Ashwin P. Dani,et al.  Intention Inference for Human-Robot Collaboration in Assistive Robotics , 2017 .

[23]  Haibin Ling,et al.  An Efficient Earth Mover's Distance Algorithm for Robust Histogram Comparison , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Gabriel Peyré,et al.  Iterative Bregman Projections for Regularized Transportation Problems , 2014, SIAM J. Sci. Comput..

[25]  Soon-Jo Chung,et al.  Vision‐based Localization and Robot‐centric Mapping in Riverine Environments , 2017, J. Field Robotics.

[26]  Christopher E. Hann,et al.  Fast normalized cross correlation for motion tracking using basis functions , 2006, Comput. Methods Programs Biomed..

[27]  Marco Cuturi,et al.  Sinkhorn Distances: Lightspeed Computation of Optimal Transport , 2013, NIPS.

[28]  Soon-Jo Chung,et al.  Image moments for higher-level feature based navigation , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[29]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[30]  Ashwin P. Dani,et al.  Gyro-aided image-based tracking using mutual information optimization and user inputs , 2014, 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[31]  Qi Zhao,et al.  Differential Earth Mover's Distance with Its Applications to Visual Tracking , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Uwe D. Hanebeck,et al.  On Wasserstein Barycenters and MMOSPA Estimation , 2015, IEEE Signal Processing Letters.