HMPMR strategy for real-time tracking in aerial images, using direct methods

The vast majority of approaches make use of features to track objects. In this paper, we address the tracking problem with a tracking-by-registration strategy based on direct methods. We propose a hierarchical strategy in terms of image resolution and number of parameters estimated in each resolution, that allows direct methods to be applied in demanding real-time visual-tracking applications. We have called this strategy the Hierarchical Multi-Parametric and Multi-Resolution strategy (HMPMR). The Inverse Composition Image Alignment Algorithm (ICIA) is used as an image registration technique and is extended to an HMPMR-ICIA. The proposed strategy is tested with different datasets and also with image data from real flight tests using an Unmanned Aerial Vehicle, where the requirements of direct methods are easily unsatisfied (e.g. vehicle vibrations). Results show that using an HMPMR approach, it is possible to cope with the efficiency problem and with the small motion constraint of direct methods, conducting the tracking task at real-time frame rates and obtaining a performance that is comparable to, or even better than, the one obtained with the other algorithms that were analyzed.

[1]  Andrew Zisserman,et al.  Feature Based Methods for Structure and Motion Estimation , 1999, Workshop on Vision Algorithms.

[2]  Enrique Muñoz Corral.  Efficient Model-based 3D Tracking by Using Direct Image Registration , 2012 .

[3]  Selim Benhimane,et al.  A unified approach to visual tracking and servoing , 2005, Robotics Auton. Syst..

[4]  Alex Zelinsky,et al.  Learning OpenCV---Computer Vision with the OpenCV Library (Bradski, G.R. et al.; 2008)[On the Shelf] , 2009, IEEE Robotics & Automation Magazine.

[5]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[6]  Éric Marchand,et al.  Chasing a moving target from a flying UAV , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[7]  Robin Hess.  An Open-Source SIFT Library , 2010 .

[8]  P. Campoy,et al.  Visual Model Feature Tracking For UAV Control , 2007, 2007 IEEE International Symposium on Intelligent Signal Processing.

[9]  Jiri Matas,et al.  Tracking by an Optimal Sequence of Linear Predictors , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Jan Flusser,et al.  Image registration methods: a survey , 2003, Image Vis. Comput..

[11]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Supun Samarasekera,et al.  Aerial video surveillance and exploitation , 2001, Proc. IEEE.

[13]  Frédéric Dufaux,et al.  Efficient, robust, and fast global motion estimation for video coding , 2000, IEEE Trans. Image Process..

[14]  Getian Ye.  Image registration and super-resolution mosaicing , 2005 .

[15]  Richard Szeliski,et al.  Creating full view panoramic image mosaics and environment maps , 1997, SIGGRAPH.

[16]  Fei Yuan,et al.  Vehicle Tracking Based on Image Alignment in Aerial Videos , 2007, EMMCVPR.

[17]  Miguel A. Olivares-Méndez,et al.  3D pose estimation based on planar object tracking for UAVs control , 2010, 2010 IEEE International Conference on Robotics and Automation.

[18]  Rob Hess,et al.  An open-source SIFTLibrary , 2010, ACM Multimedia.

[19]  C. Radke.  International Conference on Computer Design , 2022 .

[20]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[21]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[22]  Gregory D. Hager,et al.  Efficient Region Tracking With Parametric Models of Geometry and Illumination , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Gaurav S. Sukhatme,et al.  A visual servoing approach for tracking features in urban areas using an autonomous helicopter , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[24]  Radka Turcajova,et al.  Hierarchical multiresolution technique for image registration , 1996, Optics & Photonics.

[25]  Richard Szeliski,et al.  Construction of Panoramic Image Mosaics with Global and Local Alignment , 2001 .

[26]  Charles V. Stewart,et al.  A Feature-Based, Robust, Hierarchical Algorithm for Registering Pairs of Images of the Curved Human Retina , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Yaser Sheikh,et al.  Feature-based Georegistration of Aerial Images , 2004 .

[28]  Takeo Kanade,et al.  Inertial-aided KLT feature tracking for a moving camera , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[29]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[30]  H. Ishiguro,et al.  Panoramic Vision , 2001, Monographs in Computer Science.

[31]  Jiri Matas,et al.  Ultra-fast tracking based on zero-shift points , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[32]  P. Anandan,et al.  About Direct Methods , 1999, Workshop on Vision Algorithms.

[33]  Simon Baker,et al.  Equivalence and efficiency of image alignment algorithms , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[34]  Xuelong Li,et al.  Vehicle detection and tracking in airborne videos by multi-motion layer analysis , 2011, Machine Vision and Applications.

[35]  Michel Dhome,et al.  Real Time Robust Template Matching , 2002, BMVC.

[36]  Michal Irani,et al.  Computing occluding and transparent motions , 1994, International Journal of Computer Vision.

[37]  Pascual Campoy Cervera,et al.  A Multi-resolution Image Alignment Technique Based on Direct Methods for Pose Estimation of Aerial Vehicles , 2011, 2011 International Conference on Digital Image Computing: Techniques and Applications.

[38]  Simon Baker,et al.  Lucas-Kanade 20 Years On: A Unifying Framework , 2004, International Journal of Computer Vision.

[39]  Harpreet S. Sawhney,et al.  True multi-image alignment and its application to mosaicing and lens distortion correction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[40]  Rogelio Lozano,et al.  Stabilization and Trajectory Tracking of a Quad-Rotor Using Vision , 2011, J. Intell. Robotic Syst..

[41]  Nassir Navab,et al.  Multilayer Adaptive Linear Predictors for Real-Time Tracking , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Gary R. Bradski,et al.  Learning OpenCV 3: Computer Vision in C++ with the OpenCV Library , 2016 .

[43]  Edward H. Adelson,et al.  PYRAMID METHODS IN IMAGE PROCESSING. , 1984 .

[44]  Zdenek Kalal,et al.  Tracking-Learning-Detection , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Amir Averbuch,et al.  Automatic segmentation of moving objects in video sequences: a region labeling approach , 2002, IEEE Trans. Circuits Syst. Video Technol..

[46]  J.-Y. Bouguet,et al.  Pyramidal implementation of the lucas kanade feature tracker , 1999 .

[47]  Pascual Campoy Cervera,et al.  A vision-based strategy for autonomous aerial refueling tasks , 2013, Robotics Auton. Syst..

[48]  Harpreet S. Sawhney,et al.  A Heterogeneous Feature-based Image Alignment Method , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[49]  Keith J. Hanna,et al.  Combining stereo and motion analysis for direct estimation of scene structure , 1993, 1993 (4th) International Conference on Computer Vision.

[50]  Harpreet S. Sawhney,et al.  Robust Video Mosaicing through Topology Inference and Local to Global Alignment , 1998, ECCV.

[51]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.