Accelerated RANSAC for Accurate Image Registration in Aerial Video Surveillance

Compared with ground views and direct overhead views (for orbital satellites), aerial robotics allow for capturing videos from diverse viewpoints and scenes, thus, the content of aerial image is complex and changeable, and aerial video has complex inter-frame transforms stemming from the blend of camera motion, platform motion and jitter. In addition, poor quality and similar texture are common in long-distance and large-scale aerial video surveillance. All of these interferences make image registration of aerial video difficult. This article puts forward one image registration method suited to aerial video on the basis of the hypothesize-and-verify of RANSAC. The proposed accelerated RANSAC, named PSSC-RANSAC (Prior Sampling & Sample Check RANSAC), incorporates prior sampling, which comes from three levels of sample evaluation, including texture magnitude, spatial consistency and feature similarity, to generate more possibly correct samples in priority. Furthermore, prior information of sample, quality of sample subset and subset invariability are together used to check the sample subsets, and the incompatible arrangements of subsets are immediately ruled out in sample check stage, which speeds up the iteration further. Results of the experiment have proved the good performance of the presented PSSC-RANSAC at 90% contamination level. For typical image pairs, the number of iterations is reduced by at least 16.67% and evaluation computation is reduced by at least 11.01% compared with SVH-RANSAC, and the re-projection error is decreased by at least 4.44% and 6.31% compared with RANSAC and SVH-RANSAC, respectively. It can overcome the interferences, and is very suitable for image registration of aerial images.

[1]  Reinhard Klette,et al.  When to use what feature? SIFT, SURF, ORB, or A-KAZE features for monocular visual odometry , 2016, 2016 International Conference on Image and Vision Computing New Zealand (IVCNZ).

[2]  Jiri Matas,et al.  Randomized RANSAC with Td, d test , 2004, Image Vis. Comput..

[3]  Andrea Fusiello,et al.  T-Linkage: A Continuous Relaxation of J-Linkage for Multi-model Fitting , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[5]  Hideyuki Tamura,et al.  Textural Features Corresponding to Visual Perception , 1978, IEEE Transactions on Systems, Man, and Cybernetics.

[6]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[7]  Ming-Ting Sun,et al.  A Fast Sub-Pixel Motion Estimation Algorithm for H.264/AVC Video Coding , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[8]  Jian Sun,et al.  SteadyFlow: Spatially Smooth Optical Flow for Video Stabilization , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Ahad Harati,et al.  Parallel RANSAC: Speeding up plane extraction in RGBD image sequences using GPU , 2015, 2015 5th International Conference on Computer and Knowledge Engineering (ICCKE).

[10]  Guilin Yang,et al.  A Deep Learning-Based Semantic Filter for RANSAC-Based Fundamental Matrix Calculation and the ORB-SLAM System , 2020, IEEE Access.

[11]  Jan-Michael Frahm,et al.  USAC: A Universal Framework for Random Sample Consensus , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Antonios Gasteratos,et al.  Review of Stereo Vision Algorithms: From Software to Hardware , 2008 .

[13]  Arturo Espinosa-Romero,et al.  A robust Graph Transformation Matching for non-rigid registration , 2009, Image Vis. Comput..

[14]  Yue Wang,et al.  Accurate hyperspectral and infrared satellite image registration method using structured topological constraints , 2020 .

[15]  Jiri Matas,et al.  Randomized RANSAC with sequential probability ratio test , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[16]  Jiri Matas,et al.  Locally Optimized RANSAC , 2003, DAGM-Symposium.

[17]  Jiri Matas,et al.  Matching with PROSAC - progressive sample consensus , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[18]  Bo Li,et al.  Real-Time Global Motion Vectors Estimation Based on Phase Correlation and Gray Projection Algorithm , 2009, 2009 2nd International Congress on Image and Signal Processing.

[19]  Torsten Sattler,et al.  SCRAMSAC: Improving RANSAC's efficiency with a spatial consistency filter , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[20]  David Nistér,et al.  Preemptive RANSAC for live structure and motion estimation , 2005, Machine Vision and Applications.

[21]  P. V. Smirnov,et al.  PIXEL-BY-PIXEL ESTIMATION OF SCENE MOTION IN VIDEO , 2017 .

[22]  Parvaneh Saeedi,et al.  Robust Weighted Graph Transformation Matching for Rigid and Nonrigid Image Registration , 2012, IEEE Transactions on Image Processing.

[23]  Gaku Nakano Accelerated RANSAC for 2D homography estimation based on global brightness consistency , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[24]  Erkan Bostanci,et al.  Spatial Statistics of Image Features for Performance Comparison , 2014, IEEE Transactions on Image Processing.

[25]  He Xu,et al.  A Hybrid Motion Estimation for Video Stabilization Based on an IMU Sensor , 2018, Sensors.

[26]  Hujun Bao,et al.  Consistent Depth Maps Recovery from a Video Sequence , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Sebastiano Battiato,et al.  A Robust Block-Based Image/Video Registration Approach for Mobile Imaging Devices , 2010, IEEE Transactions on Multimedia.

[28]  Jan-Michael Frahm,et al.  Exploiting uncertainty in random sample consensus , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[29]  Gang Wang,et al.  Two-View Geometry Estimation Using RANSAC With Locality Preserving Constraint , 2020, IEEE Access.

[30]  Yufei Chen,et al.  An automatic panoramic image mosaic method based on graph model , 2015, Multimedia Tools and Applications.

[31]  Slawomir J. Nasuto,et al.  NAPSAC: High Noise, High Dimensional Robust Estimation - it's in the Bag , 2002, BMVC.

[32]  Nivedita Tripathi,et al.  RLSAK: A recursive least square approximation with k-means for transformation model estimation in image registration techniques , 2013, 2013 International Conference on Human Computer Interactions (ICHCI).

[33]  Xueyuan Li,et al.  DL-RANSAC: An Improved RANSAC with Modified Sampling Strategy Based on the Likelihood , 2019, 2019 IEEE 4th International Conference on Image, Vision and Computing (ICIVC).

[34]  Takashi Morie,et al.  A motion detection model inspired by hippocampal function and its applications to obstacle detection , 2014, Neurocomputing.

[35]  Feng Yang,et al.  A density-based recursive RANSAC algorithm for unmanned aerial vehicle multi-target tracking in dense clutter , 2017, 2017 13th IEEE International Conference on Control & Automation (ICCA).

[36]  Jiri Matas,et al.  Epipolar geometry estimation via RANSAC benefits from the oriented epipolar constraint , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[37]  Frank Dellaert,et al.  GroupSAC: Efficient consensus in the presence of groupings , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[38]  Qiushi Zhao,et al.  A SIFT-based contactless palmprint verification approach using iterative RANSAC and local palmprint descriptors , 2014, Pattern Recognit..

[39]  In-So Kweon,et al.  Robust feature point matching by preserving local geometric consistency , 2009, Comput. Vis. Image Underst..

[40]  Magdy A. Bayoumi,et al.  A fast discrete transform architecture for Frequency Domain Motion Estimation , 2010, 2010 IEEE International Conference on Image Processing.

[41]  Michael Isard,et al.  Object retrieval with large vocabularies and fast spatial matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Haifeng Chen,et al.  Robust regression with projection based M-estimators , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[43]  Guang-Cai Sun,et al.  A 2-D Space-Variant Motion Estimation and Compensation Method for Ultrahigh-Resolution Airborne Stepped-Frequency SAR With Long Integration Time , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[44]  Peng Song,et al.  Scalable KDE-based top-n local outlier detection over large-scale data streams , 2020, Knowl. Based Syst..

[45]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[46]  Yue Wang,et al.  An improved RANSAC based on the scale variation homogeneity , 2016, J. Vis. Commun. Image Represent..

[47]  David P. Capel An Effective Bail-out Test for RANSAC Consensus Scoring , 2005, BMVC.

[48]  Gang Wang,et al.  Fuzzy correspondences guided Gaussian mixture model for point set registration , 2017, Knowl. Based Syst..

[49]  Jan Flusser,et al.  Image registration methods: a survey , 2003, Image Vis. Comput..