Sequential and Automatic Image-Sequence Registration of Road Areas Monitored from a Hovering Helicopter

In this paper, we propose an automatic and sequential method for the registration of an image sequence of a road area without ignoring scene-induced motion. This method contributes to a larger work, aiming at vehicle tracking. A typical image sequence is recorded from a helicopter hovering above the freeway. The demand for automation is inevitable due to the large number of images and continuous changes in the traffic situation and weather conditions. A framework is designed and implemented for this purpose. The registration errors are removed in a sequential way based on two homography assumptions. First, an approximate registration is obtained, which is efficiently refined in a second step, using a restricted search area. The results of the stabilization framework are demonstrated on an image sequence consisting of 1500 images and show that our method allows a registration between arbitrary images in the sequence with a geometric error of zero in pixel accuracy.

[1]  M. Pollefeys.  Self-calibration and metric 3d reconstruction from uncalibrated image sequences , 1999 .

[2]  S.J.L. Ossen,et al.  Longitudinal driving behavior: Theory and empirics , 2008 .

[3]  P. Anandan,et al.  A unified approach to moving object detection in 2D and 3D scenes , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[4]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[5]  S. Hoogendoorn,et al.  Optimization based image registration in the presence of moving objects , 2008 .

[6]  Marc Pollefeys,et al.  Pan-tilt-zoom camera calibration and high-resolution mosaic generation , 2006, Comput. Vis. Image Underst..

[7]  Zhengyou Zhang,et al.  Flexible camera calibration by viewing a plane from unknown orientations , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[8]  Michael Gleicher,et al.  Content-preserving warps for 3D video stabilization , 2009, ACM Trans. Graph..

[9]  Cordelia Schmid,et al.  A Comparison of Affine Region Detectors , 2005, International Journal of Computer Vision.

[10]  Uwe Stilla,et al.  Detection of moving objects in airborne thermal videos , 2006 .

[11]  Serge P. Hoogendoorn,et al.  Interdriver Differences in Car-Following: A Vehicle Trajectory–Based Study , 2006 .

[12]  Dean Brown,et al.  Decentering distortion of lenses , 1966 .

[13]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Irfan A. Essa,et al.  Auto-directed video stabilization with robust L1 optimal camera paths , 2011, CVPR 2011.

[15]  Dorota A. Grejner-Brzezinska,et al.  Extracting dynamic spatial data from airborne imaging sensors to support traffic flow estimation , 2006 .

[16]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[17]  Michael Gleicher,et al.  Subspace video stabilization , 2011, TOGS.

[18]  Maneesh Agrawala,et al.  User‐Assisted Video Stabilization , 2014, Comput. Graph. Forum.

[19]  David A. Forsyth,et al.  Generalizing motion edits with Gaussian processes , 2009, ACM Trans. Graph..

[20]  Duane C. Brown,et al.  Close-Range Camera Calibration , 1971 .

[21]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[22]  Matthew A. Brown,et al.  Automatic Panoramic Image Stitching using Invariant Features , 2007, International Journal of Computer Vision.

[23]  Jian Sun,et al.  Bundled camera paths for video stabilization , 2013, ACM Trans. Graph..

[24]  Serge P. Hoogendoorn,et al.  OUTLINE EXTRACTION OF A MOTORWAY FROM HELICOPTER IMAGE SEQUENCE , 2005 .

[25]  Bin Lu,et al.  An Automatic Video Image Mosaic Algorithm Based on SIFT Feature Matching , 2013 .

[26]  Feng Liu,et al.  Spatially and Temporally Optimized Video Stabilization , 2013, IEEE Transactions on Visualization and Computer Graphics.

[27]  Janne Heikkilä,et al.  A four-step camera calibration procedure with implicit image correction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[29]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[30]  Andrew W. Fitzgibbon,et al.  The Problem of Degeneracy in Structure and Motion Recovery from Uncalibrated Image Sequences , 1999, International Journal of Computer Vision.

[31]  Jiajun Bu,et al.  Video stabilization with a depth camera , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Jean-Yves Bouguet,et al.  Camera calibration toolbox for matlab , 2001 .

[33]  Leonidas J. Guibas,et al.  Shape google: Geometric words and expressions for invariant shape retrieval , 2011, TOGS.

[34]  Rama Chellappa,et al.  Evaluation of image stabilization algorithms , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).