Omnidirectional Image Stabilization for Visual Object Recognition

In this paper, we present a pipeline for camera pose and trajectory estimation, and image stabilization and rectification for dense as well as wide baseline omnidirectional images. The proposed pipeline transforms a set of images taken by a single hand-held camera to a set of stabilized and rectified images augmented by the computed camera 3D trajectory and a reconstruction of feature points facilitating visual object recognition. The paper generalizes previous works on camera trajectory estimation done on perspective images to omnidirectional images and introduces a new technique for omnidirectional image rectification that is suited for recognizing people and cars in images. The performance of the pipeline is demonstrated on real image sequences acquired in urban as well as natural environments.

[1]  Andrew Zisserman,et al.  Video Google: Efficient Visual Search of Videos , 2006, Toward Category-Level Object Recognition.

[2]  Henrik Stewenius,et al.  Gröbner Basis Methods for Minimal Problems in Computer Vision , 2005 .

[3]  Roland Siegwart,et al.  Closing the Loop in Appearance-Guided Structure-from-Motion for Omnidirectional Cameras , 2008 .

[4]  R. Hartley,et al.  A Non-iterative Method for Correcting Lens Distortion from Nine Point Correspondences , 2005 .

[5]  Stepán Obdrzálek,et al.  Object Recognition using Local Affine Frames on Distinguished Regions , 2002, BMVC.

[6]  Luc Van Gool,et al.  Omnidirectional Vision Based Topological Navigation , 2007, International Journal of Computer Vision.

[7]  Michal Havlena,et al.  Randomized structure from motion based on atomic 3D models from camera triplets , 2009, CVPR.

[8]  Luc Van Gool,et al.  A mobile vision system for robust multi-person tracking , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Olivier Stasse,et al.  MonoSLAM: Real-Time Single Camera SLAM , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Jan-Michael Frahm,et al.  Towards Urban 3D Reconstruction from Video , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[11]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[12]  Kostas Daniilidis,et al.  Structure and motion from uncalibrated catadioptric views , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[13]  Michal Havlena,et al.  Measuring camera translation by the dominant apical angle , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[15]  H. Bakstein,et al.  Panoramic mosaicing with a 180/spl deg/ field of view lens , 2002, Proceedings of the IEEE Workshop on Omnidirectional Vision 2002. Held in conjunction with ECCV'02.

[16]  Alexei A. Efros,et al.  Putting Objects in Perspective , 2006, CVPR.

[17]  Jiri Matas,et al.  Matching with PROSAC - progressive sample consensus , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[18]  Kostas Daniilidis,et al.  Monocular Visual Odometry in Urban Environments , 2008 .

[19]  David Nister,et al.  Estimating global uncertainty in epipoloar geometry for vehicle-mounted cameras , 2006, SPIE Defense + Commercial Sensing.

[20]  David G. Lowe,et al.  Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration , 2009, VISAPP.

[21]  Tomás Pajdla,et al.  Omnidirectional Camera Motion Estimation , 2008, VISAPP.

[22]  Stepán Obdrzálek,et al.  Image Retrieval Using Local Compact DCT-Based Representation , 2003, DAGM-Symposium.

[23]  Richard Szeliski,et al.  Skeletal graphs for efficient structure from motion , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Binoy Pinto,et al.  Speeded Up Robust Features , 2011 .

[25]  Luc Van Gool,et al.  Dynamic 3D Scene Analysis from a Moving Vehicle , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Noah Snavely.  Photo Tourism : Exploring image collections in 3D , 2006 .

[27]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[28]  David Nistér.  A Minimal Solution to the Generalised 3-Point Pose Problem , 2004, CVPR.

[29]  Michal Havlena,et al.  Structure from Omnidirectional Stereo Rig Motion for City Modeling , 2008, VISAPP.

[30]  Matthew A. Brown,et al.  Recognising panoramas , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[31]  Manolis I. A. Lourakis,et al.  The design and implementation of a generic sparse bundle adjustment software package based on the Le , 2004 .

[32]  Ian D. Reid,et al.  Real-Time SLAM Relocalisation , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[33]  Takeo Kanade,et al.  Quasiconvex Optimization for Robust Geometric Reconstruction , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Cordelia Schmid,et al.  A Comparison of Affine Region Detectors , 2005, International Journal of Computer Vision.

[35]  Tomás Pajdla,et al.  Robust Rotation and Translation Estimation in Multiview Reconstruction , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[37]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[38]  Richard Hartley,et al.  A non-iterative method for lens distortion correction from nine point correspondences , 2005, ICCV 2005.

[39]  Tomás Pajdla,et al.  Structure from motion with wide circular field of view cameras , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[41]  Luc Van Gool,et al.  Fast Compact City Modeling for Navigation Pre-Visualization , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[42]  F. Kahl.  Multiple View Geometry and the L-infinity Norm , 2005, ICCV 2005.

[43]  Jan-Michael Frahm,et al.  Robust 6DOF Motion Estimation for Non-Overlapping, Multi-Camera Systems , 2008, 2008 IEEE Workshop on Applications of Computer Vision.

[44]  Fredrik Kahl,et al.  Multiple view geometry and the L/sub /spl infin//-norm , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[45]  Jiri Matas,et al.  Robust wide-baseline stereo from maximally stable extremal regions , 2004, Image Vis. Comput..

[46]  Axel Pinz,et al.  Globally Optimal O(n) Solution to the PnP Problem for General Camera Models , 2008, BMVC.

[47]  Luc Van Gool,et al.  Coupled Detection and Trajectory Estimation for Multi-Object Tracking , 2007, 2007 IEEE 11th International Conference on Computer Vision.