Motion analysis for event detection and tracking with a mobile omnidirectional camera

Abstract.A mobile platform mounted with omnidirectional vision sensor (ODVS) can be used to monitor large areas and detect interesting events such as independently moving persons and vehicles. To avoid false alarms due to extraneous features, the image motion induced by the moving platform should be compensated. This paper describes a formulation and application of parametric egomotion compensation for an ODVS. Omni images give 360$^\circ$ view of surroundings but undergo considerable image distortion. To account for these distortions, the parametric planar motion model is integrated with the transformations into omni image space. Prior knowledge of approximate camera calibration and camera speed is integrated with the estimation process using a Bayesian approach. Iterative, coarse-to-fine, gradient-based estimation is used to correct the motion parameters for vibrations and other inaccuracies in prior knowledge. Experiments with a camera mounted on various types of mobile platforms demonstrate successful detection of moving persons and vehicles.

[1]  Gaudenz Danuser,et al.  Parametric Model Fitting: From Inlier Characterization to Outlier Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[3]  Eero P. Simoncelli.  Coarse-to-fine estimation of visual motion , 1993 .

[4]  Shree K. Nayar,et al.  Ego-motion and omnidirectional cameras , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[5]  Ales Leonardis,et al.  Robust localization using panoramic view-based recognition , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[6]  M. Hebert,et al.  Omni-directional structure from motion , 2000, Proceedings IEEE Workshop on Omnidirectional Vision (Cat. No.PR00704).

[7]  J. Gaspar,et al.  Omni-directional vision for robot navigation , 2000, Proceedings IEEE Workshop on Omnidirectional Vision (Cat. No.PR00704).

[8]  Tarak Gandhi,et al.  Detection of obstacles in the flight path of an aircraft , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[9]  Tarak Gandhi,et al.  Application of planar motion segmentation for scene text extraction , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[10]  A. Makadia,et al.  Image processing in catadioptric planes: spatiotemporal derivatives and optical flow computation , 2002, Proceedings of the IEEE Workshop on Omnidirectional Vision 2002. Held in conjunction with ECCV'02.

[11]  S. Lippman,et al.  The Scripps Institution of Oceanography , 1959, Nature.

[12]  Grzegorz Cielniak,et al.  Appearance-based Tracking of Persons with an Omnidirectional Vision Sensor , 2003, 2003 Conference on Computer Vision and Pattern Recognition Workshop.

[13]  Arthur Stanley,et al.  Yes , 1923, The Hospital and health review.

[14]  Tarak Gandhi,et al.  Driver's view and vehicle surround estimation using omnidirectional video stream , 2003, IEEE IV2003 Intelligent Vehicles Symposium. Proceedings (Cat. No.03TH8683).

[15]  Mohan M. Trivedi,et al.  Real-Time Traffic Flow Analysis using Omnidirectional Video Network and Flatplane Transformation , 2002 .

[16]  C. Qian,et al.  Frame-rate Multi-body Tracking for Surveillance , 1998 .

[17]  Pradeep K. Khosla,et al.  Agent-based moving object correspondence using differential discriminative diagnosis , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[18]  Kenichi Kanatani,et al.  Geometric computation for machine vision , 1993 .

[19]  M.M. Trivedi,et al.  Motion based vehicle surround analysis using an omni-directional camera , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[20]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[21]  S. Shankar Sastry,et al.  Omnidirectional Egomotion Estimation From Back-projection Flow , 2003, 2003 Conference on Computer Vision and Pattern Recognition Workshop.

[22]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[23]  Tarak Gandhi,et al.  Detection of obstacles on runways using ego-motion compensation and tracking of significant features , 2000, Image Vis. Comput..

[24]  P. Anandan,et al.  A Unified Approach to Moving Object Detection in 2D and 3D Scenes , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Tarak Gandhi,et al.  Motion analysis of Omni-Directional video streams for a mobile sentry , 2003, IWVS '03.

[26]  Takeo Kanade,et al.  Transforming camera geometry to a virtual downward-looking camera: robust ego-motion estimation and ground-layer detection , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[27]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Mohan M. Trivedi,et al.  Video arrays for real-time tracking of person, head, and face in an intelligent room , 2003, Machine Vision and Applications.

[29]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..

[30]  S. B. Kang,et al.  Panoramic vision : sensors, theory, and applications , 2001 .

[31]  W. K. Üger.  Robust real-time ground plane motion compensation from a moving vehicle , .

[32]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[33]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Jan-Olof Eklundh,et al.  Object detection using model based prediction and motion parallax , 1990, ECCV.

[35]  Jean-Marc Odobez,et al.  Direct incremental model-based image motion segmentation for video analysis , 1998, Signal Process..

[36]  Mohan M. Trivedi,et al.  N-Ocular stereo for real-time human tracking , 2001 .

[37]  Bernd Jähne,et al.  Computer vision and applications: a guide for students and practitioners , 2000 .

[38]  Hans J. Schneebeli,et al.  A general approach for egomotion estimation with omnidirectional images , 2002, Proceedings of the IEEE Workshop on Omnidirectional Vision 2002. Held in conjunction with ECCV'02.

[39]  T. B. Hall,et al.  A novel interactivity environment for integrated intelligent transportation and telematic systems , 2002, Proceedings. The IEEE 5th International Conference on Intelligent Transportation Systems.

[40]  Thia Kirubarajan,et al.  Estimation with Applications to Tracking and Navigation: Theory, Algorithms and Software , 2001 .