Panoramic virtual stereo vision of cooperative mobile robots for localizing 3D moving objects

Flexible, reconfigurable vision systems can provide an extremely rich sensing modality for sophisticated multiple robot platforms. We propose a cooperative and adaptive approach of panoramic vision to the problem of finding and protecting humans by a robot team in an emergency circumstance (e.g., a rescue in an office building). A panoramic virtual stereo vision method is proposed for this cooperative approach, which features omni-directional visual sensors, cooperative mobile platforms, selected 3D matching, and real-time moving object (people) detection and tracking. The problems of dynamic self-calibration and robust 3D estimation of moving objects are discussed. A careful error analysis of the panoramic stereo triangulation is presented in order to derive rules for optimal view planning. Experimental results are given for detecting and localizing multiple moving objects using two cooperative robot platforms.

[1]  Yasushi Yagi,et al.  Panorama scene analysis with conic projection , 1990, EEE International Workshop on Intelligent Robots and Systems, Towards a New Frontier of Applications.

[2]  Hiroshi Ishiguro,et al.  Omni-Directional Stereo , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  I. Powell.  Panoramic lens. , 1994, Applied optics.

[4]  Shree K. Nayar,et al.  A theory of catadioptric image formation , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[5]  Xueyin Lin,et al.  Fast road classification and orientation estimation using omni-view images and neural networks , 1998, IEEE Trans. Image Process..

[6]  Larry S. Davis,et al.  W4S : A real-time system for detecting and tracking people in 2 D , 1998, eccv 1998.

[7]  Hironobu Fujiyoshi,et al.  Moving target classification and tracking from real-time video , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[8]  Takushi Sogo,et al.  Monitoring dynamically changing environments by ubiquitous vision system , 1999, Proceedings Second IEEE Workshop on Visual Surveillance (VS'99) (Cat. No.98-89223).

[9]  Terrance E. Boult,et al.  Frame-rate omnidirectional surveillance and tracking of camouflaged and occluded targets , 1999, Proceedings Second IEEE Workshop on Visual Surveillance (VS'99) (Cat. No.98-89223).

[10]  E. Riseman,et al.  Geometrical Modeling and Real-Time Vision Applications of a Panoramic Annular Lens (PAL) Camera System , 1999 .

[11]  Allen R. Hanson,et al.  3D Localization of Multiple Moving People by a Omnidirectional Stereo System of Cooperative Mobile Robots , 2000 .