Statistical modeling and performance characterization of a real-time dual camera surveillance system

The engineering of computer vision systems that meet application specific computational and accuracy requirements is crucial to the deployment of real-life computer vision systems. This paper illustrates how past work on a systematic engineering methodology for vision systems performance characterization can be used to develop a real-time people detection and zooming system to meet given application requirements. We illustrate that by judiciously choosing the system modules and performing a careful analysis of the influence of various tuning parameters on the system it is possible to: perform proper statistical inference, automatically set control parameters and quantify limits of a dual-camera real-time video surveillance system. The goal of the system is to continuously provide a high resolution zoomed-in image of a person's head at any location of the monitored area. An omni-directional camera video is processed to detect people and to precisely control a high resolution foveal camera, which has pan, tilt and zoom capabilities. The pan and tilt parameters of the foveal camera and its uncertainties are shown to be functions of the underlying geometry, lighting conditions, background color/contrast, relative position of the person with respect to both cameras as well as sensor noise and calibration errors. The uncertainty in the estimates is used to adaptively estimate the zoom parameter that guarantees with a user specified probability, /spl alpha/, that the detected person's face is contained and zoomed within the image.

[1]  V. Ramesh,et al.  Automatic selection of tuning parameters for feature extraction sequences , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[2]  G. Wyszecki,et al.  Color Science Concepts and Methods , 1982 .

[3]  Bir Bhanu,et al.  Predicting Performance of Object Recognition , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Shree K. Nayar,et al.  Global measures of coherence for edge detector evaluation , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[5]  Benno Heigl,et al.  Color Normalization and Object Localization , 1998 .

[6]  Donald A. Adjeroh,et al.  Robust and Efficient Transform Domain Video Sequence Analysis: An Approach from the Generalized Color Ratio Model , 1997, J. Vis. Commun. Image Represent..

[7]  Glenn Healey,et al.  The Illumination-Invariant Matching of Deterministic Local Structure in Color Images , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[9]  Yuntao Cui,et al.  Indoor monitoring via the collaboration between a peripheral sensor and a foveal sensor , 1998, Proceedings 1998 IEEE Workshop on Visual Surveillance.

[10]  Katsushi Ikeuchi,et al.  Appearance-based visual learning and object recognition with illumination invariance , 2000, Machine Vision and Applications.

[11]  Glenn Healey,et al.  Computing illumination-invariant descriptors of spatially filtered color image regions , 1997, IEEE Trans. Image Process..

[12]  Shree K. Nayar,et al.  Omnidirectional Vision Systems: 1998 PI Report , 1998 .

[13]  L. Wixson.  Detecting Salient Motion by Accumulating Directionally-Consistent Flow , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  G. Healey,et al.  Illumination-invariant recognition of texture in color images , 1995 .

[15]  Patrick Courtney,et al.  Algorithmic Modeling for Performance Evaluation , 1997 .

[16]  Yali Amit,et al.  Shape Quantization and Recognition with Randomized Trees , 1997, Neural Computation.

[17]  Edward M. Riseman,et al.  Image-based homing , 1991, IEEE Control Systems.

[18]  Shree K. Nayar,et al.  Catadioptric omnidirectional camera , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  S. Nayar,et al.  Catadioptric Image Formation , 1997 .

[20]  R. Nelson,et al.  Low level recognition of human motion (or how to get your man without finding his body parts) , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[21]  Kenro Miyamoto,et al.  Fish Eye Lens , 1964 .

[22]  T. Mattfeldt.  Stochastic Geometry and Its Applications , 1996 .

[23]  Glenn Healey,et al.  The Illumination-Invariant Recognition of 3D Objects Using Local Color Invariants , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  W. Eric L. Grimson,et al.  On the Verification of Hypothesized Matches in Model-Based Recognition , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Joachim Denzler,et al.  An Efficient Combination of 2D and 3D Shape Descriptions for Contour Based Tracking of Moving Objects , 1998, ECCV.

[26]  Alex Pentland,et al.  Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Heinrich Niemann,et al.  2D-Object Tracking Based on Projection-Histograms , 1998, ECCV.

[28]  Shenchang Eric Chen,et al.  QuickTime VR: an image-based approach to virtual environment navigation , 1995, SIGGRAPH.

[29]  Robert E. Jones.  Machine vision applications , 1991 .

[30]  G. Healey,et al.  Using Illumination Invariant Color Histogram Descriptors for Recognit ion , 1994 .

[31]  Chris Goad,et al.  Special purpose automatic programming for 3D model-based vision , 1987 .

[32]  Patrick Bouthemy,et al.  Real-Time Tracking of Moving Persons by Exploiting Spatio-Temporal Image Slices , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Dorin Comaniciu,et al.  Real-time tracking of non-rigid objects using mean shift , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[34]  D. Petkovic.  The need for accuracy verification of machine vision algorithms and systems , 1989, Proceedings CVPR '89: IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[35]  Stephen M. Smith,et al.  ASSET-2: Real-Time Motion Segmentation and Shape Tracking , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Xiang Gao,et al.  Error analysis of background adaption , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[37]  Yasushi Yagi,et al.  Panorama scene analysis with conic projection , 1990, EEE International Workshop on Intelligent Robots and Systems, Towards a New Frontier of Applications.

[38]  Glenn Healey,et al.  Using Zernike moments for the illumination and geometry invariant classification of multispectral texture , 1998, IEEE Trans. Image Process..

[39]  J. Canny.  Finding Edges and Lines in Images , 1983 .

[40]  Brian V. Funt,et al.  Color Angular Indexing , 1996, ECCV.

[41]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[42]  Sang Wook Lee,et al.  Using chromaticity distributions and eigenspace analysis for pose-, illumination-, and specularity-invariant recognition of 3D objects , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[43]  Thomas M. Strat,et al.  Natural Object Recognition , 1992, Springer Series in Perception Engineering.

[44]  Glenn Healey,et al.  Using steerable filters for illumination-invariant recognition in multispectral images , 1997, Proceedings of International Conference on Image Processing.

[45]  Kevin W. Bowyer,et al.  Comparison of Edge Detectors Using an Object Recognition Task , 1999, CVPR.

[46]  Kevin W. Bowyer,et al.  Introduction to the Special Section on Empirical Evaluation of Computer Vision Algorithms , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[47]  G. Healey,et al.  Global color constancy: recognition of objects by use of illumination-invariant properties of color distributions , 1994 .

[48]  Dorin Comaniciu,et al.  Design, analysis, and engineering of video monitoring systems: an approach and a case study , 2001, Proc. IEEE.

[49]  Jiri Matas,et al.  Illumination Invariant Colour Recognition , 1994, BMVC.

[50]  Sudeep Sarkar,et al.  Robust Visual Method for Assessing the Relative Performance of Edge-Detection Algorithms , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  Stuart J. Russell,et al.  Image Segmentation in Video Sequences: A Probabilistic Approach , 1997, UAI.

[52]  Takeo Kanade,et al.  Advances in Cooperative Multi-Sensor Video Surveillance , 1999 .

[53]  Keith Price,et al.  Anything you can do, I can do better (No you can't) , 1986, Comput. Vis. Graph. Image Process..

[54]  Peter Meer,et al.  Performance Assessment Through Bootstrap , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[55]  Robert M. Haralick,et al.  Random perturbation models and performance characterization in computer vision , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[56]  Rafael Wiemker.  The Color Constancy Problem: An Illumination Invariant Mapping Approach , 1995, CAIP.

[57]  D. Mumford.  Pattern theory: a unifying perspective , 1996 .

[58]  Y. Bar-Shalom.  Tracking and data association , 1988 .

[59]  Robert M. Haralick,et al.  Computer vision theory: The lack thereof , 1986, Comput. Vis. Graph. Image Process..

[60]  Y. Yeshurun,et al.  Detection of regions of interest and camouflage breaking by direct convexity estimation , 1998, Proceedings 1998 IEEE Workshop on Visual Surveillance.

[61]  Ramin Zabih,et al.  An Algorithm for Real-Time Tracking of Non-Rigid Objects , 1991, AAAI.

[62]  Saburo Tsuji,et al.  Panoramic representation of scenes for route understanding , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[63]  Glenn Healey,et al.  Modeling the sensitivity of moment invariants in a recognition system , 1998 .

[64]  Josef Bigün,et al.  Segmentation of moving objects by robust motion parameter estimation over multiple frames , 1994, ECCV.

[65]  Terrance E. Boult,et al.  Frame-rate omnidirectional surveillance and tracking of camouflaged and occluded targets , 1999, Proceedings Second IEEE Workshop on Visual Surveillance (VS'99) (Cat. No.98-89223).

[66]  Heinrich Niemann,et al.  The systematic design and analysis cycle of a vision system: a case study in video surveillance , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[67]  Alex Waibel,et al.  Face locating and tracking for human-computer interaction , 1994, Proceedings of 1994 28th Asilomar Conference on Signals, Systems and Computers.

[68]  Bruce A. Draper,et al.  Learning 3D object recognition strategies , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[69]  Narendra Ahuja,et al.  Panoramic image acquisition , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[70]  C. Qian,et al.  Frame-rate Multi-body Tracking for Surveillance , 1998 .

[71]  Rama Chellappa,et al.  Knowledge-based control of vision systems , 1999, Image Vis. Comput..

[72]  W. Eric L. Grimson,et al.  On the sensitivity of geometric hashing , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[73]  Visvanathan Ramesh,et al.  Real-Time Video Surveillance and Monitoring for Automotive Applications , 2000 .

[74]  H. Niemann,et al.  Adaptive change detection for real-time surveillance applications , 2000, Proceedings Third IEEE International Workshop on Visual Surveillance.

[75]  Terrance E. Boult,et al.  Into the woods: visual surveillance of noncooperative and camouflaged targets in complex outdoor settings , 2001, Proc. IEEE.

[76]  Glenn Healey,et al.  Combining color and geometric information for the illumination invariant recognition of 3-D objects , 1995, Proceedings of IEEE International Conference on Computer Vision.

[77]  Alan L. Yuille,et al.  Fundamental bounds on edge detection: an information theoretic evaluation of different edge cues , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[78]  S. Nayar.  Omnidirectional Video Camera , 1997 .

[79]  Glenn Healey,et al.  Object recognition using invariant profiles , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[80]  Malayappan Shridhar,et al.  Hardware considerations for illumination-invariant image processing , 1994, Other Conferences.

[81]  Robert M. Haralick,et al.  Random perturbation models for boundary extraction sequence , 1997, Machine Vision and Applications.

[82]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[83]  Qian Huang,et al.  Auto Cameraman Via Collaborative Sensing Agents , 1998, ACCV.

[84]  Daniel P. Huttenlocher,et al.  Tracking non-rigid objects in complex scenes , 1993, 1993 (4th) International Conference on Computer Vision.

[85]  Andrew Blake,et al.  Statistical Background Modelling for Tracking with a Virtual Camera , 1995, BMVC.

[86]  James W. Davis,et al.  The representation and recognition of human movement using temporal templates , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[87]  Visvanathan Ramesh,et al.  Performance characterization of image and video analysis systems at Siemens Corporate Research , 2000, Medical Imaging: Image Processing.

[88]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[89]  Ernest L. Hall,et al.  Guidance Of A Mobile Robot Using An Omnidirectional Vision Navigation System , 1987, Other Conferences.

[90]  Robert C. Vogt.  Automatic generation of simple morphological algorithms , 1988, Proceedings CVPR '88: The Computer Society Conference on Computer Vision and Pattern Recognition.

[91]  Larry S. Davis,et al.  Non-parametric Model for Background Subtraction , 2000, ECCV.

[92]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[93]  Robert M. Haralick,et al.  Propagating covariance in computer vision , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[94]  Glenn Healey,et al.  Using Linear Models for the Illumination-Invariant Classification of Color Textures , 1995, Color Imaging Conference.

[95]  Alok Gupta,et al.  Optimal polyline tracking for artery motion compensation in coronary angiography , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[96]  John G. Proakis,et al.  Probability, random variables and stochastic processes , 1985, IEEE Trans. Acoust. Speech Signal Process..

[97]  Yali Amit,et al.  A Computational Model for Visual Selection , 1999, Neural Computation.

[98]  Kevin W. Bowyer,et al.  Empirical evaluation techniques in computer vision , 1998 .

[99]  Donald Geman,et al.  Graded Learning for Object Detection , 1999 .

[100]  Martin Bichsel,et al.  Illumination Invariant Motion Segmentation of Simple Connected Objects , 1994, BMVC.

[101]  Bea Thai,et al.  Representing multiscale N-folded symmetry in color texture , 1997, Proceedings of International Conference on Image Processing.

[102]  Paul L. Rosin.  Detecting and Classifying Intruders in Image Sequences , 1991, BMVC.

[103]  Alex Pentland,et al.  LAFTER: lips and face real time tracker , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[104]  Nikos Paragios,et al.  Real-Time Video Analysis at Siemens Corporate Research , 2002 .

[105]  Steven D. Blostein,et al.  Detecting small, moving objects in image sequences using sequential hypothesis testing , 1991, IEEE Trans. Signal Process..