Omnidirectional Video Capturing, Multiple People Tracking and Identification for Meeting Monitoring

Meetings are a very important part of every days life for professionals working in universities, companies or governmental institutions. In fact, it is estimated that a midlevel manager or professional spends around 35% of his/her time in meetings. We have designed a physical awareness system called CAMEO (Camera Assisted Meeting Event Observer), a hardware/software component to record and monitor people’s activities in meetings. CAMEO captures the audio and a high resolution omnidirectional view of the meeting by stitching images coming from almost concentric cameras. Besides recording capabilities, CAMEO automatically detects people and automatically learns a personspecific facial appearance model (PSFAM) for each of the participants. The PSFAMs allow more robust, reliable and faster tracking and identification. Several novelties in the video capturing device, multiple person identification and tracking are proposed. The effectiveness and robustness of the proposed system is demonstrated over several real time experiments and a large data set of videos.

[1]  Ken-ichi Maeda,et al.  Face recognition using temporal image sequence , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[2]  Ming-Hsuan Yang,et al.  Adaptive Probabilistic Visual Tracking with Incremental Subspace Update , 2004, ECCV.

[3]  Kohji Fukunaga,et al.  Introduction to Statistical Pattern Recognition-Second Edition , 1990 .

[4]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[5]  Trevor Darrell,et al.  Face Recognition from Long-Term Observations , 2002, ECCV.

[6]  Matthew Brand,et al.  Incremental Singular Value Decomposition of Uncertain Data with Missing Values , 2002, ECCV.

[7]  Janne Heikkilä,et al.  A four-step camera calibration procedure with implicit image correction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[9]  Alexander H. Waibel,et al.  A real-time face tracker , 1996, Proceedings Third IEEE Workshop on Applications of Computer Vision. WACV'96.

[10]  Michael J. Black,et al.  Robust parameterized component analysis: theory and applications to 2D facial appearance models , 2003, Comput. Vis. Image Underst..

[11]  Brett Browning,et al.  CAMEO: Camera Assisted Meeting Event Observer , 2007 .

[12]  Alex Pentland,et al.  Probabilistic Visual Learning for Object Representation , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Richard Szeliski,et al.  Creating full view panoramic image mosaics and environment maps , 1997, SIGGRAPH.

[14]  Franc Solina,et al.  Panoramic Depth Imaging: Single Standard Camera Approach , 2002, International Journal of Computer Vision.

[15]  Mohan M. Trivedi,et al.  Activity monitoring and summarization for an intelligent meeting room , 2000, Proceedings Workshop on Human Motion.

[16]  Max Van Kleek,et al.  Virtual mouse vision based interface , 2004, IUI '04.

[17]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[18]  Anoop Gupta,et al.  Viewing meeting captured by an omni-directional camera , 2001, CHI.

[19]  Hans-Hermann Bock,et al.  Information Systems and Data Analysis , 1994 .

[20]  Henry Schneiderman,et al.  Learning a restricted Bayesian network for object detection , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[21]  Takeo Kanade,et al.  Multimodal oriented discriminant analysis , 2005, ICML.

[22]  W. Krzanowski.  Between-Groups Comparison of Principal Components , 1979 .

[23]  Jianbo Shi,et al.  Multiclass spectral clustering , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[24]  M. Veloso,et al.  Using Sparse Visual Data to Model Human Activities in Meetings , 2004 .

[25]  Anoop Gupta,et al.  Distributed meetings: a meeting capture and broadcasting system , 2002, MULTIMEDIA '02.

[26]  Shmuel Peleg,et al.  Panoramic mosaics by manifold projection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[27]  Don Kimber,et al.  FlyCam: practical panoramic video and automatic camera control , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[28]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[29]  Gérard G. Medioni,et al.  GlobeAll: panoramic video for an intelligent room , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[30]  Zhengyou Zhang,et al.  Flexible camera calibration by viewing a plane from unknown orientations , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.