Vision-based production of personalized video

In this paper we present a novel vision-based system for the automated production of personalized video souvenirs for visitors in leisure and cultural heritage venues. Visitors are visually identified and tracked through a camera network. The system produces a personalized DVD souvenir at the end of a visitor's stay allowing visitors to relive their experiences. We analyze how we identify visitors by fusing facial and body features, how we track visitors, how the tracker recovers from failures due to occlusions, as well as how we annotate and compile the final product. Our experiments demonstrate the feasibility of the proposed approach.

[1]  L. Davis,et al.  M2Tracker: A Multi-View Approach to Segmenting and Tracking People in a Cluttered Scene , 2003, International Journal of Computer Vision.

[2]  Henning Schulzrinne,et al.  Proceedings of the 12th annual ACM international conference on Multimedia , 2004, MM 2004.

[3]  J. Krumm,et al.  Multi-camera multi-person tracking for EasyLiving , 2000, Proceedings Third IEEE International Workshop on Visual Surveillance.

[4]  Tieniu Tan,et al.  A survey on visual surveillance of object motion and behaviors , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[5]  Rajeev Sharma,et al.  Adaptive texture and color segmentation for tracking moving objects , 2002, Pattern Recognit..

[6]  Michael Mateas,et al.  Generation of Ideologically-Biased Historical Documentaries , 2000, AAAI/IAAI.

[7]  Simon J. Godsill,et al.  On sequential Monte Carlo sampling methods for Bayesian filtering , 2000, Stat. Comput..

[8]  Alice J. O'Toole,et al.  Face Recognition Algorithms Surpass Humans Matching Faces Over Changes in Illumination , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Tao Xiong,et al.  A combined SVM and LDA approach for classification , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[10]  Lei Chen,et al.  Rule-based scene extraction from video , 2002, Proceedings. International Conference on Image Processing.

[11]  Alan P. Parkes,et al.  The Application of Video Semantics and Theme Representation in Automated Video Editing , 2004, Multimedia Tools and Applications.

[12]  Nando de Freitas,et al.  An Introduction to MCMC for Machine Learning , 2004, Machine Learning.

[13]  Brian V. Funt,et al.  A comparison of computational color constancy algorithms. I: Methodology and experiments with synthesized data , 2002, IEEE Trans. Image Process..

[14]  Brian V. Funt,et al.  A comparison of computational color constancy Algorithms. II. Experiments with image data , 2002, IEEE Trans. Image Process..

[15]  Alan P. Parkes,et al.  Film Sequence Generation Strategies for Automatic Intelligent Video Editing , 1997, Appl. Artif. Intell..

[16]  David J. Fleet,et al.  Stochastic Tracking of 3D Human Figures Using 2D Image Motion , 2000, ECCV.

[17]  Mubarak Shah,et al.  A Multiview Approach to Tracking People in Crowded Scenes Using a Planar Homography Constraint , 2006, ECCV.

[18]  Jake K. Aggarwal,et al.  Tracking Human Motion in Structured Environments Using a Distributed-Camera System , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Ferdinand van der Heijden,et al.  Efficient adaptive density estimation per image pixel for the task of background subtraction , 2006, Pattern Recognit. Lett..

[20]  Sergios Theodoridis,et al.  Hierarchical Feature Fusion for Visual Tracking , 2007, 2007 IEEE International Conference on Image Processing.

[21]  M. Hahnel,et al.  Color and texture features for person recognition , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[22]  Sergios Theodoridis,et al.  A hierarchical feature fusion framework for adaptive visual tracking , 2011, Image Vis. Comput..

[23]  J. F. Reid,et al.  RGB calibration for color image analysis in machine vision , 1996, IEEE Trans. Image Process..

[24]  Luc Van Gool,et al.  An adaptive color-based particle filter , 2003, Image Vis. Comput..

[25]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[26]  Jean-Marc Odobez,et al.  Embedding Motion in Model-Based Stochastic Tracking , 2004, IEEE Transactions on Image Processing.

[27]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[28]  Andy Adler,et al.  Comparing Human and Automatic Face Recognition Performance , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[29]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[30]  Michael Isard,et al.  ICONDENSATION: Unifying Low-Level and High-Level Tracking in a Stochastic Framework , 1998, ECCV.

[31]  Bernt Schiele,et al.  Towards robust multi-cue integration for visual tracking , 2001, Machine Vision and Applications.

[32]  Eugenia Leu,et al.  The automatic video editor , 2003, MULTIMEDIA '03.

[33]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Stefano Bocconi.  Semantic-aware automatic video editing , 2004, MULTIMEDIA '04.

[35]  J.-P. Renno,et al.  Application and Evaluation of Colour Constancy in Visual Surveillance , 2005, 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance.

[36]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[37]  J. Giarratano.  The CLIPS User?s Guide , 1998 .

[38]  Luc Van Gool,et al.  Interactive Museum Guide: Accurate Retrieval of Object Descriptions , 2006, Adaptive Multimedia Retrieval.

[39]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[40]  James F. Allen.  Natural language understanding , 1987, Bejnamin/Cummings series in computer science.

[41]  Patrick Pérez,et al.  Color-Based Probabilistic Tracking , 2002, ECCV.

[42]  Patrick J. Flynn,et al.  Preliminary Face Recognition Grand Challenge Results , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[43]  Nikolas P. Galatsanos,et al.  An Analytic Distance Metric for Gaussian Mixture Models with Application in Image Retrieval , 2005, ICANN.

[44]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[45]  Tomaso A. Poggio,et al.  Full-body person recognition system , 2003, Pattern Recognit..

[46]  Ramesh C. Jain,et al.  An architecture for multiple perspective interactive video , 1995, MULTIMEDIA '95.

[47]  Patrick Pérez,et al.  Data fusion for visual tracking with particles , 2004, Proceedings of the IEEE.

[48]  Tieniu Tan,et al.  Principal axis-based correspondence between multiple cameras for people tracking , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  T. List,et al.  Comparison of target detection algorithms using adaptive background models , 2005, 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance.

[50]  James F. Allen.  Natural language understanding (2nd ed.) , 1995 .

[51]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..