Content analysis of video using principal components

We use principal component analysis (PCA) to reduce the dimensionality of features of video frames for the purpose of content description. This low-dimensional description makes practical the direct use of all the frames of a video sequence in later analysis. The PCA representation circumvents or eliminates several of the stumbling blocks in current analysis methods and makes new analyses feasible. We demonstrate this with two applications. The first accomplishes high-level scene description without shot detection and key-frame selection. The second uses the time sequences of motion data from every frame to classify sports sequences.

[1]  A. Dale Magoun,et al.  Decision, estimation and classification , 1989 .

[2]  Boon-Lock Yeo,et al.  A unified approach to temporal segmentation of motion JPEG and MPEG compressed video , 1995, Proceedings of the International Conference on Multimedia Computing and Systems.

[3]  Shih-Fu Chang,et al.  A fully automated content-based video search engine supporting spatiotemporal queries , 1998, IEEE Trans. Circuits Syst. Video Technol..

[4]  Boon-Lock Yeo,et al.  Time-constrained clustering for segmentation of video into story units , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[5]  Richard A. Harshman,et al.  Indexing by Latent Semantic Analysis , 1990, J. Am. Soc. Inf. Sci..

[6]  Giridharan Iyengar,et al.  Models for automatic classification of video sequences , 1997, Electronic Imaging.

[7]  Ryuichi Oka,et al.  A theoretical consideration of pattern space trajectory for gesture spotting recognition , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[8]  Christos Faloutsos,et al.  Developing high-level representations of video clips using VideoTrails , 1997, Electronic Imaging.

[9]  Nuno Vasconcelos,et al.  Towards semantically meaningful feature spaces for the characterization of video content , 1997, Proceedings of International Conference on Image Processing.

[10]  Wayne H. Wolf,et al.  Hidden Markov model parsing of video programs , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[11]  Alex Pentland,et al.  Photobook: tools for content-based manipulation of image databases , 1994, Other Conferences.

[12]  Forouzan Golshani,et al.  Motion recovery for video content classification , 1995, TOIS.

[13]  Alex Pentland,et al.  Photobook: tools for content-based manipulation of image databases , 1994, Electronic Imaging.

[14]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[15]  Giridharan Iyengar,et al.  VideoBook: an experiment in characterization of video , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[16]  Ahmed H. Tewfik,et al.  Eigen-image based video segmentation and indexing , 1997, Proceedings of International Conference on Image Processing.

[17]  Shih-Fu Chang,et al.  Visual information retrieval from large distributed online repositories , 1997, CACM.

[18]  T.S. Huang,et al.  A relevance feedback architecture for content-based multimedia information retrieval systems , 1997, 1997 Proceedings IEEE Workshop on Content-Based Access of Image and Video Libraries.

[19]  Avideh Zakhor,et al.  Motion indexing of video , 1997, Proceedings of International Conference on Image Processing.

[20]  Masahiko Yachida,et al.  Gesture recognition based on subspace method and hidden Markov model , 1997, Proceedings of the 1997 IEEE/RSJ International Conference on Intelligent Robot and Systems. Innovative Robotics for Real-World Applications. IROS '97.

[21]  Alex Pentland,et al.  Real-time American Sign Language recognition from video using hidden Markov models , 1995 .

[22]  Christos Faloutsos,et al.  FastMap: a fast algorithm for indexing, data-mining and visualization of traditional and multimedia datasets , 1995, SIGMOD '95.