Video classification using spatial-temporal features and PCA

We investigate the problem of automated video classification by analysing the low-level audio-visual signal patterns along the time course in a holistic manner. Five popular TV broadcast genre are studied including sports, cartoon, news, commercial and music. A novel statistically based approach is proposed comprising two important ingredients designed for implicit semantic content characterisation and class identities modelling. First, a spatial-temporal audio-visual "concatenated" feature vector is composed, aiming to capture crucial clip-level video structure information inherent in a video genre. Second, the feature vector is further processed using principal component analysis to reduce the spatial-temporal redundancy while exploiting the correlations between feature elements. This gives rise to a compact representation fro effective probabilistic modelling of each video genre. Extensive experiments are conducted assessing various aspects of the approach and their influence on the overall system performance.

[1]  Mark Pawlewski,et al.  Video genre classification using dynamics , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[2]  Tsuhan Chen,et al.  Audio Feature Extraction and Analysis for Scene Segmentation and Classification , 1998, J. VLSI Signal Process..

[3]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[4]  Juyang Weng,et al.  Using Discriminant Eigenfeatures for Image Retrieval , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Roach Matthew.  Classification of Non-edited Broadcast Video Using Holistic Low-level Features , 2002 .

[6]  Wolfgang Effelsberg,et al.  Automatic audio content analysis , 1997, MULTIMEDIA '96.

[7]  Avideh Zakhor,et al.  Content analysis of video using principal components , 1998, IEEE Trans. Circuits Syst. Video Technol..

[8]  Shaogang Gong,et al.  Recognising trajectories of facial identities using kernel discriminant analysis , 2003, Image Vis. Comput..

[9]  Ba Tu Truong,et al.  Automatic genre identification for content-based video categorization , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[10]  Gang Wei,et al.  Video Classification Using Object Tracking , 2001, Int. J. Image Graph..

[11]  Wolfgang Effelsberg,et al.  Automatic recognition of film genres , 1995, MULTIMEDIA '95.

[12]  B. S. Manjunath,et al.  Color and texture descriptors , 2001, IEEE Trans. Circuits Syst. Video Technol..