On-line knowledge- and rule-based video classification system for video indexing and dissemination

Current information and communication technologies provide the infrastructure to transport bits anywhere, but do not indicate how to easily and precisely access and/or route information at the semantic level. To facilitate intelligent access to the rich multimedia data over the Internet, we develop an on-line knowledge- and rule-based video classification system that supports automatic "indexing" and "filtering" based on the semantic concept hierarchy. This paper investigates the use of video and audio content analysis, feature extraction and clustering techniques for further video semantic concept classification. A supervised rule-based video classification system is proposed using video automatic segmentation, annotation and summarization techniques for seamless information browsing and updating. In the proposed system, a real-time scene-change detection proxy performs an initial video-structuring process by splitting a video clip into scenes. Motional, visual and audio features are extracted in real-time for every detected scene by using on-line feature-extraction proxies. Higher semantics are then derived through a joint use of low-level features along with classification rules in the knowledge base. Classification rules are derived through a supervised learning process that relies on some representative samples from each semantic category. An indexing and filtering process can now be built using the semantic concept hierarchy to personalize multimedia data based on users' interests. In real-time filtering, multiple video streams are blocked, combined, or sent to certain channels depending on whether or not the video streams are matched with the user's profile. We have extensively experimented and evaluated the classification and filtering techniques using basketball sports video data. In particular, in our experiment, the basketball video structure is examined and categorized into different classes according to distinct motional, visual and audio characteristics features by a rule-based classifier. The concept hierarchy describing the motional/visual/audio feature descriptors and their statistical relationships are reported in this paper along with detailed experimental results using on-line sports videos.

[1]  Giridharan Iyengar,et al.  Models for automatic classification of video sequences , 1997, Electronic Imaging.

[2]  C.-C. Jay Kuo,et al.  Content-based classification and retrieval of audio , 1998, Optics & Photonics.

[3]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[4]  Douglas Keislar,et al.  Content-Based Classification, Search, and Retrieval of Audio , 1996, IEEE Multim..

[5]  Wensheng Zhou,et al.  Real-time content-based processing of multicast video , 1998, Conference Record of Thirty-Second Asilomar Conference on Signals, Systems and Computers (Cat. No.98CH36284).

[6]  Shih-Fu Chang,et al.  VideoQ: an automated content based video search system using visual cues , 1997, MULTIMEDIA '97.

[7]  Richard Lepage,et al.  Knowledge-Based Image Understanding Systems: A Survey , 1997, Comput. Vis. Image Underst..

[8]  Tom Minka,et al.  Interactive learning with a "Society of Models" , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[10]  B.S. Manjunath,et al.  Spatio-temporal relationships and video object extraction , 1998, Conference Record of Thirty-Second Asilomar Conference on Signals, Systems and Computers (Cat. No.98CH36284).

[11]  S. Sclaroff,et al.  ImageRover: a content-based image browser for the World Wide Web , 1997, 1997 Proceedings IEEE Workshop on Content-Based Access of Image and Video Libraries.

[12]  Sanjeev R. Kulkarni,et al.  Automated analysis and annotation of basketball video , 1997, Electronic Imaging.

[13]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[14]  Miodrag Potkonjak,et al.  Semantic multicast: intelligently sharing collaborative sessions , 1999, CSUR.

[15]  Shih-Fu Chang,et al.  Model-based classification of visual information for content-based retrieval , 1998, Electronic Imaging.

[16]  Anil K. Jain,et al.  Automatic classification of tennis video for high-level content-based retrieval , 1998, Proceedings 1998 IEEE International Workshop on Content-Based Access of Image and Video Database.

[17]  Raimondo Schettini,et al.  Content-Based Classification of Digital Photos , 2002, Multiple Classifier Systems.

[18]  Julius T. Tou,et al.  Information Systems , 1973, GI Jahrestagung.

[19]  B. S. Manjunath,et al.  Content-based search of video using color, texture, and motion , 1997, Proceedings of International Conference on Image Processing.

[20]  Boon-Lock Yeo,et al.  Rapid scene analysis on compressed video , 1995, IEEE Trans. Circuits Syst. Video Technol..

[21]  Forouzan Golshani,et al.  Motion recovery for video content classification , 1995, TOIS.

[22]  Martin D. Levine,et al.  Low Level Image Segmentation: An Expert System , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Rosalind W. Picard.  A Society of Models for Video and Image Libraries , 1996, IBM Syst. J..

[24]  Thomas D. C. Little,et al.  A Survey of Technologies for Parsing and Indexing Digital Video1 , 1996, J. Vis. Commun. Image Represent..

[25]  Paul S. Heckbert.  Color image quantization for frame buffer display , 1998 .

[26]  C.-C. Jay Kuo,et al.  Online scene change detection of multicast (MBone) video , 1998, Other Conferences.

[27]  HongJiang Zhang,et al.  Automatic parsing of TV soccer programs , 1995, Proceedings of the International Conference on Multimedia Computing and Systems.