Movie scenes detection with MIGSOM based on shots semi-supervised clustering

The segmentation into scenes helps users to browse movie archives and to select the interesting ones. In a given movie, we have two kinds of scenes: action scenes and non-action scenes. To detect action scenes, we rely on tempo features as motion and audio energy. However, to detect non-action scenes, we have to use the content information. In this paper, we present a new approach to detect non-action movie scenes. The main idea is the use of a new dynamic variant of the self-organizing maps called MIGSOM (Multilevel Interior Growing self-organizing maps) to detect agglomerations of shots in movie scenes. The originality of MIGSOM model lies in its architecture for evolving the structure of the network. The MIGSOM algorithm is generated by a growth process by adding nodes where it is necessary, whether from the boundaries or the interior of the map. In addition, the advantage of the proposed MIGSOM algorithm is their ability to find the best structure of the output space through the training process and to represent better the semantics of the data. Our system is tested on a varied database and compared to the classical SOM and others works. The obtained results show the merit of our approach in term of recall and precision rates and that our assumptions are well founded.

[1]  Adel M. Alimi,et al.  Scene pathfinder: unsupervised clustering techniques for movie scenes extraction , 2010, Multimedia Tools and Applications.

[2]  Bernd Fritzke,et al.  Growing cell structures--A self-organizing network for unsupervised and supervised learning , 1994, Neural Networks.

[3]  Teuvo Kohonen,et al.  Self-organization and associative memory: 3rd edition , 1989 .

[4]  Victoria J. Hodge,et al.  Hierarchical growing cell structures: TreeGCS , 2000, KES'2000. Fourth International Conference on Knowledge-Based Intelligent Engineering Systems and Allied Technologies. Proceedings (Cat. No.00TH8516).

[5]  T. Kohonen.  Self-organized formation of topology correct feature maps , 1982 .

[6]  HongJiang Zhang,et al.  Automatic video scene extraction by shot grouping , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[7]  Thouraya Ayadi,et al.  A new data topology matching technique with Multilevel Interior Growing Self-Organizing Maps , 2010, 2010 IEEE International Conference on Systems, Man and Cybernetics.

[8]  Glenn Fung,et al.  Reducing a Biomarkers List via Mathematical Programming: Application to Gene Signatures to Detect Time-Dependent Hypoxia in Cancer , 2007, ICMLA 2007.

[9]  RauberA.,et al.  The growing hierarchical self-organizing map , 2002 .

[10]  Shih-Fu Chang,et al.  Video scene segmentation using video and audio features , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[11]  Liang-Hua Chen,et al.  Movie scene segmentation using background information , 2008, Pattern Recognit..

[12]  Mustapha Lebbah,et al.  Visualization and clustering of categorical data with probabilistic self-organizing map , 2009, Neural Computing and Applications.

[13]  Adel M. Alimi,et al.  Genetic algorithm for summarizing news stories , 2007, VISAPP.

[14]  Ba Tu Truong,et al.  Scene extraction in motion pictures , 2003, IEEE Trans. Circuits Syst. Video Technol..

[15]  Bala Srinivasan,et al.  Dynamic self-organizing maps with controlled growth for knowledge discovery , 2000, IEEE Trans. Neural Networks Learn. Syst..

[16]  Kate Smith-Miles,et al.  HDGSOM: a modified growing self-organizing map for high dimensional data clustering , 2004, Fourth International Conference on Hybrid Intelligent Systems (HIS'04).

[17]  T. Kohonen,et al.  Statistical pattern recognition with neural networks: benchmarking studies , 1988, IEEE 1988 International Conference on Neural Networks.

[18]  Haruhiko Nishimura,et al.  Evolving neural networks with iterative learning scheme for associative memory , 2005, Neural Processing Letters.

[19]  Adel M. Alimi,et al.  Conflict Negotiation Process with Stress Parameters Control for New Classifier Decision Fusion Scheme , 2010, IPCV.

[20]  Mubarak Shah,et al.  Detection and representation of scenes in videos , 2005, IEEE Transactions on Multimedia.

[21]  T. Kohonen.  Self-organized formation of topographically correct feature maps , 1982 .

[22]  Alan Hanjalic,et al.  Automated high-level movie segmentation for advanced video-retrieval systems , 1999, IEEE Trans. Circuits Syst. Video Technol..

[23]  Damminda Alahakoon,et al.  Batch implementation of Growing Self-Organizing Map , 2006, 2006 International Conference on Computational Inteligence for Modelling Control and Automation and International Conference on Intelligent Agents Web Technologies and International Commerce (CIMCA'06).

[24]  Teuvo Kohonen,et al.  Visual Explorations in Finance , 1998 .

[25]  Stefan Wermter,et al.  Data mining using rule extraction from Kohonen self-organising maps , 2006, Neural Computing & Applications.

[26]  T. Kohonen,et al.  Visual Explorations in Finance with Self-Organizing Maps , 1998 .

[27]  Thouraya Ayadi,et al.  2IBGSOM: interior and irregular boundaries growing self-organizing maps , 2007, Sixth International Conference on Machine Learning and Applications (ICMLA 2007).

[28]  Andreas Rauber,et al.  The growing hierarchical self-organizing map , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[29]  A. Alimi,et al.  On the use of cluster validity for evaluation of MIGSOM clustering , 2011, 2011 5th International Symposium on Computational Intelligence and Intelligent Informatics (ISCIII).

[30]  Bernd Fritzke.  Growing Grid — a self-organizing network with constant neighborhood range and adaptation strength , 1995, Neural Processing Letters.

[31]  Risto Miikkulainen,et al.  Incremental grid growing: encoding high-dimensional structure into a two-dimensional feature map , 1993, IEEE International Conference on Neural Networks.

[32]  Li Zhao,et al.  Video scene detection using slide windows method based on temporal constrain shot similarity , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[33]  Boon-Lock Yeo,et al.  Segmentation of Video by Clustering and Graph Analysis , 1998, Comput. Vis. Image Underst..

[34]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[35]  Adel M. Alimi,et al.  Regim, research group on intelligent machines, tunisia, at TRECVID 2008, BBC rushes summarization , 2008, TVS '08.

[36]  Chokri Ben Amar,et al.  A New System for Event Detection from Video Surveillance Sequences , 2010, ACIVS.

[37]  Adel M. Alimi,et al.  IM(S)2: Interactive movie summarization system , 2010, J. Vis. Commun. Image Represent..

[38]  Hujun Yin,et al.  Adaptive topological tree structure for document organisation and visualisation , 2004, Neural Networks.

[39]  Adel M. Alimi,et al.  An Iterative Method for Deciding SVM and Single Layer Neural Network Structures , 2011, Neural Processing Letters.

[40]  Ba Tu Truong,et al.  Automatic Scene Extraction in Motion Pictures , 2003 .

[41]  Wallapak Tavanapong,et al.  Shot clustering techniques for story browsing , 2004, IEEE Transactions on Multimedia.

[42]  Zhu Liu,et al.  Integration of audio and visual information for content-based video segmentation , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[43]  R. Brunelli,et al.  A Survey on the Automatic Indexing of Video Data, , 1999, J. Vis. Commun. Image Represent..

[44]  Teuvo Kohonen,et al.  Statistical pattern recognition with neural networks , 1988, Neural Networks.

[45]  Kien A. Hua,et al.  Content-based scene change detection and classification technique using background tracking , 1999, Electronic Imaging.

[46]  Fakhri Karray,et al.  Enhancing the structure and parameters of the centers for BBF Fuzzy Neural Network classifier construction based on data structure , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).