Extraction of Film Takes for Cinematic Analysis

In this paper, we focus on the ‘reverse editing’ problem in movie analysis, i.e., the extraction of film takes, original camera shots that a film editor extracts and arranges to produce a finished scene. The ability to disassemble final scenes and shots into takes is essential for nonlinear browsing, content annotation and the extraction of higher order cinematic constructs from film. A two-part framework for take extraction is proposed. The first part focuses on the filtering out action-driven scenes for which take extraction is not useful. The second part focuses on extracting film takes using agglomerative hierarchical clustering methods along with different similarity metrics and group distances and demonstrates our findings with 10 movies.

[1]  Andreas Girgensohn,et al.  Time-Constrained Keyframe Selection Technique , 2004, Multimedia Tools and Applications.

[2]  Stefan Sharff.  The Elements of Cinema: Toward a Theory of Cinesthetic Impact , 1982 .

[3]  Frank Eugene Beaver.  Dictionary of film terms : the aesthetic companion to film analysis , 1994 .

[4]  Ying Li,et al.  An Overview of Video Abstraction Techniques , 2001 .

[5]  Jing Huang,et al.  Spatial Color Indexing and Applications , 2004, International Journal of Computer Vision.

[6]  H. Zettl.  Sight, Sound, Motion: Applied Media Aesthetics , 1973 .

[7]  Chong-Wah Ngo,et al.  On clustering and retrieval of video shots , 2001, MULTIMEDIA '01.

[8]  Li Zhao,et al.  Video shot grouping using best-first model merging , 2001, IS&T/SPIE Electronic Imaging.

[9]  Stefanos D. Kollias,et al.  Video content representation using optimal extraction of frames and scenes , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[10]  Ba Tu Truong,et al.  New enhancements to cut, fade, and dissolve detection processes in video segmentation , 2000, ACM Multimedia.

[11]  Svetha Venkatesh,et al.  Toward automatic extraction of expressive elements from motion pictures: tempo , 2002, IEEE Trans. Multim..

[12]  Nevenka Dimitrova,et al.  Color superhistograms for video representation , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[13]  Patrick Bouthemy,et al.  From video shot clustering to sequence segmentation , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[14]  Alberto Del Bimbo,et al.  Structured representation and automatic indexing of movie information content , 1998, Pattern Recognit..

[15]  Rainer Lienhart,et al.  Reliable Transition Detection in Videos: A Survey and Practitioner's Guide , 2001, Int. J. Image Graph..

[16]  Ba Tu Truong,et al.  Application of computational media aesthetics methodology to extracting color semantics in film , 2002, MULTIMEDIA '02.

[17]  Wolfgang Effelsberg,et al.  Robust clustering-based video-summarization with integration of domain-knowledge , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[18]  Mark S. Drew,et al.  Video keyframe production by efficient clustering of compressed chromaticity signatures (poster session) , 2000, ACM Multimedia.

[19]  Ba Tu Truong,et al.  Scene extraction in motion pictures , 2003, IEEE Trans. Circuits Syst. Video Technol..

[20]  Jeho Nam,et al.  Dynamic video summarization and visualization , 1999, MULTIMEDIA '99.

[21]  B. S. Everitt,et al.  Cluster analysis , 2014, Encyclopedia of Social Network Analysis and Mining.

[22]  G. W. Milligan,et al.  An examination of procedures for determining the number of clusters in a data set , 1985 .

[23]  Tat-Seng Chua,et al.  A cinematic-based framework for scene boundary detection in video , 2003, The Visual Computer.

[24]  Shih-Fu Chang,et al.  Clustering methods for video browsing and annotation , 1996, Electronic Imaging.

[25]  Yueting Zhuang,et al.  Adaptive key frame extraction using unsupervised clustering , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[26]  Tsuyoshi Moriyama,et al.  Video summarisation based on the psychological content in the track structure , 2000, MULTIMEDIA '00.

[27]  A. Murat Tekalp,et al.  Automatic Soccer Video Analysis and Summarization , 2003, IS&T/SPIE Electronic Imaging.

[28]  Shigenobu Kobayashi.  Colorist: A Practical Handbook for Personal and Professional Use , 1999 .

[29]  Ba Tu Truong,et al.  Discovering semantics from visualizations of film takes , 2004, 10th International Multimedia Modelling Conference, 2004. Proceedings..

[30]  Boon-Lock Yeo,et al.  Segmentation of Video by Clustering and Graph Analysis , 1998, Comput. Vis. Image Underst..

[31]  Thomas S. Huang,et al.  Constructing table-of-content for videos , 1999, Multimedia Systems.

[32]  Ba Tu Truong,et al.  Automatic Scene Extraction in Motion Pictures , 2003 .