Movie2Comics: Towards a Lively Video Content Presentation

As a type of artwork, comics is prevalent and popular around the world. However, despite the availability of assistive software and tools, the creation of comics is still a labor-intensive and time-consuming process. This paper proposes a scheme that is able to automatically turn a movie clip to comics. Two principles are followed in the scheme: 1) optimizing the information preservation of the movie; and 2) generating outputs following the rules and the styles of comics. The scheme mainly contains three components: script-face mapping, descriptive picture extraction, and cartoonization. The script-face mapping utilizes face tracking and recognition techniques to accomplish the mapping between characters' faces and their scripts. The descriptive picture extraction then generates a sequence of frames for presentation. Finally, the cartoonization is accomplished via three steps: panel scaling, stylization, and comics layout design. Experiments are conducted on a set of movie clips and the results have demonstrated the usefulness and the effectiveness of the scheme.

[1]  Carlos D. Correa,et al.  Dynamic video narratives , 2010, ACM Trans. Graph..

[2]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Hwan-Gue Cho,et al.  Cinema comics: Cartoon generation from video stream , 2006, GRAPP.

[4]  Holger Winnemöller,et al.  Real-time video abstraction , 2006, SIGGRAPH 2006.

[5]  Tao Mei,et al.  Video collage: presenting a video sequence using a single image , 2008, The Visual Computer.

[6]  Andrew Zisserman,et al.  Hello! My name is... Buffy'' -- Automatic Naming of Characters in TV Video , 2006, BMVC.

[7]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[8]  Patrick Bouthemy,et al.  A unified approach to shot change detection and camera motion characterization , 1999, IEEE Trans. Circuits Syst. Video Technol..

[9]  Meng Wang,et al.  Beyond Distance Measurement: Constructing Neighborhood Similarity for Video Annotation , 2009, IEEE Transactions on Multimedia.

[10]  Stephen R. Gulliver,et al.  Introduction to special issue on eye-tracking applications in multimedia systems , 2007, TOMCCAP.

[11]  Quan Pan,et al.  Real-time multiple objects tracking with occlusion handling in dynamic scenes , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[12]  Atreyi Kankanhalli,et al.  Automatic partitioning of full-motion video , 1993, Multimedia Systems.

[13]  Meng Wang,et al.  Semi-supervised kernel density estimation for video annotation , 2009, Comput. Vis. Image Underst..

[14]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  David Salesin,et al.  Comic Chat , 1996, SIGGRAPH.

[16]  Meng Wang,et al.  Movie2Comics: a feast of multimedia artwork , 2010, ACM Multimedia.

[17]  Holger Winnemöller,et al.  Real-time video abstraction , 2006, ACM Trans. Graph..

[18]  Andrew Zisserman,et al.  Automatic face recognition for film character retrieval in feature-length films , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[19]  Jenq-Neng Hwang,et al.  Object-based video abstraction for video surveillance systems , 2002, IEEE Trans. Circuits Syst. Video Technol..

[20]  Irfan A. Essa,et al.  Video-based nonphotorealistic and expressive illustration of motion , 2005, International 2005 Computer Graphics.

[21]  Andreas Girgensohn,et al.  Stained-glass visualization for highly condensed video summaries , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[22]  Harry Shum,et al.  Video tooning , 2004, ACM Trans. Graph..

[23]  Shingo Uchihashi,et al.  An interactive comic book presentation for exploring video , 2000, CHI.

[24]  Michal Irani,et al.  Video indexing based on mosaic representations , 1998, Proc. IEEE.

[25]  Jacob Scharcanski,et al.  Hierarchical Summarization of Diagnostic Hysteroscopy Videos , 2006, 2006 International Conference on Image Processing.

[26]  Shingo Uchihashi,et al.  Video Manga: generating semantically meaningful video summaries , 1999, MULTIMEDIA '99.

[27]  Leonard McMillan,et al.  Computational time-lapse video , 2007, SIGGRAPH '07.

[28]  Andrea Fusiello,et al.  High resolution video mosaicing with global alignment , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[29]  Jung-Hwan Oh,et al.  Scenario based dynamic video abstractions using graph matching , 2005, MULTIMEDIA '05.

[30]  David Salesin,et al.  Keyframe-based tracking for rotoscoping and animation , 2004, ACM Trans. Graph..

[31]  Hyung-Myung Kim,et al.  Efficient camera motion characterization for MPEG video indexing , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[32]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[33]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[34]  Ariel Shamir,et al.  Generating comics from 3D interactive computer graphics , 2006, IEEE Computer Graphics and Applications.

[35]  Janko Calic,et al.  Efficient Layout of Comic-Like Video Summaries , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[36]  Hwan-Gue Cho,et al.  An Automated Procedure for Word Balloon Placement in Cinema Comics , 2006, ISVC.

[37]  Anne Magnussen,et al.  Comics and Culture: Analytical and Theoretical Approaches to Comics , 2000 .

[38]  Meng Wang,et al.  Unified Video Annotation via Multigraph Learning , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[39]  Ben Taskar,et al.  Joint covariate selection and joint subspace selection for multiple classification problems , 2010, Stat. Comput..

[40]  Ba Tu Truong,et al.  Video abstraction: A systematic review and classification , 2007, TOMCCAP.