Movie2Comics: a feast of multimedia artwork

As a type of artwork, comics are prevalent and popular around the world. However, although there are several assistive software and tools available, the creation of comics is still a tedious and labor intensive process. This paper proposes a scheme that is able to automatically turn a movie to comics with two principles: (1) optimizing the information reservation of movie; and (2) generating outputs following the rules and styles of comics. The scheme mainly contains three components: script-face mapping, key-scene extraction, and cartoonization. Script-face mapping utilizes face recognition and tracking techniques to accomplish the mapping between character's faces and their scripts. Key-scene extraction then combines the frames derived from subshots and the extracted index frames based on subtitle to select a sequence of frames for cartoonization. Finally, the cartoonization is accomplished via four steps: panel scale, stylization, word balloon placement and comics layout. Experiments conducted on a set of movie clips have demonstrates the usefulness and e®ectiveness of the scheme.

[1]  Holger Winnemöller,et al.  Real-time video abstraction , 2006, ACM Trans. Graph..

[2]  Tat-Seng Chua,et al.  Mediapedia: Mining Web Knowledge to Construct Multimedia Encyclopedia , 2010, MMM.

[3]  Ben Taskar,et al.  Joint covariate selection and joint subspace selection for multiple classification problems , 2010, Stat. Comput..

[4]  Yue Gao,et al.  Clip based video summarization and ranking , 2008, CIVR '08.

[5]  Hyung-Myung Kim,et al.  Efficient camera motion characterization for MPEG video indexing , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[6]  Tao Mei,et al.  Near-lossless video summarization , 2009, MM '09.

[7]  Andrew Zisserman,et al.  Automatic face recognition for film character retrieval in feature-length films , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[8]  Trevor Darrell,et al.  Visual speech recognition with loosely synchronized feature streams , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[9]  Jenq-Neng Hwang,et al.  Object-based video abstraction for video surveillance systems , 2002, IEEE Trans. Circuits Syst. Video Technol..

[10]  Holger Winnemöller,et al.  Real-time video abstraction , 2006, SIGGRAPH 2006.

[11]  Andrew Zisserman,et al.  Hello! My name is... Buffy'' -- Automatic Naming of Characters in TV Video , 2006, BMVC.

[12]  Jörn Loviscach,et al.  From movie to comic, informed by the screenplay , 2007, SIGGRAPH '07.

[13]  Hwan-Gue Cho,et al.  Cinema comics: Cartoon generation from video stream , 2006, GRAPP.

[14]  Ariel Shamir,et al.  Generating comics from 3D interactive computer graphics , 2006, IEEE Computer Graphics and Applications.

[15]  Hwan-Gue Cho,et al.  An Automated Procedure for Word Balloon Placement in Cinema Comics , 2006, ISVC.

[16]  Michal Irani,et al.  Video indexing based on mosaic representations , 1998, Proc. IEEE.

[17]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.