Dynamic Manga: Animating Still Manga via Camera Movement

We propose a method for animating still manga imagery through camera movements. Given a series of existing manga pages, we start by automatically extracting panels, comic characters, and balloons from the manga pages. Then, we use a data-driven graphical model to infer per-panel motion and emotion states from low-level visual patterns. Finally, by combining domain knowledge of film production and characteristics of manga, we simulate camera movements over the manga pages, yielding an animation. The results augment the still manga contents with animated motion that reveals the mood and tension of the story, while maintaining the original narrative. We have tested our method on manga series of different genres, and demonstrated that our method can generate animations that are more effective in storytelling and pacing, with less human efforts, as compared with prior works. We also show two applications of our method, mobile comic reading, and comic trailer generation.

[1]  Zhi Tang,et al.  Unsupervised Speech Text Localization in Comic Images , 2013, 2013 12th International Conference on Document Analysis and Recognition.

[2]  Yaser Sheikh,et al.  Attention-guided Algorithms to Retarget and Augment Animations, Stills, and Videos , 2012 .

[3]  Andrew Chi-Sing Leung,et al.  Animating animal motion from still , 2008, SIGGRAPH Asia '08.

[4]  Edward H. Adelson,et al.  Motion without movement , 1991, SIGGRAPH.

[5]  Ira Kemelmacher-Shlizerman,et al.  Exploring photobios , 2011, SIGGRAPH 2011.

[6]  Taku Komura,et al.  Automatic Panel Extraction of Color Comic Images , 2007, PCM.

[7]  Fang Liu,et al.  Stroke extraction in cartoon images using edge-enhanced isotropic nonlinear filter , 2010, VRCAI '10.

[8]  Rynson W. H. Lau,et al.  A Robust Panel Extraction Method for Manga , 2014, ACM Multimedia.

[9]  Rynson W. H. Lau,et al.  Look over here , 2014, ACM Trans. Graph..

[10]  E. Kensinger,et al.  Remembering Emotional Experiences: The Contribution of Valence and Arousal , 2004, Reviews in the neurosciences.

[11]  Lance Williams,et al.  Animating images with drawings , 1994, SIGGRAPH.

[12]  Ken Tsutsuguchi,et al.  Dynamic texture: physically based 2D animation , 1999, SIGGRAPH '99.

[13]  Kohei Arai,et al.  Method for Automatic E-Comic Scene Frame Extraction for Reading Comic on Mobile Devices , 2010, 2010 Seventh International Conference on Information Technology: New Generations.

[14]  Rynson W. H. Lau,et al.  Automatic stylistic manga layout , 2012, ACM Trans. Graph..

[15]  Li Xu,et al.  Structure extraction from texture via relative total variation , 2012, ACM Trans. Graph..

[16]  David Salesin,et al.  Animating pictures with stochastic motion textures , 2005, SIGGRAPH 2005.

[17]  Jr. G. Forney,et al.  The viterbi algorithm , 1973 .

[18]  Leif Kobbelt,et al.  Character animation from 2D pictures and 3D motion data , 2007, TOGS.

[19]  A. Lewis Making Comics: Storytelling Secrets of Comics, Manga and Graphic Novels , 2007 .

[20]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[21]  Meng Sun,et al.  Video input driven animation (VIDA) , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[22]  Ralph R. Martin,et al.  Vectorizing Cartoon Animations , 2009, IEEE Transactions on Visualization and Computer Graphics.

[23]  Jitendra Malik,et al.  Textons, contours and regions: cue integration in image segmentation , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[24]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[25]  Ira Kemelmacher-Shlizerman,et al.  Exploring photobios , 2011, ACM Trans. Graph..

[26]  Jitendra Malik,et al.  Scale-invariant contour completion using conditional random fields , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[27]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..