Narrative Collage of Image Collections by Scene Graph Recombination

A narrative collage is an interesting image editing method for summarizing the main theme or storyline behind an image collection. We present a novel method to generate narrative images with plausible semantic scene structures. To achieve this goal, we introduce a layer graph and a scene graph to represent the relative depth order and semantic relationship between image objects, respectively. We first cluster the input image collection to select representative images, and then we extract a group of semantic salient objects from each representative image. Both layer graphs and scene graphs are constructed and combined according to our specific rules for reorganizing the extracted objects in every image. We design an energy model to appropriately locate every object on the final canvas. The experimental results show that our method can produce competitive narrative collage results and that it performs well on a wide range of image collections.

[1]  Julie Dorsey,et al.  Understanding and improving the realism of image composites , 2012, ACM Trans. Graph..

[2]  Seungyong Lee,et al.  Bilateral texture filtering , 2014, ACM Trans. Graph..

[3]  Tao Mei,et al.  Video collage: presenting a video sequence using a single image , 2008, The Visual Computer.

[4]  Michael F. Cohen,et al.  Digital photography with flash and no-flash image pairs , 2004, ACM Trans. Graph..

[5]  Delbert Dueck,et al.  Clustering by Passing Messages Between Data Points , 2007, Science.

[6]  Alexei A. Efros,et al.  Scene completion using millions of photographs , 2008, Commun. ACM.

[7]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[8]  Steven M. Seitz,et al.  Scene Summarization for Online Image Collections , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[9]  Stephen Lin,et al.  Semantic colorization with internet images , 2011, ACM Trans. Graph..

[10]  Antonio Torralba,et al.  LabelMe: A Database and Web-Based Tool for Image Annotation , 2008, International Journal of Computer Vision.

[11]  Ralph R. Martin,et al.  Internet visual media processing: a survey with graphics and vision applications , 2013, The Visual Computer.

[12]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[13]  Lin Lu,et al.  Content-Aware Photo Collage Using Circle Packing , 2014, IEEE Transactions on Visualization and Computer Graphics.

[14]  Meng Liu,et al.  Efficient Mean‐shift Clustering Using Gaussian KD‐Tree , 2010, Comput. Graph. Forum.

[15]  Sanja Fidler,et al.  The Role of Context for Object Detection and Semantic Segmentation in the Wild , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Nanning Zheng,et al.  Learning to Detect a Salient Object , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Hua Huang,et al.  Arcimboldo-like collage using internet images , 2011, ACM Trans. Graph..

[18]  Dani Lischinski,et al.  A Closed-Form Solution to Natural Image Matting , 2008 .

[19]  Adam Finkelstein,et al.  Video tapestries with continuous temporal zoom , 2010, SIGGRAPH 2010.

[20]  Andrew Blake,et al.  Digital tapestry [automatic image synthesis] , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[21]  Michelle R. Greene,et al.  PSYCHOLOGICAL SCIENCE Research Article The Briefest of Glances The Time Course of Natural Scene Understanding , 2022 .

[22]  Zhuowen Tu,et al.  Deeply Supervised Salient Object Detection with Short Connections , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Shi-Min Hu,et al.  Visual storylines: Semantic visualization of movie sequence , 2012, Comput. Graph..

[24]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Ilan Shimshoni,et al.  Mean shift based clustering in high dimensions: a texture classification example , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[26]  Ce Liu,et al.  Scene Collaging: Analysis and Synthesis of Natural Images with Semantic Layers , 2013, 2013 IEEE International Conference on Computer Vision.

[27]  Lihi Zelnik-Manor,et al.  Puzzle‐like Collage , 2010, Comput. Graph. Forum.

[28]  Shi-Min Hu,et al.  Sketch2Photo: internet image montage , 2009, ACM Trans. Graph..

[29]  Hua Huang,et al.  Hierarchical Narrative Collage For Digital Photo Album , 2012, Comput. Graph. Forum.

[30]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[31]  Kwan-Liu Ma,et al.  Dynamic video narratives , 2010, SIGGRAPH 2010.