Tree-Based Visualization and Optimization for Image Collection

The visualization of an image collection is the process of displaying a collection of images on a screen under some specific layout requirements. This paper focuses on an important problem that is not well addressed by the previous methods: visualizing image collections into arbitrary layout shapes while arranging images according to user-defined semantic or visual correlations (e.g., color or object category). To this end, we first propose a property-based tree construction scheme to organize images of a collection into a tree structure according to user-defined properties. In this way, images can be adaptively placed with the desired semantic or visual correlations in the final visualization layout. Then, we design a two-step visualization optimization scheme to further optimize image layouts. As a result, multiple layout effects including layout shape and image overlap ratio can be effectively controlled to guarantee a satisfactory visualization. Finally, we also propose a tree-transfer scheme such that visualization layouts can be adaptively changed when users select different “images of interest.” We demonstrate the effectiveness of our proposed approach through the comparisons with state-of-the-art visualization techniques.

[1]  T. J. Jankun-Kelly,et al.  MoireGraphs: radial focus+context visualization and interaction for graphs with visual nodes , 2003, IEEE Symposium on Information Visualization 2003 (IEEE Cat. No.03TH8714).

[2]  Gerald Schaefer,et al.  A next generation browsing environment for large image repositories , 2010, Multimedia Tools and Applications.

[3]  Craig S. Kaplan,et al.  Cut-out image mosaics , 2008, NPAR.

[4]  Klaus Schöffmann,et al.  Similarity-Based Visualization for Image Browsing Revisited , 2011, 2011 IEEE International Symposium on Multimedia.

[5]  Yong Wang,et al.  Coherent image annotation by learning semantic distance , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Benjamin B. Bederson,et al.  Quantum Treemaps and Bubblemaps for a Zoomable Image Browser , 2001 .

[7]  Ramana Rao,et al.  The Hyperbolic Browser: A Focus + Context Technique for Visualizing Large Hierarchies , 1996, J. Vis. Lang. Comput..

[8]  Helge J. Ritter,et al.  On interactive visualization of high-dimensional data using the hyperbolic plane , 2002, KDD.

[9]  Harry Shum,et al.  Picture Collage , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[10]  Jiwen Lu,et al.  Regularized Locality Preserving Projections and Its Extensions for Face Recognition , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[11]  Xuelong Li,et al.  Hessian Regularized Support Vector Machines for Mobile Image Annotation on the Cloud , 2013, IEEE Transactions on Multimedia.

[12]  D. Kerby The Simple Difference Formula: An Approach to Teaching Nonparametric Correlation1: , 2014 .

[13]  Hua Huang,et al.  Arcimboldo-like collage using internet images , 2011, ACM Trans. Graph..

[14]  Xuelong Li,et al.  Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Xing Xie,et al.  Effective browsing of web image search results , 2004, MIR '04.

[16]  Pavel Berkhin,et al.  A Survey on PageRank Computing , 2005, Internet Math..

[17]  Marcel Worring,et al.  Interactive access to large image collections using similarity-based visualization , 2008, J. Vis. Lang. Comput..

[18]  Sungdae Cho,et al.  Mobile photo collage , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[19]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[20]  Lihi Zelnik-Manor,et al.  Puzzle‐like Collage , 2010, Comput. Graph. Forum.

[21]  Xuelong Li,et al.  Image Annotation by Multiple-Instance Learning With Discriminative Feature Mapping and Selection , 2014, IEEE Transactions on Cybernetics.

[22]  R. Keys Cubic convolution interpolation for digital image processing , 1981 .

[23]  Tommy W. S. Chow,et al.  M-Isomap: Orthogonal Constrained Marginal Isomap for Nonlinear Dimensionality Reduction , 2013, IEEE Transactions on Cybernetics.

[24]  Weisi Lin,et al.  Integrating visual saliency and consistency for re-ranking image search results , 2011, 2010 IEEE International Conference on Image Processing.

[25]  Yuan Yan Tang,et al.  Social Image Tagging With Diverse Semantics , 2014, IEEE Transactions on Cybernetics.

[26]  Klaus Schöffmann,et al.  3-D Interfaces to Improve the Performance of Visual Known-Item Search , 2014, IEEE Transactions on Multimedia.

[27]  Elias Salomão Helou Neto,et al.  Similarity Preserving Snippet-Based Visualization of Web Search Results , 2014, IEEE Transactions on Visualization and Computer Graphics.

[28]  Luis Gustavo Nonato,et al.  Semantically Aware Dynamic Layouts , 2014, 2014 27th SIBGRAPI Conference on Graphics, Patterns and Images.

[29]  J. Neyman Outline of a Theory of Statistical Estimation Based on the Classical Theory of Probability , 1937 .

[30]  Tao Mei,et al.  Video collage: presenting a video sequence using a single image , 2008, The Visual Computer.

[31]  Chunxiao Liu,et al.  An improved belief propagation method for dynamic collage , 2009, The Visual Computer.

[32]  Hans-Peter Seidel,et al.  Interactive by-example design of artistic packing layouts , 2013, ACM Trans. Graph..

[33]  Yangqiu Song,et al.  ImageHive: Interactive Content-Aware Image Summarization , 2012, IEEE Computer Graphics and Applications.

[34]  Kerry Rodden,et al.  Does organisation by similarity assist image browsing? , 2001, CHI.

[35]  Jakub Lokoc,et al.  Towards efficient multimedia exploration using the metric space approach , 2014, 2014 12th International Workshop on Content-Based Multimedia Indexing (CBMI).

[36]  Cordelia Schmid,et al.  Local Features and Kernels for Classification of Texture and Object Categories: A Comprehensive Study , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[37]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[38]  Qi Tian,et al.  Visualization and User-Modeling for Browsing Personal Photo Libraries , 2004, International Journal of Computer Vision.

[39]  Gerald Schaefer,et al.  Visualisation and Browsing of Image Databases , 2011, Multimedia Analysis, Processing and Communications.

[40]  Jianping Fan,et al.  An Interactive Approach for Filtering Out Junk Images From Keyword-Based Google Search Results , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[41]  Daniel Heesch,et al.  A survey of browsing models for content based image retrieval , 2008, Multimedia Tools and Applications.

[42]  Nikolaos G. Bourbakis,et al.  A 3-D visualization method for image-guided brain surgery , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[43]  Daming Shi,et al.  TPSLVM: A Dimensionality Reduction Algorithm Based On Thin Plate Splines , 2014, IEEE Transactions on Cybernetics.

[44]  Minglun Gong,et al.  Self-Sorting Map: An Efficient Algorithm for Presenting Multimedia Data in Structured Layouts , 2014, IEEE Transactions on Multimedia.

[45]  Takayuki Itoh,et al.  CAT: A Hierarchical Image Browser Using a Rectangle Packing Technique , 2008, 2008 12th International Conference Information Visualisation.