A Viewable Indexing Structure for the Interactive Exploration of Dynamic and Large Image Collections

Thanks to the capturing devices cost reduction and the advent of social networks, the size of image collections is becoming extremely huge. Many works in the literature have addressed the indexing of large image collections for search purposes. However, there is a lack of support for exploratory data mining. One may want to wander around the images and experience serendipity in the exploration process. Thus, effective paradigms not only for organising, but also visualising these image collections become necessary. In this article, we present a study to jointly index and visualise large image collections. The work focuses on satisfying three constraints. First, large image collections, up to million of images, shall be handled. Second, dynamic collections, such as ever-growing collections, shall be processed in an incremental way, without reprocessing the whole collection at each modification. Finally, an intuitive and interactive exploration system shall be provided to the user to allow him to easily mine image collections. To this end, a data partitioning algorithm has been modified and proximity graphs have been used to fit the visualisation purpose. A custom web platform has been implemented to visualise the hierarchical and graph-based hybrid structure. The results of a user evaluation we have conducted show that the exploration of the collections is intuitive and smooth thanks to the proposed structure. Furthermore, the scalability of the proposed indexing method is proved using large public image collections.

[1]  Piotr Indyk,et al.  Similarity Search in High Dimensions via Hashing , 1999, VLDB.

[2]  Bart Thomee,et al.  New trends and ideas in visual concept detection: the MIR flickr retrieval evaluation initiative , 2010, MIR '10.

[3]  Denis Gracanin,et al.  Large Image Collections - Comprehension and Familiarization by Interactive Visual Analysis , 2009, Smart Graphics.

[4]  Jeremiah D. Deng Content-based image collection summarization and comparison using self-organizing maps , 2007, Pattern Recognit..

[5]  Mohamed Abdel-Mottaleb,et al.  Image browsing using hierarchical clustering , 1999, Proceedings IEEE International Symposium on Computers and Communications (Cat. No.PR00250).

[6]  Godfried T. Toussaint,et al.  The relative neighbourhood graph of a finite planar set , 1980, Pattern Recognit..

[7]  Gerald Schaefer,et al.  Visualisation and Browsing of Image Databases , 2011, Multimedia Analysis, Processing and Communications.

[8]  Gerald Schaefer,et al.  Hierarchical Image Database Navigation on a Hue Sphere , 2006, ISVC.

[9]  Mor Naaman,et al.  Generating summaries and visualization for large collections of geo-referenced photographs , 2006, MIR '06.

[10]  John C. Dalton,et al.  Similarity pyramids for browsing and organization of large image databases , 1998, Electronic Imaging.

[11]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[12]  Leonidas J. Guibas,et al.  A metric for distributions with applications to image databases , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[13]  James Abello,et al.  ASK-GraphView: A Large Scale Graph Visualization System , 2006, IEEE Transactions on Visualization and Computer Graphics.

[14]  Stephen G. Kobourov,et al.  Force-Directed Drawing Algorithms , 2013, Handbook of Graph Drawing and Visualization.

[15]  B. S. Manjunath,et al.  Color and texture descriptors , 2001, IEEE Trans. Circuits Syst. Video Technol..

[16]  Chao Wang,et al.  GRAPHIE: graph based histology image explorer , 2015, BMC Bioinformatics.

[17]  Chaomei Chen,et al.  Similarity-Based Image Browsing , 2000 .

[18]  Kerry Rodden,et al.  Does organisation by similarity assist image browsing? , 2001, CHI.

[19]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[20]  R. Sokal,et al.  A New Statistical Approach to Geographic Variation Analysis , 1969 .

[21]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[22]  Aileen R. Buckley GEOGRAPHIC VISUALIZATION , 1998 .

[23]  Roger W. Schvaneveldt,et al.  Pathfinder associative networks: studies in knowledge organization , 1990 .

[24]  Xing Xie,et al.  Effective browsing of web image search results , 2004, MIR '04.

[25]  Benjamin B. Bederson,et al.  PhotoMesa: a zoomable image browser using quantum treemaps and bubblemaps , 2001, UIST '01.

[26]  Jun Ma,et al.  Similarity-based visualization of large image collections , 2015, Inf. Vis..

[27]  Marco Porta,et al.  Browsing large collections of images through unconventional visualization techniques , 2006, AVI '06.

[28]  Colin J. Ihrig JavaScript Object Notation , 2013 .

[29]  Fernando Pereira,et al.  MPEG-7: A Standard for Multimedia Content Description , 2001, Int. J. Image Graph..

[30]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[31]  Gilles Venturini,et al.  An Approximate Proximity Graph Incremental Construction for Large Image Collections Indexing , 2015, ISMIS.

[32]  Anil K. Jain,et al.  Algorithms for Clustering Data , 1988 .

[33]  Yifan Hu,et al.  Multilevel agglomerative edge bundling for visualizing large graphs , 2011, 2011 IEEE Pacific Visualization Symposium.

[34]  Stefan M. Rüger,et al.  NNk Networks for Content-Based Image Retrieval , 2004, ECIR.

[35]  Mark J. Huiskes,et al.  The MIR flickr retrieval evaluation , 2008, MIR '08.

[36]  Donald W. Dearholt,et al.  Proceedings of the Workshop on Proximity Graphs (1st) Held in Las Cruces, New Mexico on 30 November-2 December 1989 , 1991 .

[37]  James Ze Wang,et al.  Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Jorge E. Camargo,et al.  Visualization , Summarization and Exploration of Large Collections of Images : State Of The Art , 2009 .

[39]  Danny Holten,et al.  Hierarchical Edge Bundles: Visualization of Adjacency Relations in Hierarchical Data , 2006, IEEE Transactions on Visualization and Computer Graphics.

[40]  Takayuki Itoh,et al.  CAT: A Hierarchical Image Browser Using a Rectangle Packing Technique , 2008, 2008 12th International Conference Information Visualisation.

[41]  Roderick Urquhart,et al.  Some properties of the planar Euclidean relative neighbourhood graph , 1983, Pattern Recognit. Lett..

[42]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[43]  Hakim Hacid,et al.  An Effective Method for Locally Neighborhood Graphs Updating , 2005, DEXA.

[44]  Tian Zhang,et al.  BIRCH: an efficient data clustering method for very large databases , 1996, SIGMOD '96.